Alex Auvolat
390a5d97fe
nix, ci: build with Crane
...
This removes our dependency on cargo2nix, which was causing us some
issues. Whereas cargo2nix creates one Nix derivation per crate, Crane
uses only two derivations:
1. Build dependencies only
2. Build the final binary
This means that during the second step, no caching can be done. For
instance, if we do a change in garage_model, we need to recompile all of
the Garage crates including those that do not depend on garage_model.
On the upside, this allows all of the Garage crates to be built at once
using cargo build logic, which is optimized for high parallelism and
better pipelining between all of the steps of the build. All in all,
this makes most builds faster than cargo2nix.
A few other changes have been made to the build scripts and CI:
- Unit tests are now run within a Nix derivation. In fact, we have
different derivations to run the tests using LMDB and Sqlite as
metadata db engines.
- For debug builds, most CI steps now run in parallel (with the notable
exception of the smoke test that runs after the build, which is
inevitable).
- We no longer pass the GIT_VERSION argument when building debug builds
and running the tests. This means that dev binaries and test
binaries don't know the exact version of Garage they are from. That
shouldn't be an issue in most cases.
- The not-dynamic.sh scripts has been fixed to fail if the file does not
exist.
2025-02-03 16:39:50 +01:00
Alex Auvolat
4563313f87
use cargo-shear to remove many unused dependencies between crates
2025-01-31 18:47:30 +01:00
Stefan Majer
2eb9fcae20
Fix all typos
2025-01-16 13:22:00 +01:00
Alex Auvolat
295237476e
fix formatting to comply with latest rustfmt
2025-01-12 17:36:25 +01:00
Alex Auvolat
7a143f46fc
Bump to version 1.0.1
2024-09-22 14:25:32 +02:00
Alex Auvolat
afad62939e
[next-0.10] bump version number to 1.0
2024-03-28 15:19:44 +01:00
Alex Auvolat
8bfc16ba7d
Merge branch 'main' into next-0.10
2024-03-28 15:01:05 +01:00
Alex Auvolat
85f580cbde
[fix-buffering] change request sending strategy and fix priorities
...
remove LAS, priorize new requests but otherwise just do standard queuing
2024-03-27 16:22:40 +01:00
Alex Auvolat
0d3e285d13
[fix-buffering] implement block_ram_buffer_max to avoid excessive RAM usage
2024-03-27 16:22:40 +01:00
Alex Auvolat
f7cd4eb600
Merge branch 'main' into next-0.10
2024-03-26 16:34:40 +01:00
Alex Auvolat
ba33bb31f1
[check-data-dir] add marker files in data directories ( fix #601 )
2024-03-20 15:20:25 +01:00
Alex Auvolat
3165ab926c
[block-ref-repair] rename rc's rc field to rc_table
2024-03-19 16:20:22 +01:00
Alex Auvolat
dc0b78cdb8
[block-ref-repair] Block refcount recalculation and repair
...
- We always recalculate the reference count of a block before deleting
it locally, to make sure that it is indeed zero.
- If we had to fetch a remote block but we were not able to get it,
check that refcount is indeed > 0.
- Repair procedure that checks everything
2024-03-19 16:20:22 +01:00
Alex Auvolat
0038ca8a78
Merge branch 'main' into next-0.10
2024-03-18 20:19:30 +01:00
Alex Auvolat
7c86ff6c37
[disable-scrub] implement a disable_scrub configuration option
2024-03-14 17:01:16 +01:00
Alex Auvolat
05c92204ec
[rm-sled] Remove counted_tree_hack
2024-03-08 15:09:57 +01:00
Alex Auvolat
44454aac01
[rm-sled] Remove the Sled database engine
2024-03-08 14:11:02 +01:00
Alex Auvolat
57acc60082
[sse-c] Implement SSE-C encryption
2024-03-07 15:43:47 +01:00
Alex Auvolat
8670140358
[rel-0.9.3] Bump version to 0.9.3
2024-03-04 14:00:55 +01:00
Alex Auvolat
6a7623e90d
[rel-0.9.2] Bump version to v0.9.2
2024-03-01 16:54:39 +01:00
Alex Auvolat
4b978b7533
Merge branch 'main' into next-0.10
2024-02-26 18:55:24 +01:00
Alex Auvolat
b76c0c102e
[refactor-put] add ordering tag to blocks being sent to storage nodes
2024-02-26 18:35:11 +01:00
Alex Auvolat
916c67ccf4
Merge branch 'main' into next-0.10
2024-02-23 16:50:34 +01:00
Alex Auvolat
6ee691e65f
[refactor-block] simplify some more
2024-02-23 12:50:10 +01:00
Alex Auvolat
e9c42bca34
[refactor-block] add DataBlockStream type
2024-02-23 12:22:29 +01:00
Alex Auvolat
cd1069c1d4
[refactor-block] refactor DataBlock and DataBlockPath
2024-02-23 12:15:52 +01:00
Alex Auvolat
07c7895948
[refactor-block] simplify rpc_get_block
2024-02-23 11:54:40 +01:00
Alex Auvolat
9b41f4ff20
[refactor-block] move read_stream_to_end to garage_net
2024-02-23 11:46:57 +01:00
Alex Auvolat
93552b9275
[refactor-block] Remove redundant BlockStream type
2024-02-23 11:33:38 +01:00
Alex Auvolat
eb4a6ce106
Merge branch 'main' into next-0.10
2024-02-15 14:06:34 +01:00
Alex Auvolat
5ea24254a9
[import-netapp] import Netapp code into Garage codebase
2024-02-15 12:15:07 +01:00
Alex Auvolat
cf2af186fc
Merge branch 'main' into next-0.10
2024-02-13 11:36:28 +01:00
Alex Auvolat
22332e6c35
[dep-upgrade-202402] simplify/refactor GetObject
2024-02-05 20:26:33 +01:00
Alex Auvolat
fe1af5d98b
[dep-upgrade-202402] refactor dependencies: move all as workspace deps
2024-02-05 13:02:02 +01:00
Alex Auvolat
ee57dd922b
Bump version to 0.9.1
2024-01-16 16:28:17 +01:00
Alex Auvolat
db48dd3d6c
bump crate versions to 0.10.0
2024-01-11 12:05:51 +01:00
Alex Auvolat
5dd200c015
layout: move block_read_nodes_of to rpc_helper to avoid double-locking
...
(in theory, this could have caused a deadlock)
2023-12-08 12:02:24 +01:00
Alex Auvolat
d6d239fc79
block manager: read_block using old layout versions if necessary
2023-11-27 11:52:57 +01:00
Alex Auvolat
33c8a489b0
layou: implement ack locking
2023-11-15 15:40:44 +01:00
Alex Auvolat
90e1619b1e
table: take into account multiple write sets in inserts
2023-11-14 15:40:46 +01:00
Alex Auvolat
3b361d2959
layout: prepare for write sets
2023-11-14 14:28:16 +01:00
Alex Auvolat
8a2b1dd422
wip: split out layout management from System into separate LayoutManager
2023-11-09 12:55:36 +01:00
Alex Auvolat
952c9570c4
bump version to v0.9.0
2023-10-10 14:08:11 +02:00
Alex Auvolat
f97168f805
garage_db: refactor transactions and add on_commit mechanism
2023-09-21 15:35:31 +02:00
Alex Auvolat
ad6b1cc0be
Merge branch 'main' into next
2023-09-11 13:14:18 +02:00
Alex Auvolat
ba7ac52c19
block repair: simpler/more robust iterator progress calculation
2023-09-11 12:31:34 +02:00
Alex Auvolat
9526328d38
scrub: clear saved checkpoint when canceling scrub
2023-09-11 12:10:48 +02:00
Alex Auvolat
7f9ba49c71
block manager: remove data_dir field
2023-09-11 11:57:36 +02:00
Alex Auvolat
de5d792181
block manager: fix indentation (why not detected by cargo fmt?)
2023-09-11 11:52:57 +02:00
Alex Auvolat
be91ef6294
block manager: fix bug where rebalance didn't delete old copies
2023-09-07 16:04:03 +02:00
