Conversation
Adds a backend-neutral API for compare-and-swap deletes:
delete object
ONLY IF
current object version == expected version
Conditional delete provides optimistic concurrency control. A backend that
supports the supplied precondition performs the check atomically with the
deletion, as part of a single conditional request, and must not emulate it
with a separate metadata read followed by an unconditional delete.
- `DeleteOptions::precondition: Option<UpdateVersion>` reuses the existing
version identity model (`e_tag` + `version`) rather than introducing a
second one, and a `From<ObjectMeta>` impl is added for ergonomics
- `ObjectStore::delete_opts` has a default implementation returning
`NotSupported` for preconditions, so existing implementations keep
compiling; `ObjectStoreExt::delete` now delegates to it with default
options, preserving existing behavior (including S3's bulk `DeleteObjects`)
- GCS uses `x-goog-if-generation-match` (generation), S3 and Azure use
`If-Match` (ETag); `InMemory` checks and removes under the same lock
- Backends that cannot evaluate the precondition report `NotSupported` /
`NotImplemented`, so callers can fall back to an unconditional delete
- `ObjectStore::delete_opts` is forwarded through the `Arc`/`Box` and
wrapper (`ChunkedStore`, `LimitStore`, `PrefixStore`, `ThrottledStore`)
implementations
- Covered by trait-level integration tests and by request-shape unit tests
for GCS/S3/Azure; verified against Azurite (Azure) and LocalStack (S3
reports `501` and is skipped)
Addresses apache#298
|
Context for reviewers — this supersedes my earlier #862 (same branch, same diff, Design-wise it matches the direction already discussed on #610 and in
Happy to fold any of this into #610 or to align naming — just say which. |
feat: add conditional deletes (
ObjectStore::delete_opts)Addresses #298.
Summary
Adds a backend-neutral API for compare-and-swap deletes: an object is deleted
only if its current version matches the version the caller last observed.
Conditional delete provides optimistic concurrency control. A backend that
supports the supplied precondition performs the check atomically with the
deletion, as part of a single conditional request. Implementations must not
emulate it with a separate metadata read followed by an unconditional delete —
that sequence has a race between the two requests.
API
Usage:
UpdateVersion(the existinge_tag+versionidentity, already used byPutMode::Update) is reused rather than introducing a second version model; eachbackend picks the field its native mechanism needs.
From<ObjectMeta> for UpdateVersionis added for ergonomics.delete/delete_opts/delete_streamdelete_optsis a provided trait method, so existing implementations keepcompiling. Its default returns
Error::NotSupportedwhen a precondition issupplied, and otherwise behaves exactly like today's single-object delete
(drives
delete_streamwith one location).ObjectStoreExt::deleteis nowdelete_opts(location, DeleteOptions::default()),so unconditionally deleting is unchanged for every backend, including the S3
bulk
DeleteObjectspath.delete_streamis deliberately unchanged: it is a bulk API, whereasconditional delete is per-object. Bulk conditional deletes are out of scope
here and can be added later without breaking this design.
Backends
x-goog-if-generation-matchwithObjectMeta::version(generation)If-Matchwith the ETag, reusing the existingS3ConditionalPutgateIf-Matchwith the ETag onDelete BlobNotSupported; unconditional delete unchangedNotImplementedErrors reuse the existing taxonomy: a failed precondition surfaces as
Error::Precondition(the status mapping for412already exists), so callerscan distinguish "object does not exist" (
NotFound) from "object changed"(
Precondition). Backends that cannot evaluate the precondition returnNotSupported/NotImplementedinstead of silently ignoring it.For S3, some S3-compatible endpoints do not implement
If-MatchonDeleteObjectand answer501 Not Implemented; that is surfaced asError::NotSupportedso callers can fall back to an unconditional delete, ratherthan reporting an opaque transport error.
S3ConditionalPut::Disabledalsodisables conditional deletes, consistent with conditional puts.
Performance
No additional round trips.
delete(path)remains one request (bulkDeleteObjectson S3, since the existing single-object behavior is preserved),and a conditional delete is a single conditional
DELETE, neverHEAD+DELETE.Compatibility
Additive: new types plus a defaulted trait method, no signature changes, no
behavior change for existing paths. This makes it suitable for a minor release
(
0.14.x) rather than requiring0.15.0.One practical consequence: wrapper stores that use
#[deny(clippy::missing_trait_methods)]— the pattern recommended in thiscrate's own docs — must forward
delete_optsexplicitly once the default exists.This PR does that for
ChunkedStore,LimitStore,PrefixStoreandThrottledStore.Testing
integration::conditional_delete): unconditionaldelete, correct-version delete, stale-version delete returning
Preconditionwith the newer object and its content untouched, and a
NotSupported/NotImplementedpath that asserts the object is left alone while ordinary deletes keep working.
Wired into the in-memory, local filesystem,
Box/Arc, S3, Azure and GCS suites.exactly one
DELETEcarrying the expected precondition header (a second queuedhandler fails the test, proving there is no preceding
HEAD),412surfacingas
Error::Precondition, and — for S3 — that an unconditional delete stilluses bulk
DeleteObjects.412, matchinggeneration →
204and object gone).412 Precondition Failed, newer object preserved, matching version deletes. LocalStack reports501forIf-MatchonDELETEand the test skips viaNotSupported.Notes for reviewers
ifGenerationMatchon delete, so the GCS conditional-delete scenario onlyruns when pointed at the real GCS endpoint (same gating as the existing
ifGenerationMatch-dependent tests).S3ConditionalPut, or introduce a separate knob?DeleteOptions::preconditionthe preferred shape, or wouldmaintainers prefer a
DeleteMode-style enum (as withPutMode/CopyMode)?