feat(s3): implement object lock (per-key version-state tree) - #78
Conversation
48f3055 to
4ba8cae
Compare
hannahhoward
left a comment
There was a problem hiding this comment.
LGTM broadly. A few non-blocking comments.
It's really nice this didn't turn out to be that bad.
|
|
||
| // EnvelopedVersionState reads/writes one version-state block under its | ||
| // "/versionstate/0" union key. | ||
| type EnvelopedVersionState struct { |
There was a problem hiding this comment.
Not clear the value of this. Why wouldn't you just use the base version? It's just a cborgen annotation worth of difference, unless the nil check on state on marshall is relevant
There was a problem hiding this comment.
This wrapper is for decoding since cbor-gen's generated decoders ignore map keys they don't recognize. So if the wrong block ever gets decoded as a StateUnion, we wouldn't get an error, instead we'd get State == nil. The lock code would read that as "no retention, no legal hold" and happily allow a delete, which would be incorrect. So this wrapper turns it into a hard error instead. Same reason EnvelopedManifest and EnvelopedLeaf exist.
| return cid.Undef, fmt.Errorf("state get: %w", err) | ||
| } | ||
| } | ||
| mutate(&vs) |
There was a problem hiding this comment.
should this whole thing be in a transaction? It seems like all the lookups before this line are non mutating? Or is this a multiwriter neccesity so it doesn't mutate from under you?
There was a problem hiding this comment.
Yeah, it's for concurrent writers, and this is a read-modify-write. Read the state block, change one field, keep the rest. So if for example, one request is setting retention and another is setting legal hold on the same version, and both read the block before taking the lock, whichever writes second wipes out the other's change. Doing the read inside the lock addresses that edge case. It also means the version can't be deleted out from under us between lookup and write. The reads should be fairly cheap and these calls should be fairly rare, so holding the lock a little longer (ideally) costs (almost) nothing...
Implements docs/s3-object-lock.md. WORM enforcement lives in versitygw's controller (auth.CheckObjectAccess); the backend stores lock state and returns the exact absent-case sentinels. Per-version retention and legal holds live in a per-key version-state tree beside the prev tree: ObjectLeaf.State roots an MST mapping revSeqKey(seq) to VersionState blocks under the new "/versionstate/0" union arm. The State field joins ObjectLeaf under the existing "/objectleaf/0" key with no compatibility shim, per the repo's dev-only data posture. Mutations rewrite only positional blocks, so manifests stay immutable and versioning invariant 5 holds verbatim. VersionState reserves the Tags field for object tagging (handlers, not format, in the follow-up). Creation-time stamping (PutObject / CopyObject / CompleteMultipartUpload lock headers) and delete-path cleanup run inside the same commits that create and remove versions. The bucket-level lock configuration is a registry column beside versioning; CreateBucket with x-amz-bucket-object-lock-enabled creates the bucket versioned and locked in one insert, and PutBucketVersioning refuses to suspend a lock bucket. Conformance notes pinned by the run: the creation-time header paths report the NoSpaces variant of the missing-configuration error while the four per-version methods report the spaced one; key existence outranks the lock-enabled gate (GetObjectRetention_non_existing_object); the controller passes an absent retain-until header as a pointer to the zero time; and NoSuchVersionError / InvalidArgumentError now pass through mapCommitError verbatim like PreconditionFailedError. itest gains the six lock groups, a WORMProtection xfail table (bucket-policy-dependent bypass cases), and a LockCreation category for lock-enabled-bucket cases from plain-conf groups that need the versioned teardown; the versioning tables absorb the Versioning_* lock rows and promote DeleteObject_non_existing_objects. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
4ba8cae to
40342c0
Compare
Implements docs/s3-object-lock.md — stacked on #77 (the design). Rebases onto
mainonce the doc lands.WORM enforcement lives in versitygw's controller (
auth.CheckObjectAccess); this backend stores lock state and returns the exact absent-case sentinels (§2). The pieces:bucket/):VersionState{Retention, LegalHold, Tags}blocks under the new"/versionstate/0"union arm;ObjectLeaf.Stateroots the per-key version-state MST (samerevSeqKeykeying as the prev tree), joining the existing"/objectleaf/0"key with no compatibility shim per the dev-only data posture (§4.1).Tagsis reserved so object tagging lands as handlers with no format change. Manifests are never rewritten: versioning invariant 5 holds verbatim.buckets.object_lock_config(verbatimBucketLockConfigJSON),Createtakes an initial state sox-amz-bucket-object-lock-enabledbuckets are born versioned+locked atomically,multipart_sessionscarries the MPU lock headers to Complete.s3frontend/objectlock.go),Put/GetObjectLockConfiguration, thePutBucketVersioningsuspend guard.x-amz-object-lock-*fields (§8).Conformance (full
TestForgeVersitypartition green against the smelt stack): six new lock categories, allVersioning_*lock/retention/legal-hold/WORM rows in the pass table, aLockCreationversioned-conf category for lock-enabled-bucket cases from plain-conf groups (their teardown needsListObjectVersions), theWORMProtectionbypass family as XFail (flips when bucket policies land), and promotions incl.Versioning_DeleteObject_non_existing_objectsandCreateBucket_default_object_lock. Three details the run pinned are folded back into the doc: key existence outranks the lock gate, the missing-config error's variant split, and the controller's zero-time retain-until pointer.Note on the earlier CI run:
TestForgeReadAfterCatalogRetentionfailed at stack boot with a Dockersetnsexec error (compose up), before any request ran — the same binary passed six other stack boots in that job. It passes locally against this head.🤖 Generated with Claude Code