Skip to content

feat(s3): implement object lock (per-key version-state tree) - #78

Merged
frrist merged 1 commit into
design/s3-object-lockfrom
frrist/feat/object-lock
Aug 14, 2026
Merged

feat(s3): implement object lock (per-key version-state tree)#78
frrist merged 1 commit into
design/s3-object-lockfrom
frrist/feat/object-lock

Conversation

@frrist

@frrist frrist commented Aug 12, 2026

Copy link
Copy Markdown
Member

Implements docs/s3-object-lock.md — stacked on #77 (the design). Rebases onto main once the doc lands.

WORM enforcement lives in versitygw's controller (auth.CheckObjectAccess); this backend stores lock state and returns the exact absent-case sentinels (§2). The pieces:

  • Schema (bucket/): VersionState{Retention, LegalHold, Tags} blocks under the new "/versionstate/0" union arm; ObjectLeaf.State roots the per-key version-state MST (same revSeqKey keying as the prev tree), joining the existing "/objectleaf/0" key with no compatibility shim per the dev-only data posture (§4.1). Tags is reserved so object tagging lands as handlers with no format change. Manifests are never rewritten: versioning invariant 5 holds verbatim.
  • Registry (§4.2): buckets.object_lock_config (verbatim BucketLockConfig JSON), Create takes an initial state so x-amz-bucket-object-lock-enabled buckets are born versioned+locked atomically, multipart_sessions carries the MPU lock headers to Complete.
  • Backend methods: the four per-version methods over the §6 check order (s3frontend/objectlock.go), Put/GetObjectLockConfiguration, the PutBucketVersioning suspend guard.
  • Write paths: creation-time stamping and delete-path state cleanup run inside the same commits that create/remove versions (§7, §9) — no post-commit windows; Head/Get echo the x-amz-object-lock-* fields (§8).

Conformance (full TestForgeVersity partition green against the smelt stack): six new lock categories, all Versioning_* lock/retention/legal-hold/WORM rows in the pass table, a LockCreation versioned-conf category for lock-enabled-bucket cases from plain-conf groups (their teardown needs ListObjectVersions), the WORMProtection bypass family as XFail (flips when bucket policies land), and promotions incl. Versioning_DeleteObject_non_existing_objects and CreateBucket_default_object_lock. Three details the run pinned are folded back into the doc: key existence outranks the lock gate, the missing-config error's variant split, and the controller's zero-time retain-until pointer.

Note on the earlier CI run: TestForgeReadAfterCatalogRetention failed at stack boot with a Docker setns exec error (compose up), before any request ran — the same binary passed six other stack boots in that job. It passes locally against this head.

🤖 Generated with Claude Code

@frrist
frrist force-pushed the frrist/feat/object-lock branch 2 times, most recently from 48f3055 to 4ba8cae Compare August 12, 2026 02:55

@hannahhoward hannahhoward left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM broadly. A few non-blocking comments.

It's really nice this didn't turn out to be that bad.

Comment thread bucket/leaf.go

// EnvelopedVersionState reads/writes one version-state block under its
// "/versionstate/0" union key.
type EnvelopedVersionState struct {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not clear the value of this. Why wouldn't you just use the base version? It's just a cborgen annotation worth of difference, unless the nil check on state on marshall is relevant

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wrapper is for decoding since cbor-gen's generated decoders ignore map keys they don't recognize. So if the wrong block ever gets decoded as a StateUnion, we wouldn't get an error, instead we'd get State == nil. The lock code would read that as "no retention, no legal hold" and happily allow a delete, which would be incorrect. So this wrapper turns it into a hard error instead. Same reason EnvelopedManifest and EnvelopedLeaf exist.

Comment thread s3frontend/objectlock.go
return cid.Undef, fmt.Errorf("state get: %w", err)
}
}
mutate(&vs)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this whole thing be in a transaction? It seems like all the lookups before this line are non mutating? Or is this a multiwriter neccesity so it doesn't mutate from under you?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's for concurrent writers, and this is a read-modify-write. Read the state block, change one field, keep the rest. So if for example, one request is setting retention and another is setting legal hold on the same version, and both read the block before taking the lock, whichever writes second wipes out the other's change. Doing the read inside the lock addresses that edge case. It also means the version can't be deleted out from under us between lookup and write. The reads should be fairly cheap and these calls should be fairly rare, so holding the lock a little longer (ideally) costs (almost) nothing...

Implements docs/s3-object-lock.md. WORM enforcement lives in versitygw's
controller (auth.CheckObjectAccess); the backend stores lock state and
returns the exact absent-case sentinels.

Per-version retention and legal holds live in a per-key version-state
tree beside the prev tree: ObjectLeaf.State roots an MST mapping
revSeqKey(seq) to VersionState blocks under the new "/versionstate/0"
union arm. The State field joins ObjectLeaf under the existing
"/objectleaf/0" key with no compatibility shim, per the repo's dev-only
data posture. Mutations rewrite only positional blocks, so manifests
stay immutable and versioning invariant 5 holds verbatim. VersionState
reserves the Tags field for object tagging (handlers, not format, in the
follow-up). Creation-time stamping (PutObject / CopyObject /
CompleteMultipartUpload lock headers) and delete-path cleanup run inside
the same commits that create and remove versions. The bucket-level lock
configuration is a registry column beside versioning; CreateBucket with
x-amz-bucket-object-lock-enabled creates the bucket versioned and locked
in one insert, and PutBucketVersioning refuses to suspend a lock bucket.

Conformance notes pinned by the run: the creation-time header paths
report the NoSpaces variant of the missing-configuration error while the
four per-version methods report the spaced one; key existence outranks
the lock-enabled gate (GetObjectRetention_non_existing_object); the
controller passes an absent retain-until header as a pointer to the zero
time; and NoSuchVersionError / InvalidArgumentError now pass through
mapCommitError verbatim like PreconditionFailedError. itest gains the six
lock groups, a WORMProtection xfail table (bucket-policy-dependent bypass
cases), and a LockCreation category for lock-enabled-bucket cases from
plain-conf groups that need the versioned teardown; the versioning tables
absorb the Versioning_* lock rows and promote
DeleteObject_non_existing_objects.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@frrist
frrist force-pushed the frrist/feat/object-lock branch from 4ba8cae to 40342c0 Compare August 14, 2026 00:56
@frrist
frrist merged commit 542fc16 into main Aug 14, 2026
12 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants