FEE wrap material on blob_locations (FIL-480)

[Peeja]

tl;dr: This adds everything Ingot needs to do the read-path decryption. It stores a wrapped CEK, as well as other info that's in the FEE header that's needed for decryption. That means that Ingot can perform a range read by grabbing just the bytes it needs to decrypt (rounded up to encryption chunk boundaries), and not also fetch the header, which is likely not contiguous.

It's all stored on ingot.blob_locations because that feels like the right entity shape-wise, but the name may no longer be appropriate. @frrist, should we broaden the name of this table?

---

Summary

Implements FIL-480. Extends the existing ingot.blob_locations table with the per-blob FEE (FilOne encryption envelope) wrap material that the read path needs to decrypt an encrypted blob — so a (range) GET can unwrap the CEK and go straight to a body-range fetch, with no COSE envelope-header round-trip.

Previously stacked on #9 (feat/arch-migration-phase1). #9 has merged, so this is now rebased onto main and targets main; only the single FIL-480 commit remains.

Why extend blob_locations rather than add a table

An earlier draft of the issue proposed a new object_encryption / per-segment table with plaintext_size / plaintext_etag / a generation counter. On review those turned out redundant, and the issue was rewritten to an extension:

• Object-level values already exist. ObjectManifest carries ETag and Body.Size/MD5/SHA256 — properties of the plaintext, computed pre-encryption in the PUT pipeline (FIL-481). No new columns.
• No generation counter. Overwrite already swaps the MST leaf to a new manifest CID atomically; a reader resolves one manifest or the other, never a partial mix.
• The wrap material is a 1:1 fact about a (space, digest) row. A fresh CEK per encryption event makes every ciphertext digest unique to one encryption (never shared across objects, even for identical plaintext) — exactly like the existing provider/url/size columns. No region column either: one Ingot instance is one region, so region_key_version alone identifies the KEK version.

What changed

• migrations/sql/00004_blob_encryption.sql — additive ALTER TABLE ingot.blob_locations ADD COLUMN for six nullable columns (region_wrapped_cek, region_key_version, tenant_recipient_kid, base_nonce, chunk_size, protected_header), plus a Down. An unencrypted blob (the appliance topology today) leaves them NULL.
• registry.BlobLocation gains the six fields (documented); PutLocation/GetLocation carry them (nullString/nullInt64 for optional text/bigint, bytea via nil). The upsert overwrites full row state, so a rotation re-wrap is a read-modify-write.
• inmem.MemStore deep-copies the new byte-slice fields so stored/returned copies don't alias.
• Tests — inmem FEE round-trip + slice-aliasing safety + re-wrap-in-place; live-Postgres bytea/NULL round-trip, re-wrap, and the unencrypted (all-NULL) case; migration column existence + nullability.

Security / acceptance

• Raw CEK bytes are never stored — only the region-KEK-wrapped CEK plus opaque key-version / recipient identifiers.
• region_key_version / tenant_recipient_kid are opaque, so no schema change is needed if the region-key or Hilt wrap-key cardinality decisions (FIL-572 / FIL-574) later go multi-key.
• Per-blob crypto-shred = nulling these columns (or DeleteLocation); no separate mechanism. Worth confirming against FIL-489's reference-counted delete.

Testing

• GOWORK=off go build ./... ✅ · go vet ./... ✅
• go test ./... passes for all packages except the pre-existing, environment-only failure in ./testing/ (unable to add custom RootCAs HTTPClient … — reproduced identically on main with none of these changes; it's the sandbox's HTTPS-proxy CA setup, not this diff).
• The live Postgres/migration tests are gated behind INGOT_TEST_DSN and were not run here (no DB in this sandbox). Please run them in CI/local:

  INGOT_TEST_DSN=postgres://postgres:pw@127.0.0.1:55432/ingot \
    GOWORK=off go test ./registry/ ./migrations/ -run Live -v
  

Open question for reviewers (non-blocking)

blob_locations's doc comment described it narrowly as the (space, digest) → provider/URL mapping. Adding encryption material broadens its role to "the full per-blob record." I've updated the doc comments to reflect that but left the table name as-is. Should it be renamed (e.g. blob_records)? Flagging per the issue — happy to do the rename in a follow-up if we want it.

[Copilot]

Pull request overview

Extends the local ingot.blob_locations record shape to include the per-blob FEE (FilOne encryption envelope) wrap material needed for range reads to decrypt encrypted blobs without an extra envelope-header fetch, along with SQL migration and store/test updates.

Changes:

• Adds a new SQL migration to extend ingot.blob_locations with six nullable FEE wrap-material columns (plus Down migration).
• Extends registry.BlobLocation and updates the Postgres/inmem LocationStore implementations to read/write these new fields (including deep-copying byte slices in-memory).
• Adds/extends live-Postgres + migration + inmem tests to validate NULL/bytea round-trips, aliasing safety, and re-wrap updates.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
registry/stores.go	Expands BlobLocation to carry FEE wrap material and updates doc comments/contracts.
registry/stores_postgres.go	Updates PutLocation/GetLocation SQL to write/read new nullable columns and adds nullInt64.
registry/postgres_live_test.go	Adds live-Postgres tests for NULL semantics, bytea round-trips, and re-wrap-in-place behavior.
migrations/up_live_test.go	Verifies new columns exist and are nullable after migration.
migrations/sql/00004_blob_encryption.sql	Adds the actual schema migration for FEE wrap material columns on blob_locations.
inmem/stores.go	Ensures in-memory store deep-copies new byte-slice fields to avoid aliasing.
inmem/stores_test.go	Adds inmem tests for wrap-material round-trip, aliasing safety, and re-wrap-in-place behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[pyropy]

Should we use bigint here since we're using it to store the chunk size in the database? Alternativly we could use uint64 here as per the FEE FIP.