fullhistory: delete dead pre-wired surface + stale contract docs (closes #841) - #846
Conversation
…cs (#841) Deletes the freeze-handoff mirror surface (ConcurrentBitmaps.Snapshot/Len, ConcurrentLedgerOffsets.Snapshot/EventIDs + the COW ownership contract), Reader.Lookup and its hand-rolled cold resolve pipeline (retiring ErrTermNotFound), events.TermsFor, the dead IndexBaseChunk/IndexFileName naming scheme, and the census tail (ColdWriter.ChunkID/Dir, eventstore.ErrWriterClosed re-export, unreachable Finalize branch and ConcurrentBitmaps.Get guards, redundant per-package closed sentinels in favor of stores.ErrStoreClosed, cold-writer mkdir moved to the ingest layer, tip-knob double-defaulting, validateConfig's unconsumed return, and the test-only exports ReadColdBin / ChunksPerIndex / NewConcurrentBitmaps / FirstSeq). Corrects the stale contract docs that would misdirect the #772 read-path work: Reader.All/Offsets claims, the three IngestLedgerEvents writer attributions, db/event.go's per-event-index claim, the 4x-wrong RocksDB block-size baseline, the RecSplit promise, cold_bin.go's guard credits and duplicate-key claim, the three false packfile-laziness comments, and documents GetTransaction's lookup-incomplete shape as deliberate. Tests exercising deleted API were ported to the surviving surface.
tamirms
left a comment
There was a problem hiding this comment.
Verified the deletion claims against the code rather than the PR body: the ConcurrentBitmaps.Get invariant holds on every publish path, the zero-append Finalize branch is unreachable because drain errors unless the full chunk range was consumed, the tip-knob single-defaulting is behavior-preserving (withDefaults fills the same constants), and the query path touches none of the deleted surface (Query resolves exclusively through LookupKeys at query.go:231, misses are nil slots, and offsets flow through Offsets() -> View() -> LedgerOffsets.EventIDs). The doc corrections I spot-checked are all accurate, including the packfile.Open background-open description and the 4 KiB RocksDB BBTO default.
The comments below are all one class: a few stale references of exactly the kind this PR exists to remove survived, mostly in files the diff didn't touch. None are blocking on their own, but since the PR's whole purpose is killing misdirection ahead of #772, it seems worth sweeping them in the same pass.
The failing Build / dependency-sanity-checker / complete checks fail identically on #844, so they look pre-existing on the base branch rather than caused by this PR.
| @@ -114,8 +90,8 @@ func (m *ConcurrentLedgerOffsets) EndLedger() uint32 { | |||
| // View returns a *LedgerOffsets sharing the live backing array, | |||
There was a problem hiding this comment.
The type comment over in ledgeroffsets.go (lines 11 and 18) still names ConcurrentLedgerOffsets.Snapshot: once as the freeze-handoff example in the build-then-read list, and once as the recommended conversion path ("call Snapshot to convert when handing off to the cold reader's build-then-read shape"). With Snapshot deleted here, both now point at API that no longer exists, which is the same misdirection class this PR is clearing out. That file isn't in the diff, so I'm flagging it on the deletion instead.
Suggested rewording: drop the Snapshot example from the list (cold backfill and DecodeLedgerOffsets are the two remaining build-then-read producers), and replace the second sentence with something like: "For live ingest paths (HotStore) use ConcurrentLedgerOffsets; queries borrow a read-only view of it via View."
There was a problem hiding this comment.
Done in a83d4a3 — dropped the Snapshot example from the build-then-read list (cold backfill and DecodeLedgerOffsets remain) and replaced the second paragraph with your suggested wording: live ingest uses ConcurrentLedgerOffsets, queries borrow a read-only view via View.
| return key | ||
| } | ||
|
|
||
| // TermsFor returns the 16-byte hashed term keys (contractID + topics |
There was a problem hiding this comment.
One reference to the deleted function survives outside this diff: eventstore/query.go:47 says "(see events.TermsFor)" in the Filter doc. Pointing it at events.TermsForBytes instead would keep the topic-count rationale reachable. query.go isn't part of the diff, so I can't anchor a comment there; noting it on the deletion instead.
There was a problem hiding this comment.
Done in a83d4a3 — query.go:47 now points at events.TermsForBytes. Grepped the tree for other surviving events.TermsFor references; this and cold_index.go were the only two.
| // case. | ||
| // LookupKeys entry against it to a nil-bitmap miss through the | ||
| // ordinary path, so neither readers nor orchestrators need a | ||
| // pack-without-index special case. |
There was a problem hiding this comment.
A few lines below this hunk (line 54), the comment still says both builders re-derive terms via "per-event events.TermsFor + Bitmaps.AddTo". With TermsFor deleted in this PR, this should now name events.TermsForBytes, which is what the cold backfill ingester actually calls (ingest/events.go).
There was a problem hiding this comment.
Done in a83d4a3 — the cold_index.go builder comment now names events.TermsForBytes, matching what the cold backfill ingester actually calls.
| @@ -231,7 +169,7 @@ func TestConcurrentBitmaps_GetReturnsImmutableSnapshot(t *testing.T) { | |||
| // freeze time after ingest stops), so multi-goroutine Snapshot | |||
There was a problem hiding this comment.
This comment still explains why Snapshot is intentionally not exercised by the test, but Snapshot no longer exists after this PR. The sentence starting "Snapshot is intentionally NOT exercised here" can go; the remaining rationale (Get is lock-free and returns an immutable snapshot, run under -race) stands on its own.
There was a problem hiding this comment.
Done in a83d4a3 — removed the stale Snapshot sentence; the comment now ends with the Get lock-free/immutable-snapshot rationale and the -race note.
| require.NoError(t, err) | ||
| seededKeys = append(seededKeys, keys...) | ||
| } | ||
| expected := make(map[events.TermKey]uint64) |
There was a problem hiding this comment.
Small observation on the port, not blocking: the old Snapshot-based comparison checked both directions, so it would also have caught warmup inventing terms that were never seeded. This version checks that every seeded term survives with the right cardinality, but a phantom extra term in the reopened mirror would now pass unnoticed. I don't see a way to restore that direction without giving ConcurrentBitmaps term enumeration back, which would defeat the point of the deletion, so maybe just a sentence in the test comment acknowledging the narrower scope.
There was a problem hiding this comment.
Agreed there's no way to restore the reverse direction without re-adding term enumeration, which would defeat the deletion. Added a sentence to the test comment in a83d4a3 acknowledging the check is one-directional by design (phantom extra terms in the reopened mirror would pass unnoticed).
Closes #841.
Deletes the dead pre-wired surface the freeze rework left behind and corrects the stale contract docs that would misdirect the #772 read-path work. Pure-negative diff plus doc corrections — no behavior change on any production path.
Deletions
Freeze-handoff mirror surface.
ConcurrentBitmaps.Snapshot/Len,ConcurrentLedgerOffsets.Snapshot/EventIDs, and the Clone-safe/COW ownership contract that existed only for them.Snapshot's doc cited a WriteColdIndex call chain that does not exist — the cold index is built from a single-threadedBitmapsiningest/events.go, and the live freeze re-derives from raw LCMs.Reader.Lookup.QueryusesLookupKeysexclusively;Lookup(k) ≡ LookupKeys([k])[0], andColdReader.Lookuphand-duplicated the whole resolve pipeline for one key. Deleting it retiresErrTermNotFound— a per-key miss is a nil bitmap fromLookupKeys, and the closed-store sentinel is unified below.events.TermsFor. Test-only; its doc named cold backfill as the consumer, which actually usesTermsForBytes.Dead
IndexBaseChunk/IndexFileNamenaming scheme.OpenColdReader's doc told future wiring to compose a filename that never exists on disk; it now points at the real layout (geometry.Layout.TxHashIndexFilePath).Census tail.
ColdWriter.ChunkID()/Dir()+ backing fields, and the unusedeventstore.ErrWriterClosedre-export (docs now namepackfile.ErrWriterClosed).ledgerCold.Finalize+ theappendedlatch; a zero-append Finalize now fails loudly viaCommit, which is the honest shape for driver misuse.ConcurrentBitmaps.Get(a published termState always carries a non-nil bm or non-empty ids).eventstore.ErrClosedandtxhash.ErrClosed(no caller distinguished them) are gone; both stores now returnstores.ErrStoreClosedper thestores/errors.gotranslation contract, matching the ledger store. (The metastore half of this bullet was already resolved by the fullhistory: simplify LastCommittedLedger to the design signature LastCommittedLedger(cat) #835 fold —catalog.Opendelegates its guards torocksdb.New.)NewEventsColdIngester), matching the ledger/txhash siblings.daemon.goleavesStartConfig.TipBackoff/TipMaxAttemptszero forwithDefaultsto fill, andvalidateConfigreferences the named defaults directly.validateConfignow returns onlyerror— its returned earliest ledger had no consumer (tests read the pin viacat.EarliestLedger()).txhash.ReadColdBin(tests use anfhtesthelper / in-package copy),TxHashIndexLayout.ChunksPerIndex(in-package tests readcpi),events.NewConcurrentBitmaps(tests convert from an emptyBitmaps),ledger.ColdReader.FirstSeq.Doc corrections (the #772-misdirection set)
Reader.Allno longer claims the freeze loop uses it (the freeze re-derives from raw LCMs);Reader.Offsetsno longer describes the retired hot-side Snapshot allocation (it's aViewnow).IngestLedgerEventsas the single writer now name the actual writer:HotStore.applyLedgervia the post-commit hook (bitmaps.go,concurrent_bitmaps.go,concurrent_ledgeroffsets.go).db/event.gono longer claims full-history stores no per-event index — it storesEventIdxexplicitly per the byte-stable-ID decision.rocksdb.CFOptions.BlockSize: RocksDB's BBTO default is 4 KiB, not 16 KiB — so hot_store's explicit 4-KiB values pin the default rather than override it, and future CF sizing isn't reasoned from a 4× wrong baseline.cold_bin.gocreditsscanBinHeader(the production on-disk guard), and no longer claims duplicate truncated keys are "preserved" — the streamhash build fails on them.eventstore/cold_reader.gowere false:packfile.Openstarts each open (and holds its fd) in a background goroutine immediately; only the metadata decode is deferred. The ledger store's description was already accurate.TxReader.GetTransaction's "lookup incomplete" (rather than not-found) shape for an unverifiable candidate is now documented as deliberate for Wire the v2 stores into the API handlers and support both v1 and v2 #772.query.go:248line ref inConcurrentBitmaps.Get's borrow-guard note names the guard instead of a line number.Notes
streaming:error/log-prefix bullet was already closed by fullhistory: reorganize packages (#824) #844 (commit 8fa6c1a).TermsForBytesand compares per-key lookups); tests that existed only to pin a deleted symbol's own contract were removed.