Skip to content

fullhistory: delete dead pre-wired surface + stale contract docs (closes #841) - #846

Merged
chowbao merged 2 commits into
feature/full-historyfrom
chore/fullhistory-dead-surface-841
Jul 13, 2026
Merged

fullhistory: delete dead pre-wired surface + stale contract docs (closes #841)#846
chowbao merged 2 commits into
feature/full-historyfrom
chore/fullhistory-dead-surface-841

Conversation

@chowbao

@chowbao chowbao commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Closes #841.

Deletes the dead pre-wired surface the freeze rework left behind and corrects the stale contract docs that would misdirect the #772 read-path work. Pure-negative diff plus doc corrections — no behavior change on any production path.

Deletions

Freeze-handoff mirror surface. ConcurrentBitmaps.Snapshot/Len, ConcurrentLedgerOffsets.Snapshot/EventIDs, and the Clone-safe/COW ownership contract that existed only for them. Snapshot's doc cited a WriteColdIndex call chain that does not exist — the cold index is built from a single-threaded Bitmaps in ingest/events.go, and the live freeze re-derives from raw LCMs.

Reader.Lookup. Query uses LookupKeys exclusively; Lookup(k) ≡ LookupKeys([k])[0], and ColdReader.Lookup hand-duplicated the whole resolve pipeline for one key. Deleting it retires ErrTermNotFound — a per-key miss is a nil bitmap from LookupKeys, and the closed-store sentinel is unified below.

events.TermsFor. Test-only; its doc named cold backfill as the consumer, which actually uses TermsForBytes.

Dead IndexBaseChunk/IndexFileName naming scheme. OpenColdReader's doc told future wiring to compose a filename that never exists on disk; it now points at the real layout (geometry.Layout.TxHashIndexFilePath).

Census tail.

  • ColdWriter.ChunkID()/Dir() + backing fields, and the unused eventstore.ErrWriterClosed re-export (docs now name packfile.ErrWriterClosed).
  • The unreachable early-success branch in ledgerCold.Finalize + the appended latch; a zero-append Finalize now fails loudly via Commit, which is the honest shape for driver misuse.
  • The unreachable nil/empty guards in ConcurrentBitmaps.Get (a published termState always carries a non-nil bm or non-empty ids).
  • The redundant closed-store sentinels: eventstore.ErrClosed and txhash.ErrClosed (no caller distinguished them) are gone; both stores now return stores.ErrStoreClosed per the stores/errors.go translation contract, matching the ledger store. (The metastore half of this bullet was already resolved by the fullhistory: simplify LastCommittedLedger to the design signature LastCommittedLedger(cat) #835 fold — catalog.Open delegates its guards to rocksdb.New.)
  • The eventstore cold writer no longer mkdirs its own bucket dir; directory creation moved to the ingest layer (NewEventsColdIngester), matching the ledger/txhash siblings.
  • Tip knobs are no longer double-defaulted: daemon.go leaves StartConfig.TipBackoff/TipMaxAttempts zero for withDefaults to fill, and validateConfig references the named defaults directly.
  • validateConfig now returns only error — its returned earliest ledger had no consumer (tests read the pin via cat.EarliestLedger()).
  • Test-only exports: txhash.ReadColdBin (tests use an fhtest helper / in-package copy), TxHashIndexLayout.ChunksPerIndex (in-package tests read cpi), events.NewConcurrentBitmaps (tests convert from an empty Bitmaps), ledger.ColdReader.FirstSeq.

Doc corrections (the #772-misdirection set)

  • Reader.All no longer claims the freeze loop uses it (the freeze re-derives from raw LCMs); Reader.Offsets no longer describes the retired hot-side Snapshot allocation (it's a View now).
  • The three contract comments naming the deleted IngestLedgerEvents as the single writer now name the actual writer: HotStore.applyLedger via the post-commit hook (bitmaps.go, concurrent_bitmaps.go, concurrent_ledgeroffsets.go).
  • db/event.go no longer claims full-history stores no per-event index — it stores EventIdx explicitly per the byte-stable-ID decision.
  • rocksdb.CFOptions.BlockSize: RocksDB's BBTO default is 4 KiB, not 16 KiB — so hot_store's explicit 4-KiB values pin the default rather than override it, and future CF sizing isn't reasoned from a 4× wrong baseline.
  • The txhash package doc no longer promises a future "RecSplit-backed" cold reader; it names the existing streamhash-MPHF-backed one.
  • cold_bin.go credits scanBinHeader (the production on-disk guard), and no longer claims duplicate truncated keys are "preserved" — the streamhash build fails on them.
  • The three packfile-laziness comments in eventstore/cold_reader.go were false: packfile.Open starts each open (and holds its fd) in a background goroutine immediately; only the metadata decode is deferred. The ledger store's description was already accurate.
  • TxReader.GetTransaction's "lookup incomplete" (rather than not-found) shape for an unverifiable candidate is now documented as deliberate for Wire the v2 stores into the API handlers and support both v1 and v2 #772.
  • The stale query.go:248 line ref in ConcurrentBitmaps.Get's borrow-guard note names the guard instead of a line number.

Notes

  • The streaming: error/log-prefix bullet was already closed by fullhistory: reorganize packages (#824) #844 (commit 8fa6c1a).
  • Tests exercising deleted API were ported to the surviving surface (e.g. warmup mirror comparison now derives expected term keys via TermsForBytes and compares per-key lookups); tests that existed only to pin a deleted symbol's own contract were removed.

…cs (#841)

Deletes the freeze-handoff mirror surface (ConcurrentBitmaps.Snapshot/Len,
ConcurrentLedgerOffsets.Snapshot/EventIDs + the COW ownership contract),
Reader.Lookup and its hand-rolled cold resolve pipeline (retiring
ErrTermNotFound), events.TermsFor, the dead IndexBaseChunk/IndexFileName
naming scheme, and the census tail (ColdWriter.ChunkID/Dir,
eventstore.ErrWriterClosed re-export, unreachable Finalize branch and
ConcurrentBitmaps.Get guards, redundant per-package closed sentinels in
favor of stores.ErrStoreClosed, cold-writer mkdir moved to the ingest
layer, tip-knob double-defaulting, validateConfig's unconsumed return,
and the test-only exports ReadColdBin / ChunksPerIndex /
NewConcurrentBitmaps / FirstSeq).

Corrects the stale contract docs that would misdirect the #772 read-path
work: Reader.All/Offsets claims, the three IngestLedgerEvents writer
attributions, db/event.go's per-event-index claim, the 4x-wrong RocksDB
block-size baseline, the RecSplit promise, cold_bin.go's guard credits
and duplicate-key claim, the three false packfile-laziness comments, and
documents GetTransaction's lookup-incomplete shape as deliberate.

Tests exercising deleted API were ported to the surviving surface.

@tamirms tamirms left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the deletion claims against the code rather than the PR body: the ConcurrentBitmaps.Get invariant holds on every publish path, the zero-append Finalize branch is unreachable because drain errors unless the full chunk range was consumed, the tip-knob single-defaulting is behavior-preserving (withDefaults fills the same constants), and the query path touches none of the deleted surface (Query resolves exclusively through LookupKeys at query.go:231, misses are nil slots, and offsets flow through Offsets() -> View() -> LedgerOffsets.EventIDs). The doc corrections I spot-checked are all accurate, including the packfile.Open background-open description and the 4 KiB RocksDB BBTO default.

The comments below are all one class: a few stale references of exactly the kind this PR exists to remove survived, mostly in files the diff didn't touch. None are blocking on their own, but since the PR's whole purpose is killing misdirection ahead of #772, it seems worth sweeping them in the same pass.

The failing Build / dependency-sanity-checker / complete checks fail identically on #844, so they look pre-existing on the base branch rather than caused by this PR.

@@ -114,8 +90,8 @@ func (m *ConcurrentLedgerOffsets) EndLedger() uint32 {
// View returns a *LedgerOffsets sharing the live backing array,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type comment over in ledgeroffsets.go (lines 11 and 18) still names ConcurrentLedgerOffsets.Snapshot: once as the freeze-handoff example in the build-then-read list, and once as the recommended conversion path ("call Snapshot to convert when handing off to the cold reader's build-then-read shape"). With Snapshot deleted here, both now point at API that no longer exists, which is the same misdirection class this PR is clearing out. That file isn't in the diff, so I'm flagging it on the deletion instead.

Suggested rewording: drop the Snapshot example from the list (cold backfill and DecodeLedgerOffsets are the two remaining build-then-read producers), and replace the second sentence with something like: "For live ingest paths (HotStore) use ConcurrentLedgerOffsets; queries borrow a read-only view of it via View."

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in a83d4a3 — dropped the Snapshot example from the build-then-read list (cold backfill and DecodeLedgerOffsets remain) and replaced the second paragraph with your suggested wording: live ingest uses ConcurrentLedgerOffsets, queries borrow a read-only view via View.

return key
}

// TermsFor returns the 16-byte hashed term keys (contractID + topics

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One reference to the deleted function survives outside this diff: eventstore/query.go:47 says "(see events.TermsFor)" in the Filter doc. Pointing it at events.TermsForBytes instead would keep the topic-count rationale reachable. query.go isn't part of the diff, so I can't anchor a comment there; noting it on the deletion instead.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in a83d4a3 — query.go:47 now points at events.TermsForBytes. Grepped the tree for other surviving events.TermsFor references; this and cold_index.go were the only two.

// case.
// LookupKeys entry against it to a nil-bitmap miss through the
// ordinary path, so neither readers nor orchestrators need a
// pack-without-index special case.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few lines below this hunk (line 54), the comment still says both builders re-derive terms via "per-event events.TermsFor + Bitmaps.AddTo". With TermsFor deleted in this PR, this should now name events.TermsForBytes, which is what the cold backfill ingester actually calls (ingest/events.go).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in a83d4a3 — the cold_index.go builder comment now names events.TermsForBytes, matching what the cold backfill ingester actually calls.

@@ -231,7 +169,7 @@ func TestConcurrentBitmaps_GetReturnsImmutableSnapshot(t *testing.T) {
// freeze time after ingest stops), so multi-goroutine Snapshot

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment still explains why Snapshot is intentionally not exercised by the test, but Snapshot no longer exists after this PR. The sentence starting "Snapshot is intentionally NOT exercised here" can go; the remaining rationale (Get is lock-free and returns an immutable snapshot, run under -race) stands on its own.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in a83d4a3 — removed the stale Snapshot sentence; the comment now ends with the Get lock-free/immutable-snapshot rationale and the -race note.

require.NoError(t, err)
seededKeys = append(seededKeys, keys...)
}
expected := make(map[events.TermKey]uint64)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small observation on the port, not blocking: the old Snapshot-based comparison checked both directions, so it would also have caught warmup inventing terms that were never seeded. This version checks that every seeded term survives with the right cardinality, but a phantom extra term in the reopened mirror would now pass unnoticed. I don't see a way to restore that direction without giving ConcurrentBitmaps term enumeration back, which would defeat the point of the deletion, so maybe just a sentence in the test comment acknowledging the narrower scope.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed there's no way to restore the reverse direction without re-adding term enumeration, which would defeat the deletion. Added a sentence to the test comment in a83d4a3 acknowledging the check is one-directional by design (phantom extra terms in the reopened mirror would pass unnoticed).

@chowbao
chowbao merged commit 331df4e into feature/full-history Jul 13, 2026
7 of 15 checks passed
@chowbao
chowbao deleted the chore/fullhistory-dead-surface-841 branch July 13, 2026 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants