fullhistory: bootstrap frontfill-only from a history-archive tip (#833)#850
Draft
chowbao wants to merge 3 commits into
Draft
fullhistory: bootstrap frontfill-only from a history-archive tip (#833)#850chowbao wants to merge 3 commits into
chowbao wants to merge 3 commits into
Conversation
chowbao
force-pushed
the
chore/fullhistory-hardening-840
branch
3 times, most recently
from
8a10b76 to
604a021
Compare
Base automatically changed from
chore/fullhistory-hardening-840
to
feature/full-history
July 13, 2026 19:46
A frontfill-only deployment (no [backfill.datastore]) had no tip source on
first start: the only source was notConfiguredTip, which always errored, so no
earliest_ledger form could pin a floor even though captive core could ingest
from it once pinned.
Consolidate all network-tip sampling into one tipSampler (Sample(ctx), retry
defaults bound at construction), replacing the (NetworkTipBackend, TipBackoff,
TipMaxAttempts) trio threaded through validateConfig, resolveEarliestFirstStart,
StartConfig, and backfillToTip. The sampler queries its sources in order and
returns the first that answers:
- the bulk lake frontier (backend.Tip) when a datastore is configured, and
- the history archives' root HAS (GetRootHAS().CurrentLedger) as the fallback,
and the sole source for a frontfill-only daemon.
The archive URLs are the same [ingestion].history_archive_urls captive core
already needs — no new config. Because the sampler now falls back to the
archives, an unavailable tip means no source was reachable at all, so the
synthetic tip = lastCommitted degraded mode in backfillToTip is deleted: on a
tip failure the pass errors and the supervisor restarts. The archives' one-
checkpoint lag is absorbed by the existing anchor = max(tip, lastCommitted) and
the signed withinOneChunkOfTip, so no lag handling is added.
chowbao
force-pushed
the
feat/fullhistory-frontfill-tip-833
branch
from
f707825 to
914cabb
Compare
…ured A frontfill-only daemon with no [ingestion].history_archive_urls previously built an empty sampler and only died later at captive-core open, with a less on-point message; buildTipSampler now rejects the misconfiguration at startup. Also note at the sub-genesis gate that it is source-order-blind: a source must error (not return 0) when empty or it shadows a healthy fallback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #833.
A frontfill-only deployment (
[backfill.datastore].typeempty) had no tip source on first start: the only source wasnotConfiguredTip, which always errors, so noearliest_ledgerform could pin a floor — even though captive core could ingest from the pinned floor once it exists.One
tipSampler, one binding site. All network-tip sampling now goes through a singletipSampler(Sample(ctx), retry defaults bound at construction), replacing the(NetworkTipBackend, TipBackoff, TipMaxAttempts)trio previously threaded throughvalidateConfig,resolveEarliestFirstStart,StartConfig, andbackfillToTip. The retry semantics are the oldnetworkTiphelper's, moved verbatim: cenkalti/backoff constant interval, count-bounded, sub-genesis tip isbackoff.Permanent("not ready"), ctx cancellation aborts the wait.Lake first, archives as fallback. The sampler tries its sources in order: the bulk lake frontier (
backend.Tip) when a datastore is configured, then the history archives' root HAS (GetRootHAS().CurrentLedger). For a frontfill-only daemon the archives are the sole source — and how it bootstraps itsearliest_ledgerpin. The archive URLs are the same[ingestion].history_archive_urlscaptive core already needs; no new config. The archives' one-checkpoint (64-ledger) lag is absorbed by the existinganchor = max(tip, lastCommitted)and the signedwithinOneChunkOfTip, so no lag handling is added.Synthetic tip deleted. Because the sampler falls back to the archives, an unavailable tip now means no source was reachable at all — so the
tip = lastCommitteddegraded mode inbackfillToTipis gone. A tip failure errors the pass and the supervisor restarts (a transient outage self-heals on re-sample); the daemon never serves behind an unknown frontier.Not a
Backend. The issue sketched acaptiveSource Backend, but a frontfill-only deployment never freezes bulk history, so it needs noLedgerStream— a Tip-onlyBackendwould be a half-implemented interface. The archive tip is a plain tip source at the daemon layer instead;backfill.Backendkeeps its one real implementation (bsbSource), andnotConfiguredTip/backendTipare deleted.Tests: frontfill bootstrap through
buildTipSamplerwith a fake root-HAS archive, source ordering (lake preferred, archive fallback when the lake is down, all-down exhausts), the retry/sub-genesis/ctx-cancel suite carried over ontoSample, andTestBackfill_RestartTipUnreachableErrorsreplacing the deleted degraded-mode test. Fullfullhistory-shortsuite and-raceon thefullhistorypackage pass.Deliberate skip: the archive pool is built without a network passphrase, so the SDK's HAS passphrase check is off. Wiring it would reorder daemon construction (the passphrase is peeked from the captive-core TOML after
validateConfig, the sampler is needed before it), and a wrong-network archive URL already fails loudly at live ingestion — captive core consumes the same[ingestion].history_archive_urls— with recovery being a wipe of the still-empty first-start data dir.