Skip to content

feat: route auto-sync per substrate, so k8s drift stops triggering the AWS apply - #124

Merged
lex00 merged 1 commit into
mainfrom
feat/117-per-substrate-autosync
Aug 3, 2026
Merged

feat: route auto-sync per substrate, so k8s drift stops triggering the AWS apply#124
lex00 merged 1 commit into
mainfrom
feat/117-per-substrate-autosync

Conversation

@lex00

@lex00 lex00 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Closes #117. Implements the design in docs/design/per-substrate-autosync.md (#105).

The bug

Auto-sync had two assumptions that only hold on a single-substrate estate.

Detection was estate-wide. driftDigest reduced the whole overlay to one string — it answered did anything move and never what moved, so a drifted k8s Service and a drifted security group were indistinguishable.

Selection took the first match. pickAutoSyncOp was ops.find((o) => o.kind === kind) over a list sorted by name.

Together, on a mixed estate whose halves are applied by different Ops (a cloudformation ApplyOp for the cloud half, a kubectl one for the k8s half), drift in the k8s half triggered the AWS apply — silently, because the Op ran and succeeded without touching what had actually moved. That is a wrong action, not a missing feature. It has bitten nobody only because auto-sync is off by default.

What changed

  • poll.tsdriftDigestsByLexicon + changedLexicons. Every IR node already carries lexicon; the reduction was throwing it away. Nodes only, so feat: cross-substrate live overlay -- preserve the IR's cross-provider edges in --live #103's cross-substrate anchor edges cannot make one substrate appear to move because it is now joined to another that did. No new read, no chant change.
  • ops.tsOpInfo.substrate, scraped from the declared target exactly the way name: / signalName: / env: already are. chant's ApplyTarget is cloudformation | kubectl | arm -> aws | k8s | azure. An absent or unrecognised target is unscoped, never a guess.
  • autosync.tspickAutoSyncOps returns a pick per moved substrate plus the declines. Several Ops matching one substrate, or none, declines and says which — the same discipline soleManagedCluster (feat: cross-substrate live overlay -- preserve the IR's cross-provider edges in --live #103) and addValueMatchEdges follow. The declines reach the now-line, because a self-heal loop that declines silently reads as a broken one.
  • history.ts + server.ts — the rollback interlock. While a rollback branch is open, pull-request mode must not reconcile: it would re-adopt exactly what the rollback removed. apply mode is untouched, since healing toward newly-rolled-back source is the intended completion of a rollback.

Two decisions the design left open

Both are written up in the design doc's new Implemented section, since each narrows a claim it makes.

An Op declaring no target is not automatically ambiguous. Question 2 reads an undeclared target as the ambiguity case. Taken literally that disables pull-request mode everywhere — chant's ReconcileOp accepts no target at all (it reads the estate and opens a PR; there is no transport to name), so every reconcile Op is unscoped. Implemented rule: a declared substrate wins; failing that a single unscoped Op covers everything that moved; two or more competing still declines.

The interlock suspends the whole estate, not a subset. Question 3 anticipates narrowing by the rolled-back sourceDir, but behold's rollback (/api/rollback -> chant lifecycle rollback <env> --to <ref>) takes no directory scope, so the rollback really is whole-project. Narrowing would invent a distinction the command does not make. suspendedByRollback already takes the moved set, so it narrows there if rollback ever gains a scope.

One behaviour worth flagging

A substrate whose Op is refused by the one-write-at-a-time guard is not retried — the poll advances its baseline whether or not a trigger started, so it waits until it drifts again. That was equally true of the single-Op loop, which dropped the same event just as silently; the change is that the refusal is now visible.

Gate

tsc clean, 430 tests (up from 399), build clean.

Verified end to end against real .op.ts fixtures rather than only in units — discovery through routing on a mixed estate:

discovered: cloud-apply[apply/aws] k8s-apply[apply/k8s]
k8s Service drifted -> moved: ["k8s"]
routed to: k8s-apply for k8s
declined: []

The old find-by-kind returned cloud-apply for that same event.

…e AWS apply

Implements #117, to the design in docs/design/per-substrate-autosync.md (#105).

Auto-sync had two assumptions that only hold on a single-substrate estate.
Detection was estate-wide: `driftDigest` reduced the whole overlay to one
string, answering "did anything move" and never "what moved", so a drifted k8s
Service and a drifted security group were the same event. Selection took the
first match: `pickAutoSyncOp` was `ops.find((o) => o.kind === kind)` over a
name-sorted list.

Together, on a mixed estate whose halves are applied by different Ops, drift in
the k8s half triggered the AWS apply — silently, because the Op ran and
succeeded without touching what had actually moved. A wrong action, not a
missing feature. It has bitten nobody only because auto-sync is off by default.

- poll.ts: `driftDigestsByLexicon` + `changedLexicons`. Every IR node already
  carries `lexicon`; the reduction was throwing it away. Nodes only, so #103's
  cross-substrate anchor edges cannot make one substrate look like it moved
  because it is now joined to another that did. No new read.
- ops.ts: `OpInfo.substrate`, scraped from the declared `target` the same way
  `name:`/`signalName:`/`env:` already are. chant's ApplyTarget is
  cloudformation|kubectl|arm -> aws|k8s|azure. An unrecognised or absent target
  is unscoped, never a guess.
- autosync.ts: `pickAutoSyncOps` returns a pick per moved substrate plus the
  declines. Several Ops matching one substrate, or none, declines and says which
  — the discipline soleManagedCluster (#103) and addValueMatchEdges follow. The
  now-line carries it, because a self-heal loop that declines silently reads as
  a broken one.
- history.ts + server.ts: the rollback interlock. While a rollback branch is
  open, `pull-request` mode must not reconcile — it would re-adopt exactly what
  the rollback removed. `apply` mode is untouched: healing toward rolled-back
  source is the intended completion of a rollback.

Two decisions the design left open, both written up in its new Implemented
section:

An Op declaring no target is NOT automatically ambiguous. chant's ReconcileOp
accepts no `target` at all, so every reconcile Op is unscoped, and reading an
absent target as ambiguity would have disabled pull-request mode on every
project. Declared substrate wins; failing that a single unscoped Op covers the
estate; two or more competing still declines.

The interlock suspends the whole estate rather than a subset, because behold's
rollback takes no directory scope — narrowing would invent a distinction the
command does not make.

Gate: tsc clean, 430 tests (up from 399), build clean. Verified end to end
against real .op.ts fixtures: a drifted k8s Service routes to k8s-apply, where
the old find-by-kind returned cloud-apply.

Closes #117
@lex00
lex00 force-pushed the feat/117-per-substrate-autosync branch from 4de8f52 to 18a3eb2 Compare August 3, 2026 17:05
@lex00
lex00 merged commit e73b656 into main Aug 3, 2026
1 check passed
@lex00
lex00 deleted the feat/117-per-substrate-autosync branch August 3, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement per-substrate auto-sync: today k8s drift can trigger the AWS apply

1 participant