feat(nixos): delta prefetch for upgrades via pifinder-differ

[mrosseel]

What

Device-side applier for the delta-update transport. Before nixos_upgrade lets nix download whole store paths, it asks the pifinder-differ server (running beside Attic on the cache host) for byte-level zstd patches against same-stem store paths the device already holds — across all retained generations — and imports whatever it can reconstruct. Every imported path is one nix no longer downloads.

Measured server-side on real CI build pairs: 255 MiB of changed store paths compress to 1.5 MiB of patches (0.6%).

How

• delta_updates.py (new, stdlib-only like nixos_upgrade):
  • candidate bases = newest same-stem paths in /nix/store
  • POST /delta → 200 (patch ready) / 202 (computing, bounded retry) / 204 (full download)
  • reconstructs the target NAR with zstd --patch-from against nix-store --dump of the base (canonical on both ends)
  • verifies sha256 of the reconstructed NAR before import — nix-store --import does not check the NAR against the path name, so this is the line between a bad patch and a corrupt store
  • assembles the import stream locally (export wire format) from the response's references + deriver — validated byte-identical against real nix-store --export output for paths with and without references/deriver
• nixos_upgrade.py: one prefetch call between estimate_download and run_build, plus a re-estimate so progress denominators match what nix actually fetches
• services.nix: new pifinder.deltaUrl option (default "" = disabled), env wiring, zstd in the upgrade service path

Safety

• Off by default. Empty deltaUrl short-circuits before any work.
• Best-effort per path: server down, bad patch, disk full, corrupt base (nix-store --verify-path) — each falls back to the normal binary-cache download; prefetch_deltas can never raise into the upgrade.
• Decode window capped at 2^28 (256 MiB) on the device regardless of what the server offers.
• Free-space check (2× NAR + patch + slack) before applying each path.

Tests

23 new unit tests (pytest -m unit): stem/path parsing, candidate ordering, byte-exact import-stream framing, protocol states, apply guardrails, never-raises. Full test_nixos_upgrade.py suite still passes.

Not in this PR

The server is currently loopback-only on the cache host; exposing it (e.g. https://deltas.pifinder.eu via Caddy) is a separate infra change, after which enabling is a one-line pifinder.deltaUrl setting per device.

🤖 Generated with Claude Code