Skip to content

Source digest coverage: staged, parallel bulk, and TriG/N-Quads paths (closes #118's stated boundary) #124

Description

@styk-tv

Why

#118 (v0.6.31) records the source byte digest at the turtle funnel — and states its
boundary plainly: the staged loader, the v0.6.2 parallel bulk path, and TriG/N-Quads
quad ingest do not record.
For founding/module graphs the funnel suffices; for at-scale
loads (which go through staged/bulk precisely) the graphs most worth pinning are the ones
not yet covered.

What

Extend source_sha256 / source_loads recording to the remaining ingest paths:

  • Staged loader: hash incrementally as STAGE consumes the file (workers or coordinator —
    whichever honestly sees every byte exactly once); record in the coordinator's final phase.
  • Parallel bulk path: same contract, hash during the sequential read pass.
  • TriG / N-Quads quad ingest: per-destination-graph recording is ill-defined for a
    multi-graph stream — record on each touched graph with the digest of the whole input
    plus source_loads increment, and document that a multi-graph stream's digest describes
    the stream, not the slice (the honest semantics; alternatives welcome in review).

Same lock-ordering rule as #118's fix: registry writes open before partition/DDL locks.

Acceptance

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3laterenhancementImprovement to existing behaviourloaderIngest / staged loader

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions