Data Governance

Data-aware governance and security for the rossoctl agentic platform. Data-governance (DG) observes how data flows across agents, tools, and sessions: AuthBridge lineage sidecars emit facts-only spans for each HTTP hop, DG ingests and stores them, and derives interactions, payload classifications, and data lineage from the raw stream. The DG UI renders the resulting per-request interaction forest so operators can see — and eventually govern — how information moves through the system.

Architecture

AuthBridge lineage sidecars emit two facts-only spans per HTTP exchange; the DG receiver persists them, and the Layer-2 processors derive interactions, classifications, and lineage that the UI renders. The full architecture is in docs/PROJECT.md; the exact producer↔DG wire format is in docs/sidecar-wire-contract.md.

flowchart LR
  agents["Agents & tools<br/>(+ AuthBridge lineage sidecar)"]
  agents -- "facts-only spans<br/>(OTLP)" --> collector["rossoctl OTEL collector<br/>(tee)"]
  collector --> receiver["DG receiver<br/>(OTLP → spans table)"]
  receiver --> db[("Postgres<br/>spans")]
  db --> interactions["interactions<br/>processor"]
  db --> classification["classification<br/>processor"]
  db --> lineage["data-lineage<br/>processor"]
  interactions --> derived[("derived tables<br/>interactions · entities · classifications")]
  classification --> derived
  lineage --> derived
  derived --> ui["DG UI<br/>(resource tree)"]

Loading

Deploying data-governance

Everything runs on the local Kind cluster. The single entry point is deploy/dg.sh, which installs, uninstalls, and reports status for the whole DG component (Postgres, receiver, UI, the processors, the verdict service, the UI HTTPRoute, and the collector tee), and instruments a namespace's agents/tools for lineage.

This guide covers installing data-governance and instrumenting a namespace. It assumes the rossoctl platform (the cluster, Istio Gateway, and OTEL collector) is already running — installing rossoctl itself is out of scope here.

1. Install the DG component

./deploy/dg.sh component install     # build + load + apply + tee + rollout
./deploy/dg.sh component status      # what's present/ready + is the tee wired
./deploy/dg.sh component uninstall   # the inverse (add --keep-data to keep the PVC)

On a Podman-backed Kind cluster, set CONTAINER_TOOL=podman and KIND_EXPERIMENTAL_PROVIDER=podman. If the classification image is already cached on the host, use ./deploy/dg.sh component install --reuse-classification to load it into a fresh Kind node while rebuilding only the shared receiver/UI image. The command checks that the cached image exists, and holds the verdict service, before starting the build. The verdict service runs from that image, so rebuild it after changing its code.

component install is idempotent; add --no-build to skip the image build when only manifests changed. It performs a rollout restart for you — this is load-bearing and easy to forget: the manifests pin :latest with imagePullPolicy: IfNotPresent, so kubectl apply alone will not cycle pods onto a freshly-loaded image.

The per-manifest breakdown, per-workload topology, the NetworkPolicy security boundary, the collector-tee mechanics, the raw step-by-step sequence, and the migration-restart hazards are documented as deployment internals in deploy/k8s/README.md.

2. Instrument a namespace

To make DG observe a namespace, instrument its agents and tools for lineage:

./deploy/dg.sh namespaces list                 # namespaces DG can instrument
./deploy/dg.sh namespace <ns> instrument       # instrument trusted agents/tools in <ns>
./deploy/dg.sh namespace <ns> instrument <e>   # one trusted entity, once <ns> is instrumented
./deploy/dg.sh namespace <ns> status           # instrumentation state of <ns>

Instrumentation is one-way and mode-preserving. Rossoctl-managed agents/tools are selected by rossoctl.io/type, retain their enforcing sidecars and auth, and are converged by rolling the application shim, then writing the proxy pipeline into the namespace authbridge-runtime-config. Namespace-wide activation completes all preflight checks, then rolls tools before agents, alphabetically within each group. status reports subsequent drift. The reasoning is recorded in ADR-0031 and ADR-0033, and the CLI design in docs/cli.md.

3. Worked example: the travel-advisor demo

The travel_advisor multi-agent demo (in the sibling agent-examples project) is a convenient namespace to instrument end-to-end:

# 1. Deploy the demo into its namespace (see agent-examples/README-agent.md
#    for the full app-side playbook), then point DG at it:
./deploy/dg.sh namespace travel-advisor instrument

# 2. Run the demo from its dedicated demo-client pod (an external, un-instrumented caller):
kubectl -n travel-advisor exec deploy/demo-client -- python3 /app/app/demo.py

# 3. View the resulting trace in the DG UI:
#    http://dg.localtest.me:8080   (routed through the shared rossoctl Istio Gateway)

The UI landing page ("Recent traces") lists the run; click one to walk the resource tree. The full demo runbook (deploying the app, the demo-client pod, and the expected trace shape) lives with the agent-examples project. Scripted demos on a running cluster, with their commands and tools, live under demos/ (demos/README.md).

Configuration

• DATABASE_URL — Postgres DSN. Required for the migrate CLI and any caller of data_governance.db.
• DB_POOL_MIN_SIZE / DB_POOL_MAX_SIZE — connection-pool sizing (default 1 / 10).
• DB_POOL_TIMEOUT — seconds to wait for a connection from the pool before raising PoolTimeout (default 30).
• DG_FF_HIDE_CONNECT_ONLY_TRACES — when true (the default), omit each Standalone CONNECT trace from GET /api/traces. Its spans remain stored and directly accessible. Accepted values are true/1/yes/on and false/0/no/off; any other configured value prevents API startup.

Where to go next

• docs/DEVELOPMENT.md — local dev: tooling, running tests, repo layout, the two-span lineage pipeline, and loading traces/fixtures.
• deploy/k8s/README.md — deployment internals and the raw manifest-level procedure.
• docs/PROJECT.md — full architecture.
• docs/sidecar-wire-contract.md — the producer↔DG wire format.
• data_governance/risk/README.md — the risk sub-component: evidence gathering, OPA policy decisions, the /risk/* API, and how to run/test it.
• CONTEXT.md and docs/adr/ — the domain glossary and architecture decision records.