examples

Examples

Real, committed output from a full run — so you can see the whole detect → route → certify → before/after loop without spending your own LLM quota or standing up the stack first.

sample_eval_run/summary.json

One complete python eval/run_eval.py run across all 5 scenarios (2026-08-03). For each scenario it contains:

• detection — predicted vs. expected conflict, and whether one was actually found
• drift.candidate_definitions — what each source (Slack thread, doc/notes) actually claims, extracted by the LLM
• resolved_owner — the real DataHub-registered owner routed to for the decision (or the fallback team when a term has no owner, see scenario_05)
• decision / certification_result — the certified definition written back, with before/after text
• pre_answers / post_answers — a separate consumer agent's answers to the same questions, before and after certification, with pre_match_rate / post_match_rate against the certified definition

detection_stats at the bottom: precision 1.0, recall 1.0 across all 5 scenarios, including scenario_05 where the term has no owner at all and the fallback governance team is used instead.

To reproduce this yourself (needs your own OPENROUTER_API_KEY — see the root README):

python eval/run_eval.py