Repository navigation
feat(query): find code by the words in its body - #5052
rohit-jsfreaky wants to merge 2 commits into
Conversation
|
Thanks for the pull request, @rohit-jsfreaky. A maintainer will review it soon. Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions. A couple of things that speed up review: make sure the test suite passes on Python 3.10 and 3.13, and that the change keeps extraction deterministic. |
There was a problem hiding this comment.
Graphify reviewed this change.
Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.
Not checked: tests were not run; no formal proof of the changed code.
Graphify review — findings
Adds an optional body_terms field to Python file, class and function nodes, holding the unique identifier parts, string words and comment words from each node's own source lines, with credential-shaped text redacted and words limited to 3–40 characters. query uses it as a body tier below the name tiers and as a body ranking that adds start nodes after the name seeds and feeds the file ordering, while path and explain ignore it. On nine Python repos graph.json grows 11–35% with no measurable build-time cost, and _attach_python_body_terms decides which node owns each line span, so decorator lines are where that ownership can go wrong.
Worth a look
- Decorator text is assigned to the parent node instead of the decorated definition —
graphify/extract.py:1859· Escalate · medium- NOT verified (no proof, no reproducing execution) — needs human review
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 5274 functions depend on the 700 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
_query_graph_text()— 55 callers, 10 callees - worse:
_score_query()— 23 callers, 9 callees - worse:
collect_files()— 22 callers, 8 callees - worse:
_query_terms()— 32 callers, 3 callees - new:
walk()— 3 callers, 4 callees - worse:
_extract_python_rationale()— 1 callers, 8 callees - worse:
_xaml_csharp_class_nodes()— 1 callers, 8 callees
Verification — 5274 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 4636 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
527 of 527 test file(s) selected (100%) via static blast radius.
Escalated to a full run for safety (
noncode-change,uncovered-code) — the selection is not trustworthy on its own (see below). CI should run the whole suite.
tests/pi.test.ts— full-run-safetytests/test_4874_jvm_resolution.py— impact, full-run-safetytests/test_4909_reference_binding_scale.py— full-run-safetytests/test_4978_reconcile_cost.py— impact, full-run-safetytests/test_4992_ts_import_types.py— impact, full-run-safetytests/test_affected_cli.py— impact, full-run-safetytests/test_affected_defines.py— full-run-safetytests/test_affected_member_seed.py— full-run-safetytests/test_affected_overrides.py— impact, full-run-safetytests/test_affected_path_file_node.py— impact, full-run-safetytests/test_affected_seed_reference_forms.py— impact, full-run-safetytests/test_agents_platform.py— full-run-safetytests/test_alias_folding.py— impact, full-run-safetytests/test_analyze.py— impact, full-run-safetytests/test_anthropic_custom_endpoint.py— full-run-safetytests/test_antigravity_install.py— full-run-safetytests/test_apm_fallback_version.py— full-run-safetytests/test_architecture_doc.py— full-run-safetytests/test_astro_extraction.py— impact, full-run-safetytests/test_astro_import_ids.py— impact, full-run-safetytests/test_atomic_canvas_export.py— full-run-safetytests/test_atomic_version_stamp.py— full-run-safetytests/test_atomic_writes.py— full-run-safetytests/test_backend_env_isolation.py— full-run-safetytests/test_backend_extras.py— full-run-safetytests/test_backend_opt_in.py— full-run-safetytests/test_benchmark.py— impact, full-run-safetytests/test_benchmark_raw_graph.py— impact, full-run-safetytests/test_benchmark_real_query.py— impact, full-run-safetytests/test_binary_media_words.py— impact, full-run-safetytests/test_blade_extractor.py— impact, full-run-safetytests/test_build.py— impact, full-run-safetytests/test_build_located_semantic_identity.py— impact, full-run-safetytests/test_build_merge_dedup_scope.py— impact, full-run-safetytests/test_build_merge_ghost_fold_4955.py— impact, full-run-safetytests/test_build_merge_hyperedges_and_prune.py— impact, full-run-safetytests/test_build_merge_shrink_guard.py— impact, full-run-safetytests/test_builtin_global_type_refs.py— impact, full-run-safetytests/test_c_include_resolution.py— impact, full-run-safetytests/test_cache.py— impact, full-run-safetytests/test_cache_long_paths.py— impact, full-run-safetytests/test_cache_stale_import_target.py— impact, full-run-safetytests/test_cache_windows_case_4925.py— impact, full-run-safetytests/test_callflow_html.py— full-run-safetytests/test_cargo_introspect.py— full-run-safetytests/test_cargo_missing_manifest.py— full-run-safetytests/test_carried_hyperedge_remap.py— impact, full-run-safetytests/test_case_only_decl_ids_4904.py— impact, full-run-safetytests/test_case_sensitive_resolution.py— impact, full-run-safetytests/test_charmap_encoding.py— impact, full-run-safety- … and 477 more
non-code file(s) changed (
docs/how-it-works.md) → running the full suite for safety (a code graph can't see config/fixture/data deps)
changed code file(s) with no mapped test (
docs/how-it-works.md) — a coverage gap or a missing link — running the full suite rather than only the selected tests
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Docs that may be stale (advisory)
ARCHITECTURE.md§ Module responsibilities (lines 13-41): references changed symbolsgraphARCHITECTURE.md§ Edge direction in graph.json (lines 106-111): references changed symbolsgraphBENCHMARKS.md§ Reproducing (lines 190-203): references changed symbolsgraphCHANGELOG.md§ 0.9.102 (2026-11-03) (lines 27-57): references changed symbolsgraphCHANGELOG.md§ 0.9.101 (2026-11-02) (lines 58-81): references changed symbolsgraphCHANGELOG.md§ 0.9.98 (2026-10-30) (lines 118-137): references changed symbolsgraphCHANGELOG.md§ 0.9.93 (2026-10-20) (lines 243-269): references changed symbolsgraphCHANGELOG.md§ 0.9.92 (2026-10-16) (lines 270-295): references changed symbolsgraphCHANGELOG.md§ 0.9.90 (2026-10-13) (lines 309-327): references changed symbolsgraphCHANGELOG.md§ 0.9.88 (2026-10-12) (lines 342-366): references changed symbolsgraph
…and 10 more.
· 3 grounded finding(s) anchored inline below; 4 more finding(s) on lines outside this diff (see the check run).
| if target is not None and words: | ||
| target["body_terms"] = " ".join(sorted(words)) | ||
|
|
||
| def walk(node, owner_nid: str, is_function: bool) -> None: |
There was a problem hiding this comment.
walk()
high coupling complexity (Ca·Ce = 12).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| walk(root, file_nid, False) | ||
|
|
||
|
|
||
| def _extract_python_rationale(path: Path, result: dict) -> None: |
There was a problem hiding this comment.
_extract_python_rationale()
fans out to 8 callees (efferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
| return _score_query(G, terms, collect_per_term_seeds=False).ranked | ||
|
|
||
|
|
||
| def _score_query( |
There was a problem hiding this comment.
_score_query()
fans out to 9 callees (efferent coupling); 23 callers depend on it (afferent coupling).
Grounded coupling-delta finding (deterministic), not an LLM guess.
|
The bot's decorator finding was right, thanks. A decorated def was cut at its Fixed in 642cff0: a This moves only decorator words between nodes, so I haven't re-run the 293-question benchmark for it. I'll post updated numbers here if they change. The other inline notes are coupling metrics on functions this PR extends ( |
|
Updated numbers with the decorator fix (642cff0). I rebuilt all 125 graphs from scratch and re-ran
No question entered or left the top 5. 8 questions moved by one or two places: 2 up and 6 down by one. I rebuilt those graphs from the same base without the fix to separate the two effects. All 8 moves come from the fix, and the newer upstream base only changed answer sizes. So the decorator change is neutral for ranking. I'm keeping it because the words now sit on the right node. The headline numbers in the description still hold. The data is on my fork, branch |
642cff0 to
c0f6ee7
Compare
|
Rebased onto current v8 (after 0.9.104). The only conflict was |
c0f6ee7 to
b678aa3
Compare
|
Rebased again onto v8 0.9.105, which is already on schema 42. This PR now bumps |
b678aa3 to
9447c53
Compare
|
Rebased onto v8 0.9.106. Two conflicts: |
9447c53 to
7503fb8
Compare
|
Rebased onto v8 0.9.107. The only conflict was |
7503fb8 to
73eec7c
Compare
|
Rebased onto v8 0.9.108. The only conflict was |
`query` only matched node names and labels, so a question worded the way
an issue is written ("credit history transactions endpoint") ranked tests,
docs and look-alike names above the function whose body does the work.
Python file, class and function nodes now carry `body_terms`: the unique
words of their own source lines (identifier parts, string and comment
words, 3-40 chars, secret-like text redacted first, nested defs keep their
own words). In `query` only (`path` and `explain` are unchanged):
- a body tier scores below the exact/substring name tiers, so per word a
name match still beats a body match and the per-term seed guarantee holds;
- a separate body ranking adds 3 extra seeds after the name seeds, and the
answer's file order fuses the name and body rankings;
- vendored copies (vendor/, node_modules/, nested packages/, jquery-x.y.z.js,
.min.js) rank lower.
In a new process the body words are searched as one string instead of
building a full word index, so a cold CLI query stays within 0.25 s of
before. AST cache schema 47 -> 48 (Python output changes).
SWE-bench Lite + Verified, settings chosen on the 114 Django questions only:
top-5 file hit on the 179 other questions 61% -> 75% (v8 vs this,
McNemar p=1e-5), on the 116 Verified questions never used for any choice
65% -> 78%. graph.json +11-35%, build time unchanged.
Co-Authored-By: Claude <noreply@anthropic.com>
`body_terms` cut a decorated def at the `def`/`class` line, so its
decorator lines (`@app.route("/invoices/export")`) were counted in the
parent's words (the file or the class) instead of the definition they
decorate. Treat a `decorated_definition` as one span owned by the def.
Co-Authored-By: Claude <noreply@anthropic.com>
73eec7c to
1370200
Compare
|
Rebased onto v8 0.9.111. The only conflict was |
Closes #5051
What does this PR do?
querynow also looks at the words inside a node's body. This is the follow-up to #4982, built on top of the existing scoring tiers.Extraction (Python only, to keep the diff small). File, class and function nodes carry
body_terms: the unique words of their own source lines. That means identifier parts, string words and comment words, 3-40 characters long. Secret-like text is redacted first, the same way as for rationale labels. A nesteddefkeeps its words in its outer function. AST cache schema 47 -> 48.Query only.
pathandexplaindo not change.body_terms) adds 3 extra seeds after the name seeds. The answer's file order fuses the name ranking and the body ranking.vendor/,node_modules/, a nestedpackages/,jquery-3.5.1.js-style names and.min.js.Cost in a new process. Building a full word index made the first CLI query 2 s slower on Django. Instead, all body words sit in one string and each query word is found with a substring search. A long-running MCP server switches to a full index after 32 distinct words. Both paths return the same nodes, and a test checks that.
Diff:
graphify/serve.py+350 / -21,graphify/extract.py+78, one docs line, 10 tests intests/test_query_body_terms.py.One trade-off to flag
Step 2 changes the answer's file order: a file that ranks first by name can now be shown after a file that is strong in both rankings. Seed choice and the per-term guarantee are not affected.
I also measured a strict version, the body tier alone with no fused order. It only reaches 66% (vs 75%). In 52 of 54 Django misses, one exact name match took every seed slot and kept the gold file out of the top 5. If you want names to always decide the order, I can cut this PR down to the strict version.
Numbers
SWE-bench Lite + Verified, 293 questions. The gold file is the one the fix touched. Every setting was chosen on the 114 Django questions only and frozen at one commit. The rest ran once, through
graphify benchmark --questions(#4342).Paired top-5 on the 179: this PR vs 0.9.102, 29 questions gained and 4 lost (McNemar p = 1e-5). This PR vs BM25-files, 26 gained and 11 lost (p = 0.02).
Per repo top-5, 0.9.102 -> this PR:
Cost
graphify query, Django (median of 20)Limitations
body_termsthe same way.Type of change
Verification & Invariants
Invariants:
With the body tier off,
querygives exactly 0.9.102's answers.pathandexplainare unchanged.The two word-lookup paths return the same nodes.
The new process gives byte-identical answers to the full-index version on all 293 questions, at depths 2 and 3 with a fixed hash seed. With a random seed, only the EDGE line order varies, the same way it already does on v8.
Read the CONTRIBUTING.md guide.
Reproduced the issue and identified the invariant.
Made the smallest fix necessary.
Added a regression test (if bug fix) or isolated boundary test.
Kept the PR description synchronized with the final implementation.
Documented any limitations / unsupported cases explicitly.
How was this tested?
The scripts and every result file are on my fork, branch
claude/query-study, folderresearch/query/pr2/, with a README that says how each number was made.Graphify-specific checklist
uv run python -m tools.skillgen --bless) when changing their source fragments. (Not needed: no skill fragments changed.)🤖 Generated with Claude Code