Repository navigation
fix(csharp): escape contextual-keyword names the grammar rejects (#5043) - #5262
ThatAblaze wants to merge 1 commit into
Conversation
…phify-Labs#5043) tree-sitter-c-sharp 0.23.5 reads `partial`, `required`, `async`, `nint`, `nuint` and `with` as their keywords where C# reads a plain name: a statement that starts with one (`partial |= x;`, `required.Add(1);`), `async` in most expressions, `nint = 1;`, `var with = x;`. The statement becomes an ERROR, and when it is the last statement of an `if` block that an `else` block follows, the recovery swallows the rest of the file and only the file node is extracted. extract_csharp now rewrites such a name to `@name` before parsing, which is the same identifier to the compiler and a name to the grammar. The `@` takes the place of an adjacent blank, so the rewrite is byte-length preserving and every reported line and column still matches the file (Graphify-Labs#2876). It runs only on a file that fails to parse, escapes every occurrence of a word together so a local's declaration and its uses keep one spelling (a call on the local still binds through its type), restores a word the parse does not need, and is dropped unless the file then parses cleanly. The AST cache schema moves from 47 to 48 so cached C# extractions re-run. Fixes Graphify-Labs#5043. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the pull request, @ThatAblaze. A maintainer will review it soon. Want to talk it through while it is in review? Come join us on our Discord server. For longer-form discussion there is also GitHub Discussions. A couple of things that speed up review: make sure the test suite passes on Python 3.11 and 3.13, and that the change keeps extraction deterministic. |
There was a problem hiding this comment.
Graphify reviewed this change.
No issues found by static checks — no coupling regressions or blocking issues in the code graph. That is not the same as safe to merge: see what was not checked.
Not checked: tests were not run; formal verification proved 0 of 1 changed function(s) (1 not verified).
Formal verification. PR-changed functions: 0/1 verified (0 proven, 0 may-equivalent, 0 distinguished) · 1 not verified (1 vacuous).
Not verified on this run: extract\_csharp (vacuous: never exercised).
Graphify review — findings
Adds a pre-parse pass for C# that escapes contextual-keyword identifiers (a member, type or variable named like a contextual keyword) so tree-sitter-c-sharp no longer misparses them. The aim is that these declarations and calls aren't dropped from the graph. A string-literal scanner keeps the rewrite out of string contents. Bumps the AST cache schema so C# extractions cached before this change are re-run instead of reused, and a new test covers that. New tests also cover the keyword-named extraction itself.
Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 5276 functions depend on the 536 functions this change touches.
Health — this change adds coupling hotspots:
- worse:
extract()— 1259 callers, 78 callees - worse:
extract_csharp()— 24 callers, 5 callees
Verification — 5276 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 4574 function(s) in the blast radius were not formally verified this run
Test selection
Test selection
366 of 582 test file(s) selected (63%) via static blast radius.
tests/test_4874_jvm_resolution.py— impacttests/test_4978_reconcile_cost.py— impacttests/test_4992_ts_import_types.py— impacttests/test_affected_overrides.py— impacttests/test_alias_folding.py— impacttests/test_analyze.py— impacttests/test_astro_extraction.py— impacttests/test_astro_import_ids.py— impacttests/test_binary_media_words.py— impacttests/test_blade_extractor.py— impacttests/test_build.py— impacttests/test_build_located_semantic_identity.py— impacttests/test_build_merge_dedup_scope.py— impacttests/test_build_merge_ghost_fold_4955.py— impacttests/test_build_merge_hyperedges_and_prune.py— impacttests/test_build_merge_shrink_guard.py— impacttests/test_builtin_global_type_refs.py— impacttests/test_c_include_resolution.py— impacttests/test_cache.py— impacttests/test_cache_long_paths.py— impacttests/test_cache_stale_import_target.py— impacttests/test_cache_windows_case_4925.py— impacttests/test_carried_hyperedge_remap.py— impacttests/test_case_only_decl_ids_4904.py— impacttests/test_case_sensitive_resolution.py— impacttests/test_charmap_encoding.py— impacttests/test_chunking.py— impacttests/test_cjs_module_extension.py— impacttests/test_cli_export.py— impacttests/test_cluster.py— impacttests/test_cobol_extractor.py— impacttests/test_code_only_scope_4923.py— impacttests/test_confidence.py— impacttests/test_confidence_score_output.py— impacttests/test_corrupt_graph_json.py— impacttests/test_cpp_alias_member_resolution.py— impacttests/test_cpp_anonymous_namespace_receivers.py— impacttests/test_cpp_conditional_initializers.py— impacttests/test_cpp_forward_declarations.py— impacttests/test_cpp_forward_receiver_completion.py— impacttests/test_cpp_inherited_member_resolution.py— impacttests/test_cpp_local_callable_shadowing.py— impacttests/test_cpp_member_call_by_receiver_type.py— impacttests/test_cpp_member_declaration_identity.py— impacttests/test_cpp_method_declarations.py— impacttests/test_cpp_nested_and_cli.py— impacttests/test_cpp_objc_cross_file_calls.py— impacttests/test_cpp_receiver_chains.py— impacttests/test_cpp_receiver_parameters.py— impacttests/test_cpp_split_header_owner.py— impact- … and 316 more
Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.
Formal verification
Could not verify: Could not verify extract\_csharp.
The verifier did not have enough to check extract\_csharp, so it is saying so rather than guessing. No false assurance is the whole point.
Guarantee: No guarantee either way, this is an honest abstention, not a pass.
Note: Reason: no capturable inputs from the test suite; property tier: not verifiable: all 9 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly NameError — names the real obstacle, not a sampling gap)
· 2 more finding(s) on lines outside this diff (see the check run).
What does this PR do?
tree-sitter-c-sharp 0.23.5 reads
partial,required,async,nint,nuintandwithas their keywords when C# reads a plain name: a statement that starts with one (partial |= x;,required.Add(1);),asyncin most expressions,nint = 1;,var with = x;. The statement becomes an ERROR, and when it is the last statement of anifblock that anelseblock follows, the recovery swallows the rest of the file and only the file node is extracted.extract_csharpnow rewrites such a name to@namebefore parsing, which is the same identifier to the compiler and a name to the grammar. The@takes the place of an adjacent blank, so the rewrite is byte-length preserving and every reported line and column still matches the file (#2876). This runs only on a file that fails to parse, escapes every occurrence of a word together so a local's declaration and its uses keep one spelling (a call on the local still binds through its type), restores a word the parse does not need, and is dropped unless the file then parses cleanly.The AST cache schema moves from 47 to 48 so cached C# extractions re-run.
Fixes #5043.
Type of change
Verification & Invariants
Invariants: Line numbers stay true, untouched files stay untouched, keywords stay keywords, and the cache version bump makes cached C# results rebuild. Each has a test that proves it.
How was this tested?
Windows 10, Python 3.13.1, after
uv sync --all-extras.uv run --no-sync pytest tests/test_csharp_contextual_keyword_names.py -q: 17 passed. The cases cover the whole-file loss, unshifted source locations, byte-length preservation, each word in a form the grammar rejects, receiver binding through an escaped local, keyword uses left untouched, a word the parse does not need keeping its spelling, comments and strings, a healthy file and a still-broken file left alone, and the cache bump.v8'sextract.pyandcache.py: collection fails, since the function does not exist there. With a small pytest plugin stubbing the function to return None, 15 fail and 2 pass. The 2 are the guards that a healthy file and a still-broken file are left alone.python -m pytest tests -qin the synced venv: 11,301 passed, 269 skipped, 1 xfailed, 5 failed. The same 5 fail on a cleanv8on this machine:test_code_only_update_all_hosts.py::test_code_only_update_follows_its_paragraph_and_merges_the_new_function[aider]and[devin]test_language_override_lifecycle.py::test_watch_relative_roots_receive_config_events_without_rebasing_source_paths[src]test_resolve_memoization.py::test_absolute_path_lookup_skips_getcwdtest_source_key_memoization.py::test_absolute_path_lookup_skips_getcwdruff check .: all checks passed.uv run --no-sync pyright: 770 errors on Windows, the same set as onv8except thatv8also flags the new test's import of the missing function. None is in the changed code.v8and with this branch: only the file that hit the bug changes. It goes from 11 to 25 symbols and from 8 to 22 methods, and its syntax-error warning is gone. One call edge in another file now resolves, because the broken file no longer leaves a placeholder for a type the call needed. No other node or edge changes, and no label gains an@.Limitations
Use(async),-async,(required.Add(1))). If the file still fails, the rewrite is dropped and the file extracts as today.nintandnuintread as a value (return nint;) are not covered: there they are followed by the same tokens as the type keyword.async(async => async + 1) is not covered, since=>also follows property names that must keep their spelling.@, so a field or enum member of that name in the same file is labelled@name. No such label appeared in the two codebases above.partial,asyncandrequiredas identifiers at statement start tree-sitter/tree-sitter-c-sharp#449 reservespartial,asyncandrequiredas identifiers. Once a release with it is pinned, this rewrite finds nothing to do for those words and returns early.Material AI assistance: Claude Opus 5.5 helped write the change and the tests; the commit carries a
Co-Authored-Bytrailer.Graphify-specific checklist
uv run python -m tools.skillgen --bless) when changing their source fragments. Not applicable: no skill fragments changed.