Skip to content

docs: update query benchmarks (3.9.4) - #956

Merged
carlos-alm merged 2 commits into
mainfrom
benchmark/query-v3.9.4-20260418-001953
Apr 18, 2026
Merged

docs: update query benchmarks (3.9.4)#956
carlos-alm merged 2 commits into
mainfrom
benchmark/query-v3.9.4-20260418-001953

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

Automated query benchmark update for 3.9.4 from workflow run #737.

@greptile-apps

greptile-apps Bot commented Apr 18, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This automated PR records query benchmark results for version 3.9.4, updating the summary table and latest-results section in generated/benchmarks/QUERY-BENCHMARKS.md. The WASM engine shows notably elevated fnImpact depth-3/5 latencies (~41–42% up) and diffImpact latency (~35% up) vs. 3.9.3.

  • The mid benchmark target changed from enumNode (3.9.3) to ctx (3.9.4), which means all percentage deltas in the summary table compare measurements taken against different functions. Readers may misinterpret the regressions/improvements as purely engine-performance signals when some of the delta may reflect workload differences between those two symbols.

Confidence Score: 5/5

Safe to merge — documentation-only, generated by automation, no runtime impact.

All findings are P2 (readability/comparability concern about the mid target switch). The data itself is internally consistent and correctly formatted. No code paths are affected.

generated/benchmarks/QUERY-BENCHMARKS.md — the percentage deltas in the 3.9.4 rows are compared against a different mid benchmark target (enumNode → ctx), which may mislead readers interpreting the WASM fnImpact regressions.

Important Files Changed

Filename Overview
generated/benchmarks/QUERY-BENCHMARKS.md Adds 3.9.4 benchmark data; mid target changed from enumNode to ctx, making percentage deltas vs. 3.9.3 not strictly apples-to-apples; WASM fnImpact shows 41–42% increase.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Release Workflow Trigger] --> B[Run codegraph on itself]
    B --> C{For each engine}
    C --> D[Native Rust Engine]
    C --> E[WASM Engine]
    D --> F[Query: fnDeps depth 1/3/5\nhub=buildGraph mid=ctx leaf=docs]
    E --> F
    F --> G[Query: fnImpact depth 1/3/5]
    G --> H[Query: diffImpact]
    H --> I[Median over 5 runs]
    I --> J[Compute % delta vs prev version]
    J --> K[Append to QUERY-BENCHMARKS.md]
    K --> L[Open automated PR]
Loading

Fix All in Claude Code

Reviews (1): Last reviewed commit: "docs: update query benchmarks (3.9.4)" | Re-trigger Greptile

Comment on lines +8 to +9
| 3.9.4 | native | 26.9 ↑10% | 27.3 ↑13% | 27.3 ↑13% | 4.9 ↑11% | 5.1 ↑16% | 5.1 ↑16% | 7.3ms ↓15% |
| 3.9.4 | wasm | 37.8 ↑10% | 38.5 ↑12% | 39.2 ↑15% | 5.1 ↑19% | 6.2 ↑41% | 6.1 ↑42% | 14.3ms ↑35% |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Mid-target change makes percentage deltas not comparable

The mid benchmark target switched from enumNode (3.9.3) to ctx (3.9.4). All percentage arrows in the 3.9.4 rows are computed by dividing the new latency against 3.9.3 latency from a different function. A heavier or lighter ctx graph neighbourhood can explain part (or all) of the observed regression, so readers cannot tell from the table alone how much is engine performance vs. workload change. Consider adding a footnote like *mid target changed from enumNode; deltas not directly comparable* on the 3.9.4 rows, or re-running 3.9.3 with mid=ctx as a baseline.

Fix in Claude Code

@carlos-alm
carlos-alm merged commit 99b667b into main Apr 18, 2026
14 checks passed
@carlos-alm
carlos-alm deleted the benchmark/query-v3.9.4-20260418-001953 branch April 18, 2026 21:20
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 18, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant