Skip to content

hyp query grep: the full search stack, integrated on master (LLP 0265 T4-T7) - #984

Open
bgmcmullen wants to merge 19 commits into
masterfrom
grep/integration
Open

hyp query grep: the full search stack, integrated on master (LLP 0265 T4-T7)#984
bgmcmullen wants to merge 19 commits into
masterfrom
grep/integration

Conversation

@bgmcmullen

Copy link
Copy Markdown
Contributor

What this is

The whole hyp query grep feature (LLP 0265 T4-T7) as one branch against current master, so it can be checked out and exercised locally and merged with a single click. It supersedes the four-PR stack #951 / #952 / #953 / #954, which carried the same work but could not be merged in sequence: this repo squash-merges, so each merge would have re-proposed its parent's changes to the branches behind it.

Every review those four PRs received is included here, including neutral's fixes on each.

Trying it locally

git fetch && git checkout grep/integration && npm install   # the lockfile moved; see below
hyp query grep "some text you remember"
hyp query grep "needle" --session-id <id> --from 2026-08-01 --limit 5
hyp cache status          # ends with the grep index coverage line
npm run smoke -- query_grep_roundtrip

npm install matters: master bumped squirreling to 0.16.1 and a stale node_modules fails typecheck with ~75 errors in union-source / sql that have nothing to do with grep.

What it adds

  • The search service (src/core/search/grep_service.js): one newest-first walk over the cache's live data files; each file served either through its hypgrep .index.parquet sidecar or by a brute scan of the nine allowlisted columns. Purge-correct (position deletes filtered on both tiers, so a stale sidecar cannot resurrect a purged row) and visibility-correct (LLP 0105, via the same predicate the SQL path uses).
  • The verb (grep_verb.js, registered in CORE_VERBS): hyp query grep plus the grep_search MCP tool. The schema is wire-compatible with the server's own grep_search, so --remote <target> reaches a server's archive-backed search with no server-side work.
  • Sidecar builds at maintenance (sidecar_build.js + worker pair): compaction finalizes a file, then an index is built for it in a worker thread. Sidecar existence is the completion marker, the publish is write-then-rename, and a file whose build keeps failing is quarantined after three attempts and served by the scan tier forever after.
  • Surfaces and proof: a grep index: N of M data files indexed line on hyp cache status, the grep section in both copies of the hypaware-query skill, and the query_grep_roundtrip hermetic smoke (added to the release battery).

Integration fixes this branch carries

Two things only showed up once the four branches sat together on current master:

  1. sidecarPathFor moved modules during Sidecar builds at maintenance: compaction finalizes a file, the index follows (LLP 0265 T6) #953's review, while a Surfaces and proof: status coverage, the skill learns grep, and the roundtrip smoke (LLP 0265 T7) #954 review commit still imported it from its old home. Typecheck error and every maintenance test red.
  2. Master's D1 short-flag gate (LLP 0293) walks every visible core command and refuses -Z, sparing only query sql. query grep is the second verb with a greedy positional, so it trips the gate. It joins the exemption rather than opting into strictness: grep's positional is search text, and a recorded transcript is mostly command lines, so -Z or --force is an ordinary thing to search for. Same bargain rg -- -Z strikes.

Verification

  • npm run typecheck: clean.
  • npm test: 5085 pass. Two failures, neither from this work: the graph help-text case that fails identically on a clean origin/master worktree on this machine, and claude-telemetry-unparseable-body (a concurrency test that passes 3/3 in isolation and is unrelated to search).
  • npm run smoke -- query_grep_roundtrip: ok. It drives the real CLI through scan tier, hyp purge --session, maintenance building sidecars, the coverage line, the indexed tier, and LLP 0105 withholding from three caller contexts.
  • Dogfood: hyp query grep hypgrep finds this feature's own development sessions in the local cache.

Note on coverage

The allowlist is nine columns. tool_args was dropped during #953's review: it is the dataset's one VARIANT column and no tier could ever produce a hit from it, so it cost brute-scan decode time while promising coverage it could not deliver. #977 restores it once hypgrep can index VARIANT.

🤖 Generated with Claude Code

bgmcmullen and others added 18 commits August 19, 2026 10:38
…er file (LLP 0265 T4)

src/core/search/grep_service.js is the client half of LLP 0264: the same
two-tier search the server runs, folded onto the client's single cache.
executeGrepSearch flushes the dataset's pending spool first (the query
seam's own freshness move, now exported from sql.js), walks every live
data file newest message-day first, and serves each file through its
hypgrep sidecar (parquetFind, index proposes, shared matcher confirms)
when one exists beside it, by brute scan under the narrow SCAN_COLUMNS
projection when not. Budget, truncated/exhausted, hit shape and sort
order mirror the server byte for byte through the shared core/search
modules.

Two row gates the server does not need:

- Purge: a raw file read does not apply Iceberg position deletes, so
  the walk carries each file's committed delete positions (new
  listLiveDataFiles export on the iceberg store) and filters both tiers
  by them; a stale sidecar cannot resurrect a purged row (LLP 0104).
- Visibility: every surfaced row passes the LLP 0105 lattice check via
  cwdWithheldFromCaller, hoisted out of withLocalOnlyVisibility so the
  SQL path and this walk share one predicate instead of two copies. The
  check runs after the match, so withheldRows counts hits the caller
  was not allowed to see and costs no result budget.

The shared types gain GrepSearchParams, the wire shape every serving
surface accepts.

Tests cover both tiers end to end on a real Iceberg cache: locators and
snippets, newest-first truncation, from/to and session/chain scoping,
the JSON tool_args column matching through cellText, local-only
withholding at every caller rank, purge on both tiers, the sub-ngram
literal query answering exactly through an index, and spool-captured
rows surfacing after the service's own flush.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… follows (LLP 0265 T6)

Compaction is the moment a data file stops changing, so it is the one
point where a hypgrep index can be built once and stay valid against its
rows. maintainCache now follows every committed rewrite of the grep
dataset with a sidecar-build pass over the new generation's files, in a
worker thread (createIndex is seconds of straight-line CPU and the
daemon is single-threaded), one file at a time. The worker handle and
thread are ports of the server's index-worker pair, with one behavioral
fix: the worker holds an event-loop ref exactly while a build is in
flight, because an always-unref'd worker deadlocks any process whose
loop would otherwise drain while awaiting the build.

Sidecar existence is the completion marker, no ledger: the publish is a
write-then-rename, a killed daemon leaves nothing half-claimed, and the
next pass rebuilds whatever is missing. A file whose build keeps failing
is quarantined after three attempts (in-memory, process-lifetime; a
restart is the retry) and the scan tier serves it forever after: index
presence is purely a performance property. The build pass can never fail
the partition's own maintenance verdict.

Two hazards found and closed on the way:

- countDataFiles and measureDataDir counted sidecars (*.parquet in
  data/), which would have made every just-indexed partition read as
  "grew since compaction" and rewrite itself every tick through the LLP
  0199 baseline gate. Both now exclude .index.parquet; a test pins that
  a second unforced tick stays converged.
- A corrupt sidecar used to fail the whole search; the indexed tier now
  runs into local buffers and commits only on success, so an unreadable
  sidecar degrades that one file to the brute scan with no double count.

GREP_DATASET joins the shared searchable-columns module so the search
service and the build pass cannot disagree about which dataset carries
indexes.

Tests: per-file build and existence-marker idempotency, the quarantine
budget with the scan tier still serving, the corrupt-sidecar fallback,
maintenance building indexes for exactly the grep dataset, sidecars not
re-triggering compaction, and a retired generation dying whole with its
sidecars inside (the no-GC-code guarantee). The compaction-effectiveness
tests' liveDataFiles helper learns the same sidecar exclusion the
production counters did.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…sion-safe publish, and failures that name their file

- Hoist `sidecarPathFor` beside `GREP_DATASET` in searchable_columns.js.
  The build pass and the search service each carried their own copy of
  the `.index.parquet` rule; two copies of a path contract drift into a
  build that writes an index nobody probes for.
- Give the publish scratch file a random token. A fixed `<sidecar>.tmp`
  is only atomic for a single writer: the daemon tick and a hand-run
  `hyp` over the same cache would interleave into one scratch file and
  rename the mixture into place as a finished sidecar. The scratch file
  is now also removed on the failure path.
- Name the data file on `grep_index.build_failed` /
  `grep_index.file_quarantined` / `grep_search.sidecar_unreadable`, and
  add the component/operation attributes, so three warnings can be told
  apart as one poisoned file or three.
- Append rather than spread the indexed tier's buffered hits: `limit`
  reaches the service unvalidated and one file can fill the budget.
- Correct the module docs: the pass runs only behind a committed
  compaction, which always publishes a fresh generation, so it never
  re-attempts a file it skipped or failed on, and a daemon restart is
  not a retry. Add the LLP 0264#lifecycle ref the module realizes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…failing open

Review fixes on the T4 grep service (LLP 0265):

- The limit returned the OLDEST matches. `hits.length = limit` ran before
  `sortHits`, so the cut fell in walk order; rows inside one data file run
  oldest to newest, so `--limit 3` over today's session answered the first
  three matches, not the newest three. Hits are now trimmed in sort order
  (amortized, so the buffer stays bounded), and the file walk stops only
  once every file still ahead is strictly older than the oldest kept hit.
- `AbortSignal.timeout` threw out of the service. Its reason is a
  DOMException named `TimeoutError`, which the `AbortError` name check
  rejected, so the documented "partial answer, marked not exhausted"
  became an error for the deadline shape the signal exists to carry.
- `chainId` without `sessionId` was silently discarded, answering across
  every session instead of the chain's.
- `limit` is validated beside the query: an absent one made the budget NaN
  and walked the whole cache, a negative one threw a bare RangeError.
- `listLiveDataFiles` swallowed a metadata load failure and answered `[]`,
  so a corrupt table made grep report zero hits where `hyp query sql`
  raises. It now propagates, matching `dataSourceForTable`.
- `sortHits`' tiebreak returns 0 for equal keys, now that the buffer is
  sorted repeatedly.

Five regression tests, each verified to fail on the pre-fix tree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The sidecar existence probe only ruled out a missing index. A sidecar that
exists but cannot be read (a half-written index from a killed build, a
truncation from a full disk, a format the installed hypgrep refuses) throws
from inside parquetFind, where the footer is parsed and the version checked,
and nothing caught it: one poisoned sidecar failed every grep over the whole
cache, including the partitions the walk never reached. That makes index
state a correctness input, which LLP 0264 #lifecycle says it never is.

The indexed read now degrades to the scan tier when the index proves
unusable before it produced a row, and the reader catch no longer
special-cases ENOENT: any unreadable sidecar is an unindexed file. A failure
after the first row still propagates, because retrying it as a scan would
count the collected hits twice.

Also give the file-walk comparator a 0 for equal days, like sortHits: one
day is many files and the early break reads the walk as strictly
day-descending, so same-day order should come from the comparator rather
than from whatever the engine's sort happens to do.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lowlist

Merge of the base branch plus the maintainer's decision on #953.

## Conflict: src/core/search/grep_service.js

Both branches independently fixed "a poisoned sidecar fails the whole
query", so the conflict is two spellings of one fix, not two intents.

- t4 (`b590b110`) hoisted a `searchIndexed` helper that collects through
  the shared `collect`/`trimHits` machinery (its `8b694e5b` sort-order
  truncation) and degrades ONLY when the read failed before the first
  row; a later failure propagates, because the rows already pushed to
  the shared buffer could not be taken back.
- t6 buffered the attempt into a local array and committed on success,
  so any failure degrades, and logged `grep_search.sidecar_unreadable`
  naming the file that needs deleting.

Taken: t4's structure with t6's buffering and log folded in. The helper
keeps its name, the 2-argument `isAbort(err, signal)` t4 introduced (t6
still called the 1-argument form, which would have turned a timeout into
a thrown error), the day-descending early break, and sort-order
truncation; it gains the local buffer, so a sidecar that tears mid-read
degrades that one file instead of failing the query, and it gains the
warning. `trimHits` is generalized to `trimBuffer(list)` so the per-file
buffer is trimmed by the same rule as the shared one: buffering must not
trade the walk's memory bound away, and cutting the buffer in walk order
would reintroduce the bug `8b694e5b` fixed.

## Decision on #953: tool_args leaves SEARCHABLE_COLUMNS

`tool_args` is the dataset's one VARIANT column. The index worker only
indexes STRING leaves in the allowlist and the server's row predicate
gates on `typeof value === 'string'`, so the column has never produced a
hit on any tier in either repository: T3's `cellText` coercion made the
client's scan tier uniquely able to match it, on a premise ("the indexed
tier reads the column's own text") that was false on both sides. Rather
than add coverage neither repo has ever had, the column is dropped and
the loss is recorded. Follow-up: #977.

- `SEARCHABLE_COLUMNS` loses `'tool_args'`; `SCAN_COLUMNS` derives from
  it, so the brute scan stops decoding the column too.
- The module comment now gives the VARIANT reason and points at #977,
  in the spirit of server LLP 0157 #identifier-columns.
- `cellText` keeps its coercion (it is what keeps `rowTest`, `test` and
  `locate` answering identically on any cell shape, and #977 needs it in
  place), but its comment stops claiming the indexed tier reads the
  column's text. Its `@ref` gloss is corrected to match.
- `toHit` and `GrepSearchMatcher` lose the same false claim.
- Tests: the pinned allowlist drops the column, a new test pins its
  absence from both the allowlist and the scan projection, and the
  matcher tests exercise the coercion through a column that is actually
  searchable while pinning that `tool_args` no longer matches.
- New in `search-grep-service.test.js`: a row matching only in
  `tool_args` returns zero hits from BOTH tiers, scan and indexed, with
  the tier counters proving each one really served the file. The
  invariant is tier agreement, not coverage.

No LLP change: LLP 0264 never enumerates the columns (it points at this
module and defers to server LLP 0157), so nothing it settled is
contradicted. Amending server LLP 0157 and the now-wrong `grep_search`
tool blurb is hypaware-server work, tracked separately.

`sidecarPathFor` stays the single contract owner in
`searchable_columns.js`; no re-export from `sidecar_build.js` is
restored (#954 takes the import fix on its side).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… scratch is not data bytes

The merge resolution's per-file buffer degraded one file instead of failing
the query, but it also made a deadline throw away everything the index had
already produced for the file it landed in. hypgrep checks the signal at
every coalesced range boundary, so a deadline lands inside a file, and on a
newest-first walk that is the newest file the caller most wants. Committed
before the abort propagates: safe because an abort ends the walk, so the
file is never rescanned and no row can be counted twice.

grep_search.sidecar_unreadable named only the sidecar, but parquetFind opens
the source data file through the same factory and runs the row filter per
row, so a torn source parquet lands in that catch too and points the
operator at a healthy index. Renamed to grep_search.indexed_read_failed and
both files are named.

measureDataDir excluded `*.index.parquet` but not the build's publish
scratch, `<file>.index.parquet.<uuid>.tmp`, which survives a kill between
write and rename with no reaper until the generation retires. countDataFiles
already skips it, so counting its bytes broke the shared-file-set invariant
in the dangerous direction: needsCompaction compacts on a LOW average, so a
large orphan makes a fragmented partition read as healthy. Test pins it.

Also recorded, not fixed: a sidecar freezes the allowlist it was built over
(hypgrep stores hypgrep.text_columns in the index and prunes to it, and
nothing compares that stamp to today's SEARCHABLE_COLUMNS), so #977 has to
invalidate existing sidecars rather than only build new ones.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
queryGrepVerb in src/core/search/grep_verb.js joins CORE_VERBS beside
query sql: one declaration projects the `hyp query grep` CLI command and
the `grep_search` MCP tool, and because the tool name and inputSchema
match the server's own grep_search (query, regex, session_id, chain_id,
from, to, limit), `--remote <target>` reaches the server's
archive-backed search with no server-side feature work. A server host
displaces the kernel verb with its own via unregister (T2's affordance,
server LLP 0178), so shipping it is safe on every host.

The plan's open argv question resolves itself: the codec already maps
`--session-id` onto the snake_case wire name, so no alias was needed.

`include-local-only` is the one local-only parameter and deliberately
carries NO schema default: argvToParams sends every defaulted property
over the wire on --remote, the server's schema does not know the name,
and a default would therefore fail validation on every remote call. A
test pins the absence.

The render flattens hits to one row per matched column (locators lead,
snippet trails, part_id ready to pivot into query sql) and delegates to
the shared query formatter, which is what gives grep the LLP 0225
contract for free: table/markdown escape captured bytes for a human
reader, json/jsonl stay byte-exact for a pipeline, and the context
budgets and --output spill behave exactly as query sql. Truncation, an
uncovered walk, freshness, and local-only withholding each get their own
stderr line; stdout stays a valid render.

Tests: wire-schema pinning (property set, required, the no-default
hazard, the coverage clause), codec flag mapping, end-to-end CLI runs
over a real cache (hit render, newest-first, escape-vs-byte-exact,
truncation notice, out-of-range limit fallback), a bare server-shaped
result rendering without the local fields, and a stubbed remote MCP
server proving the wire params travel exactly and the same render draws
the server's answer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…he snippet last

Review fixes on top of the T5 verb, each reproduced against a real
Iceberg cache before the change:

- `--limit 5000` answered with 50 rows (the default) and then advised
  "raise --limit", advice the caller had already followed and could not
  follow further. Above the ceiling now clamps to the ceiling the flag's
  own help advertises; only an unusable value (absent, fractional, zero)
  still falls back to the default.
- `--from 2026-8-1` rendered an empty answer, exit 0, nothing on stderr:
  the window is compared lexicographically, so a mistyped day prunes every
  real one. A verb whose summary works this hard to make "zero hits" mean
  something must not let a typo forge one, so a day bound outside
  YYYY-MM-DD is refused with the flag named.
- The snippet sat between the locators, and `renderTable` bounds a
  column's width but never truncates a cell, so any snippet past 80
  characters pushed `message_id` and `part_id` out of column on exactly
  the rows a reader scans. Locators now lead and the snippet trails, as
  the render comment always claimed, with the row keys inserted in the
  same order so `--format json` and the table agree.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s from lying (#952 review 2)

Three edges the verb owned and got wrong:

- `core_commands.js` projects every CORE_VERBS entry pre-boot so `hyp
  --help` can render, so the top-level `grep_service.js` import pulled
  hypgrep, hyparquet and the Iceberg store into the front door of every
  `hyp` invocation. Measured on `hyp --help`: 173-176ms with the eager
  import, 158-159ms with it deferred, 157ms on a tree with no grep verb
  at all. Now loaded inside `operation`, the way `verb_command.js`
  already defers the remote stack.

- At the 1000-hit ceiling the truncation notice still said "raise
  --limit", which is the exact advice the clamp exists to avoid printing
  at a caller who cannot follow it. The operation now reports whether the
  ceiling was the binding limit and the notice names the ceiling instead.

- Zero hits over zero searched files rendered identically to zero hits
  over the whole cache: empty stdout, empty stderr, exit 0. The summary
  spends 500 characters making "zero hits" honest for an MCP caller, and
  the un-searched case counterfeited one. The service already returns
  `indexedFiles`/`scannedFiles`; the render now says when both are zero.
  Quiet on `--remote`, which carries no file counts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…oundtrip smoke (LLP 0265 T7)

The last task of the grep plan, on top of T5 (the verb) and T6 (the
sidecar build):

- hyp query status reports grep-index coverage: a summary line (grep
  index: N of M data files indexed) plus an indexed= extra on each
  searchable partition, computed by a pure directory scan in cacheStatus
  (indexedFileCount, grep dataset only). "Grep is slow on deep history"
  is now diagnosable where the operator already looks, and the expected
  gap (fresh files index only at compaction) is explained in the line
  itself.

- The hypaware-query SKILL.md (both host copies) documents the grep
  subcommand: when to prefer it over LIKE-SQL, the ten-column coverage
  caveat and what zero hits does not prove, the truncation notice, the
  coverage-versus-speed relationship with the status line to check, the
  sub-ngram literal cliff (a short literal defeats index pruning but
  never correctness), and the local-only withholding parity with SQL.
  The read-class verb lists gain grep_search. The host-divergence
  fixture is re-recorded (the two deliberately host-specific lines both
  carry the verb list).

- A hermetic smoke, query_grep_roundtrip, drives the real CLI through
  the whole story: scan-tier search before any index exists, hyp purge
  --session removing a row grep can then no longer surface, hyp query
  maintain building sidecars, the status coverage line, the indexed
  tier answering identically (proved from query.grep_search span
  attributes: indexed>0 scanned=0, and no query text in any span), and
  LLP 0105 withheld/visible/override from three caller contexts. Added
  to the release battery in AGENTS.md.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…he sidecar name

Findings from the neutral review of #954, all low severity, no behavior
change to the shipped surfaces:

- The roundtrip smoke's "newest visible hit leads" check accepted either
  sess-purged or sess-new, so a one-day sort inversion passed it. The seed
  dates make the answer exact; pin it to sess-new.
- The smoke's header claimed step 3 runs `hyp query maintain --force`,
  but the step calls `maintainCache` directly (it asserts on the sidecar
  counters in the returned report). Say what the code does.
- The post-shutdown telemetry assertions were wrapped in `step()`, which
  opens a root span against an already-shut-down provider: the smoke_step
  never reached the trace. Unwrapped, matching every sibling flow.
- `countIndexedDataFiles` restated the `<file>.index.parquet` pairing rule
  that `sidecarPathFor` already owns, in the one place a drift would
  silently misreport coverage. Import it instead.
- `search-sidecar-build.test.js` claimed indexedFileCount "stays absent
  elsewhere" without a partition where it could be absent. Add one.
- The hypaware-query skill pointed at `hyp query status` two paragraphs
  after routing cache operations to `hyp cache`. Both names work (alias);
  use the canonical one. The divergence fixture hashes host-only lines, so
  an identical edit to both copies leaves it unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…h completeness notices, and the two remote exceptions

The doc is the surface an agent acts from, so a caveat it omits becomes a
wrong call, not just a gap:

- `--limit` was named as the remedy for truncation with no ceiling. The
  verb accepts any integer through its schema and then silently
  substitutes the default for anything outside 1-1000, so `--limit 5000`
  returns 50, fewer than the caller asked for and with no signal. Stated.
- Only one of the verb's two stderr completeness notices was documented.
  `grep: the search stopped before covering every file` means the walk
  aborted, which a wider limit cannot fix; reading it as truncation sends
  a caller the wrong way. Both are now named and told apart.
- `--remote` was described as running "the same search", but a server
  restricts `--regex` to its operator and rejects `--include-local-only`
  outright. Both exceptions stated beside the flag.

Also aligns the new quick-reference line's trailing comment with its
neighbours (it sat one column right).

Both host copies carry identical edits; all four touched lines are shared,
so the host-divergence fixture is unchanged.
…: nine columns, tool_args moves to the not-searched list

The maintainer's unstick on PR #954 settled that #953 drops tool_args
from SEARCHABLE_COLUMNS. The verb's coverage clause interpolates the
constant so it follows on its own, and the query_grep_roundtrip smoke
never seeds or asserts a tool_args hit, so the SKILL.md enumeration
(both host copies, one shared line) was the only surface stating the
old set. The two deliberate host-only lines are untouched, so the
divergence fixture needs no re-record.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The T7 review commit that stopped restating the sidecar name imported
sidecarPathFor from sidecar_build.js, which is where it lived when T7
was branched. #953's own review round moved it beside GREP_DATASET in
searchable_columns.js, so re-stacking T7 on the current T6 tip left the
import naming an export that is no longer there: a typecheck error and
every maintenance test red. Point it at the module that exports it now.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Master's D1 short-flag gate (LLP 0293) walks every visible core command
and refuses an unknown `-Z`, sparing only `query sql`. `query grep` is
the second verb to bind a greedy positional, so integrating the two
branches trips that gate.

It belongs in the exemption rather than opting into strictness. LLP 0293
settles the mechanism ("the verb family never opts in"), and grep's
positional is search text, not a flag: a recorded transcript is mostly
command lines, so `-Z` or `--force` is an ordinary thing to search FOR.
Refusing it would make the one obvious way to find a flag in your own
history exit 2, which is the bargain `rg` already strikes with
`rg -- -Z`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two findings from the review of 237b081:

- Both `hypaware-query` SKILL.md copies told agents that `--limit 5000`
  is "silently replaced by the default rather than clamped, so it
  returns 50". The verb clamps: `Math.min(rawLimit, MAX_LIMIT)`, pinned
  by `test/core/query-grep-verb.test.js`. Only an unusable value (zero,
  negative, fractional) falls back to 50. An agent reading the old text
  would pick a limit it believed was 50 and silently over-fetch, or
  avoid the flag entirely.

- `executeGrepSearch` built its settle list from the spool walk PLUS
  partition discovery, which overlap by construction (the spool dir
  sits inside the partition dir). `settlePendingCacheForQuery` is
  per-entry, so a pending-but-debounced table pushed its "last write to
  query cache was N minutes ago" staleness line once per copy: grep
  printed it twice where sql prints it once. Deduped by table path.

Also corrects the indexed tier's comment, which claimed "the generator
is simply not pulled past the budget" when the loop drains it. It has
to: rows inside one file arrive in write order, so stopping at the
budget would keep that file's oldest matches. The sort-order trim on
`found` is what bounds the memory.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

neutral:adopt Foreign PR adopted into neutral's reconcile scope neutral:adopted Adoption completion record: merged while carrying neutral:adopt (LLP 0031)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants