Some search topics reach SQLite FTS as malformed query syntax instead of being handled as text. Search also needs representative query/document fixtures so changes can be checked for useful results, not only successful execution.
The input failures were reproduced against a built systemd documentation package. The relevant search implementation is unchanged from upstream commit 149f112.
Using the existing database setup in search.test.ts:
search(db, "ExecStart"); // works when the fixture contains ExecStart
search(db, '"ExecStart'); // throws: unterminated string
search(db, "AND"); // throws: fts5: syntax error near "AND"
buildQuery() removes some punctuation but preserves double quotes and operator-like words. searchFts() then passes that string directly to MATCH. The SQL statement is parameterized; the problem is handling the FTS query language inside the search value.
Proposed behavior:
Define whether ordinary topics are literal text or explicitly supported advanced syntax. For the normal topic interface, quote/tokenize terms safely so unmatched quotes and reserved words cannot cause an unhandled SQLite error. If phrase syntax remains supported, define its validation and error behavior.
Add a small, deterministic retrieval fixture suite built through the real ingestion pipeline. Include expected documents for topics such as ExecStart=, systemctl --user, dotted configuration names, and language-specific queries. #133's GDScript experiments illustrate why this matters: plausible matches from another language can dominate the intended documentation.
Acceptance criteria:
- Unmatched quotes, FTS operator words, punctuation-only input, and empty topics have defined behavior and do not produce an unhandled SQLite exception.
- Existing ordinary keyword searches and any deliberately supported phrase searches retain their behavior.
- Representative queries assert expected documents within a documented top-k result set.
- Fixtures check useful document attribution and preservation of relevant code examples within the result budget.
- Retrieval checks use local fixtures so CI is independent of changing remote documentation.
The existing BM25 implementation can remain the baseline; this issue establishes safe input handling and evidence for subsequent ranking changes.
Some search topics reach SQLite FTS as malformed query syntax instead of being handled as text. Search also needs representative query/document fixtures so changes can be checked for useful results, not only successful execution.
The input failures were reproduced against a built systemd documentation package. The relevant search implementation is unchanged from upstream commit
149f112.Using the existing database setup in
search.test.ts:buildQuery()removes some punctuation but preserves double quotes and operator-like words.searchFts()then passes that string directly toMATCH. The SQL statement is parameterized; the problem is handling the FTS query language inside the search value.Proposed behavior:
Define whether ordinary topics are literal text or explicitly supported advanced syntax. For the normal topic interface, quote/tokenize terms safely so unmatched quotes and reserved words cannot cause an unhandled SQLite error. If phrase syntax remains supported, define its validation and error behavior.
Add a small, deterministic retrieval fixture suite built through the real ingestion pipeline. Include expected documents for topics such as
ExecStart=,systemctl --user, dotted configuration names, and language-specific queries. #133's GDScript experiments illustrate why this matters: plausible matches from another language can dominate the intended documentation.Acceptance criteria:
The existing BM25 implementation can remain the baseline; this issue establishes safe input handling and evidence for subsequent ranking changes.