fix(picker_ui): highlight fuzzy query matches on file picker

[pompos02]

Summary

Improves file picker query highlighting so it better matches fuzzy-search behavior.

Changes

• Split queries into whitespace-separated tokens
• Match tokens case-insensitively
• Highlight exact token matches when available
• Fall back to ordered subsequence highlighting
• Merge adjacent and overlapping highlight ranges (to make extmarks job easier)
• Limit processed tokens and ranges to keep rendering bounded

Motivation

Previously, highlighting only searched for the full query as one exact literal string. This meant multi-token queries, case differences, and fuzzy-style matches were often not highlighted making it difficult to see if the match is actually correct (your scoring made it almost always correct)

Performance

• Highlighting is applied only to the entries currently rendered in the picker UI (if I understand correctly)
• The per-row work is also bounded by processing at most 8 query tokens and collecting at most 16 highlight ranges before applying extmarks.

Showcase

Before

After

[dmtrKovalenko]

wait this is how it works rn for myself

[dmtrKovalenko]

@gustav-fff review I do not undrestand what is wrong here?

[gustav-fff]

[triage-bot] DIRECTED:

Old highlighter (main, file_renderer.lua:202-212):

local match_start, match_end = string.find(line_content, ctx.query, 1, true)

Plain-text, case-sensitive, whole query as one literal. No fallback.

Old code fails to highlight in three cases the PR fixes:

1. Multi-token query. Query foo bar never finds a literal space in a path like src/foo/bar.rs, so nothing highlights. New code splits on %S+ and matches each token.
2. Case mismatch. Query Readme against README.md — old code misses (string.find with plain=true is case-sensitive). New code lowercases both sides.
3. No literal substring but fuzzy match. Query pcui scores well against picker_ui.lua via the Rust fuzzy scorer, so it appears in results, but old code finds no "pcui" substring and highlights nothing. New code falls back to greedy subsequence highlighting.

Your screenshot probably shows a single lowercase token that appears as a literal substring in the path — that's the one intersection where old and new produce identical extmarks. Try Readme or foo bar against a matching path on main to see the gap.

Caveat worth flagging: the subsequence fallback (add_subsequence_ranges) picks positions greedily left-to-right in Lua. It's not the same alignment the SIMD scorer chose (Rust does Smith-Waterman traceback in score.rs). For most queries the visual result is close enough, but it can highlight different characters than the scorer actually rewarded. The clean fix would be plumbing match_indices/match_ranges through file_item_into_lua in crates/fff-nvim/src/lua_types.rs:64 (grep already does this at :257) instead of recomputing in Lua. Out of scope for this PR unless you want it here.

Honk-Honk 🪿

[dmtrKovalenko]

wait mr @gustav-fff are we not using byte offsets from the fuzzy matcher when rendering the highlights?!

[pompos02]

As our friend Gustav said,

The change is only for the file picker UI, not the grep picker(your screenshot)

It modifies lua/fff/picker_ui/file_renderer.lua, which is used to render file-picker rows and their query highlights.

The grep picker (that you showed) has a separate renderer in lua/fff/picker_ui/grep_renderer.lua; that code does not compute fuzzy character matches itself, it only consumes item.match_ranges and applies extmarks for those ranges. Those ranges are produced from the Rust side., unlike the File picker that the extmarks were handled in the lua side.

So this PR fixes fuzzy query highlighting for the file picker list in lua, grep match highlighting already gets its match spans from Rust.

[dmtrKovalenko]

I see sorry yes I'm so dumb today (have a fever) can you please hide this behind a config option and keep it by default disabled

Thanks

[gustav-fff]

[triage-bot] DIRECTED:

On byte offsets: no, the file picker is not using them. SearchResultLua::into_lua at crates/fff-nvim/src/lua_types.rs:130-157 sends items + scores to Lua, no match_ranges field — unlike GrepResultLua which does set match_ranges from m.match_byte_offsets at :257. So file_renderer.lua has to recompute matches from ctx.query against the rendered line. pompos02's summary above is correct.

Piping frizbee's indices through file_item_into_lua is the right long-term fix but out of scope here.

On gating: I can't push (cross-repo fork). Inline diff below — adds file_picker.highlight_matched_query = false and wraps the highlight block. Two hunks:

--- a/lua/fff/conf.lua
+++ b/lua/fff/conf.lua
@@ -414,6 +414,8 @@
     -- find_files settings
     file_picker = {
       current_file_label = '(current)',
+      -- Highlight fuzzy query matches in rendered file rows (experimental; may highlight different characters than the scorer rewarded).
+      highlight_matched_query = false,
     },

--- a/lua/fff/picker_ui/file_renderer.lua
+++ b/lua/fff/picker_ui/file_renderer.lua
@@ -200,7 +200,8 @@ function M.apply_highlights(item, ctx, item_idx, buf, ns_id, line_idx, line_content)
   end

   -- 9. Query matches
-  if ctx.query and ctx.query ~= '' then
+  local hl_matched_enabled = ctx.config.file_picker and ctx.config.file_picker.highlight_matched_query
+  if hl_matched_enabled and ctx.query and ctx.query ~= '' then
     local matched_hl = ctx.config.hl.matched or 'IncSearch'

@pompos02 if you're happy with this shape, apply on top of the branch and I'll re-review. README file_picker config section doesn't currently exist so no docs update needed unless you want to add one.

Honk-Honk 🪿

[pompos02]

I see sorry yes I'm so dumb today (have a fever) can you please hide this behind a config option and keep it by default disabled

Thanks

Sorry to hear that man, I made the changes and moved this behind a file_picker.fuzzy_query_highlighting option, disabled by default.

[pompos02]

Hey,
Moved file picker fuzzy highlighting to the Rust side.

File search now computes match_byte_offsets for the paginated file results using neo_frizbee, similar to the fuzzy grep path. I kept the final offset-mapping logic in Lua because Rust knows offsets in relative_path, but the UI renders a different string: filename first, then directory path. So Lua maps the relative-path byte ranges onto the visible filename dir_path segments (keeping the mapping close to the renderer makes it easier to keep correct on feature UI features)

Notes:

• Ranges are computed only for the returned page items, not for every matched candidate.
• The range pass uses the same query parts and typo behavior as the scorer as closely as possible.
• The UI only maps directory highlights when the directory path was not shortened, because shortened paths cannot be mapped safely back to original byte offsets.

Advise:

• Changed the SearchResult struct that is public.
• We compute match_bytes_offsets also on MixedSearchResult
• This still does a second neo_frizbee pass for indices after scoring. That avoids storing heavy match-index data for all candidates, but it means highlighting is not literally reusing the original scorer’s Match.
• The Rust side computes match_byte_offsets even when file_picker.fuzzy_query_highlighting = false doing unnecessary work (I suggest that this should default to true)