Skip to content

fix(pi-fff): make home-dir scanning configurable, warn when indexing $HOME (#743) - #749

Merged
dmtrKovalenko merged 2 commits into
mainfrom
triage-bot/issue-743
Aug 7, 2026
Merged

fix(pi-fff): make home-dir scanning configurable, warn when indexing $HOME (#743)#749
dmtrKovalenko merged 2 commits into
mainfrom
triage-bot/issue-743

Conversation

@gustav-fff

Copy link
Copy Markdown
Collaborator

Closes #743

Root cause

pi-fff hardcoded enableHomeDirScanning: true, overriding the core default guard, so launching pi from $HOME always indexed the whole home tree with no opt-out:

  • packages/pi-fff/src/index.ts:370 — main finder
  • packages/pi-fff/src/aux-finders.ts:77 — aux finder pool

Only fff-enable-root-scan was configurable. @XWIlluDelu traced this to #589, which removed the home flag and forced true. The maintainer's advice "do not start pi in $HOME if you enable home indexing" was unfollowable — the extension always enabled it.

Fix

Expose it as --fff-enable-home-scan flag + FFF_ENABLE_HOME_SCAN env, default true (behavior unchanged). resolveBoolOpt now takes a fallback and accepts 0/false so the default can be turned off. Threaded through both the main finder and AuxFinderPool.

On the alerts question: pi exposes ctx.ui.notify(msg, "info" | "warning" | "error") and ctx.ui.setStatus(id, text) (footer, persists across renders) — docs extensions.md:2177, tui.md:734. Wired both: a warning notify when cwd is $HOME with scanning on, plus a footer status. After the 15s waitForScan returns, getScanProgress() is polled once; if still scanning, the footer keeps showing the live file count instead of clearing, so a long background index over a big home tree is visible.

Note: this only adds the opt-out and visibility. It does not address the other items in the report — no idle-stop, no size/count thresholds, no default excludes, no cross-process index sharing. @dmtrKovalenko those are design changes, out of scope for a triage fix.

Steps to reproduce

Pre-fix main, home scanning cannot be disabled:

git clone git@github.com:dmtrKovalenko/fff.git && cd fff
git checkout 086044f
grep -n "enableHomeDirScanning" packages/pi-fff/src/index.ts packages/pi-fff/src/aux-finders.ts
# packages/pi-fff/src/index.ts:370:        enableHomeDirScanning: true,
# packages/pi-fff/src/aux-finders.ts:77:      enableHomeDirScanning: true,

No flag exists, so launching pi from $HOME indexes the whole tree:

cd $HOME && pi   # full $HOME walk, no way to opt out

Behavioral check via the test suite:

cd packages/pi-fff && bun test test/

On pre-fix main add this test — it FAILS (Expected: false, Received: true):

test("FFF_ENABLE_HOME_SCAN=0 disables home dir scanning", async () => {
  process.env.FFF_ENABLE_HOME_SCAN = "0";
  await start();
  const opts = createCalls[0] as { enableHomeDirScanning: boolean };
  expect(opts.enableHomeDirScanning).toBe(false);
});

Post-fix it passes, and the opt-out works:

cd $HOME && FFF_ENABLE_HOME_SCAN=0 pi   # native Home guard rejects init, no scan
cd $HOME && pi                          # still scans (default true) + warning shown

How verified

  • bun test test/ in packages/pi-fff46 pass, 0 fail.
  • Confirmed the new test gates the fix: reverting enableHomeDirScanning to hardcoded true makes it fail with Expected: false, Received: true.
  • bun run lint and bun run format:check produce byte-identical output to origin/main (3 warnings, 1 info, 8 format diffs — all pre-existing). No new lint/format regressions.
  • ctx.ui.notify / ctx.ui.setStatus / getScanProgress verified against installed @earendil-works/pi-coding-agent 0.79.3 docs and packages/fff-node/src/fff-api.ts:603. setStatus called optionally (?.) since it is TUI/RPC-only.
  • Zero Rust changes, no hot path touched.

Automated triage via Gustav. Honk-Honk 🪿

@dmtrKovalenko dmtrKovalenko left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gustav-fff iterate on revie

Comment thread packages/pi-fff/src/index.ts Outdated

// Warn when launched from $HOME with home scanning on: indexing a large
// home tree can run for a long time in the background (issue #743).
const atHome =

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's cache the os.homedir once per the process


export interface AuxOpts {
enableFsRootScanning: boolean;
enableHomeDirScanning?: boolean;

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every time the llm is trying to spawn the aux picker in home dir we should send the notification to the user

Comment thread packages/pi-fff/src/index.ts Outdated
"FFF is indexing your entire home directory ($HOME) in the background — this can take a while on large trees. Disable with --fff-enable-home-scan=false (or FFF_ENABLE_HOME_SCAN=0).",
"warning",
);
ctx.ui.setStatus?.("fff", "FFF: indexing $HOME");

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ctx.ui.setStatus?.("fff", "FFF: indexing $HOME");
ctx.ui.setStatus?.("fff", "Agent is indexing $HOME, this can lead to high CPU");

Comment thread packages/pi-fff/src/index.ts Outdated

// Clear the status once the initial scan settles; if it is still running
// past the wait, keep the user informed that it continues in background.
if (atHome) {

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you sure this is actually getting called multiple times? I feel like it would be called just once so we might need to run timeout based scanning or add a callback to the ensureFinder to make sure this warning is cleaned

dmtrKovalenko pushed a commit that referenced this pull request Aug 5, 2026
…tatus

Review follow-ups on #749:

- Resolve os.homedir() once per process in src/paths.ts (HOME_DIR) instead
  of on each call; aux-finders and index both use it.
- AuxOpts.onHomeDirScan fires whenever the agent spawns an aux picker rooted
  at or above $HOME, so the user is notified every time, not only at startup.
- Replace the one-shot post-waitForScan progress check with a 1s poller.
  waitForScan() also resolves on timeout, so a single check could leave a
  stale footer forever. The interval is unref'd and cleared on
  session_shutdown and in destroyFinder().
- Footer text now reads "Agent is indexing $HOME, this can lead to high CPU".
@gustav-fff

Copy link
Copy Markdown
Collaborator Author

[triage-bot] DIRECTED: pushed f83f9f2. All three review points addressed.

1. Cache os.homedir() — new packages/pi-fff/src/paths.ts: HOME_DIR resolved once at module load, isHomeDir(dir) helper. Both src/index.ts:632 and the ~-expansion in src/aux-finders.ts:160 now use it. import os dropped from index.ts.

2. Notify on every aux picker over $HOMEAuxOpts.onHomeDirScan?: (root: string) => void, fired in AuxFinderPool.acquire() before FileFinder.create() when the requested root covers $HOME:

if (enableHomeDirScanning && rootCovers(maybeRoot, HOME_DIR)) {
  this.opts.onHomeDirScan?.(maybeRoot);
}

Wired to the same warnHomeDirScan() the startup path uses. Fires only on picker creation, not on covering-picker reuse, and not for roots below $HOME (e.g. ~/dev/foo) — those do not walk the home tree.

3. You are right, the one-shot check was wrong. waitForScan() resolves on timeout too, so on a big $HOME the single post-wait getScanProgress() could pin a stale footer forever with no path to clearing it. Replaced with a 1s poller (trackHomeScanStatus(), src/index.ts:432) that updates the live file count and clears the status once isScanning goes false. The interval is unref()'d so it never holds the process open, and stopHomeScanStatus() runs from destroyFinder() — so session_shutdown also tears it down.

4. Footer text taken verbatim from your suggestion: Agent is indexing $HOME, this can lead to high CPU.

Tests: 53 pass, 0 fail (was 46). 7 new covering the aux notification, the polling loop with isScanning: true, shutdown cleanup, and silence when FFF_ENABLE_HOME_SCAN=0. biome check output is identical to origin/main modulo line-number shifts on the two pre-existing findings. Zero Rust changes.

Honk-Honk 🪿

dmtrKovalenko pushed a commit that referenced this pull request Aug 7, 2026
…tatus

Review follow-ups on #749:

- Resolve os.homedir() once per process in src/paths.ts (HOME_DIR) instead
  of on each call; aux-finders and index both use it.
- AuxOpts.onHomeDirScan fires whenever the agent spawns an aux picker rooted
  at or above $HOME, so the user is notified every time, not only at startup.
- Replace the one-shot post-waitForScan progress check with a 1s poller.
  waitForScan() also resolves on timeout, so a single check could leave a
  stale footer forever. The interval is unref'd and cleared on
  session_shutdown and in destroyFinder().
- Footer text now reads "Agent is indexing $HOME, this can lead to high CPU".
@dmtrKovalenko

Copy link
Copy Markdown
Owner

@gustav-fff rebase

…$HOME (#743)

Expose home-dir scanning as --fff-enable-home-scan flag + FFF_ENABLE_HOME_SCAN
env, default true. resolveBoolOpt takes a fallback and accepts 0/false. Threaded
through the main finder and AuxFinderPool.

Cache os.homedir() once per process in src/paths.ts. AuxOpts.onHomeDirScan fires
whenever the agent spawns an aux picker rooted at or above $HOME. On session
start from $HOME, notify the user and track scan progress with a 1s poller that
clears the footer once the scan settles; the interval is unref'd and cleared on
shutdown and in destroyFinder().
@gustav-fff

Copy link
Copy Markdown
Collaborator Author

[triage-bot] DIRECTED: rebased onto main (3a0ce85), force-pushed 2baa107.

Two commits squashed into one. Conflict was in packages/pi-fff/src/index.ts#750 (695724e) reformatted the whole file. Re-applied my changes verbatim on top of the new structure; semantic diff vs main is unchanged.

aux-finders.ts auto-merged with #750's new create(root) split, but the merge left my onHomeDirScan guard referencing the old maybeRoot var which no longer exists in that scope — fixed to root:

if (enableHomeDirScanning && rootCovers(root, HOME_DIR)) {
  this.opts.onHomeDirScan?.(root);
}

Tests: 55 pass, 0 fail (was 53; picked up #750's 2 aux-dedup tests). No new tsc errors — the 5 tsc --noEmit findings (@ff-labs/fff-node module resolution, implicit-any on item/p at index.ts:230/241) are all pre-existing on main, none in my touched lines.

Honk-Honk 🪿

@dmtrKovalenko

Copy link
Copy Markdown
Owner

@copilot resolve the merge conflicts in this pull request

Co-authored-by: dmtrKovalenko <16926049+dmtrKovalenko@users.noreply.github.com>

Copilot AI commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

@copilot resolve the merge conflicts in this pull request

Rebased onto main (031005e) and resolved the conflict in packages/pi-fff/src/index.ts. The conflict was a whole-file clash with the reformatting from #751. Re-applied all Gustav changes (home-scan flag, resolveBoolOpt fallback, enableHomeDirScanning variable, warnHomeDirScan, trackHomeScanStatus/stopHomeScanStatus, uiCtx capture, and the session_start warning block) on top of the new reformatted structure. Commit: 88cb82b.

Copilot AI requested a review from dmtrKovalenko August 7, 2026 02:55
@dmtrKovalenko
dmtrKovalenko merged commit 7af3a9d into main Aug 7, 2026
48 of 49 checks passed
abhijit-s pushed a commit to abhijit-s/fff that referenced this pull request Aug 10, 2026
Upstream (6 commits): rescan throttling for giant folders (dmtrKovalenko#751), configurable
$HOME scanning (dmtrKovalenko#743/dmtrKovalenko#749), Lua-level fs-root/home guard (dmtrKovalenko#745/dmtrKovalenko#748), pi-fff
SDK import cache (dmtrKovalenko#758), release 0.10.3.

Conflict resolution:
- Cargo.toml/Cargo.lock (all crates): keep fork's 0.17.2 version scheme over
  upstream's 0.10.3; preserve fff-mcp's fff-ipc/dirs/libc deps and upstream's
  new rescan-stats feature.
- Makefile: union of fork's daemon/install targets and upstream's
  test-rescan/rescan-probe targets.
- install-mcp.sh: keep fork's removal of the pinned-release SHA block
  (delivery is via Homebrew tap + apt, not pinned GitHub tarballs).
- background_watcher.rs: adopt upstream's throttled try_trigger_full_rescan
  mechanism wholesale, keeping only the fork's per-root user_gi ignore filter;
  drop the obsolete need_full_rescan boolean.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pi-fff: eager full-$HOME background index at session start causes multi-hour disk I/O storm (no exclusions, no idle-stop)

3 participants