diff --git a/.claude/skills/electron-ipc-architecture/SKILL.md b/.claude/skills/electron-ipc-architecture/SKILL.md new file mode 100644 index 00000000..bc5a63fc --- /dev/null +++ b/.claude/skills/electron-ipc-architecture/SKILL.md @@ -0,0 +1,50 @@ +--- +name: electron-ipc-architecture +description: Mental model for BrainWaves' Electron process separation and how data crosses the main/preload/renderer boundary. Read this BEFORE touching anything that spans processes — IPC, the preload bridge, native modules (liblsl/koffi), Bluetooth, filesystem writes, or "why can't the renderer just call X". Pair with electron-ipc-channel when actually adding/editing a channel. +--- + +# Electron IPC architecture (BrainWaves) + +Three processes, hard boundary, no shared memory. Everything crosses via IPC. + +``` +main (Node) preload (bridge) renderer (React) +───────────── ──────────────── ──────────────── +ipcMain.handle/on ◄── ipcRenderer.invoke/send ◄── window.electronAPI.* +webContents.send ──► ipcRenderer.on(handler) ──► onX(callback) +native modules contextBridge only no Node, no require +filesystem, dialogs (contextIsolation: true) Web Bluetooth lives here +``` + +## The one rule that explains most bugs + +The renderer has **no Node access** (`contextIsolation: true`, `nodeIntegration: false`). It cannot `require`, touch the filesystem, load native modules, or open dialogs. Any such need is a method on `window.electronAPI`, defined in `src/preload/index.ts`. If a renderer file imports `fs`/`path`-for-IO/`electron`, that's the bug — route it through the bridge instead. (`pathe` for pure path string munging is fine; it's not IO.) + +## Where each thing lives + +- `src/main/index.ts` — all `ipcMain.handle`/`ipcMain.on` registrations, the `BrowserWindow`, native module owners (LSL outlets/inlets), filesystem, dialogs, the EEG write streams (`activeStreams` map), the `pyodide://` protocol handler. +- `src/preload/index.ts` — the **only** file that calls `contextBridge.exposeInMainWorld`. Exposes `electronAPI` plus a couple of synchronous globals injected via `--resource-path`/`process.platform` (renderer module-level code reads `__ELECTRON_RESOURCE_PATH__`/`__ELECTRON_PLATFORM__`). `src/preload/viewer.ts` is the separate preload for the viewer window. +- `src/renderer/types/electron.d.ts` — the `ElectronAPI` TS interface. **Must** mirror the preload object or renderer calls won't type-check. These two files drift; keep them locked. +- `src/shared/lslTypes.ts` — types imported by BOTH main and renderer. The only sanctioned cross-process type sharing. + +## Three message directions — pick by shape, not habit + +| Need | Pattern | Example | +|------|---------|---------| +| Renderer asks main, wants a result/ack | `ipcRenderer.invoke` ↔ `ipcMain.handle` (Promise) | `lsl:discoverStreams`, every `fs:*` | +| Renderer fires hot/fire-and-forget data at main | `ipcRenderer.send` ↔ `ipcMain.on` (void) | `eeg:writeData`, `lsl:sendEpoch` | +| Main pushes to renderer (events, inlet data) | `mainWindow.webContents.send` → `ipcRenderer.on` | `lsl:inletData`, `lsl:status`, `oauth:callback` | + +Hot streaming paths (per-sample EEG, LSL epochs) deliberately use `send`, not `invoke` — a Promise per sample would swamp IPC. See `eeg:writeHeader`/`eeg:writeData` and `lslBridge.ts`'s batching (`batchSamplesToEpoch`, ~125 ms batches). + +## Main → renderer subscriptions must return an unsubscribe + +Every `onX` in the bridge registers an `ipcRenderer.on` listener and **returns a teardown** that calls `removeListener` (see `onLSLInletData`, `onOAuthCallback`). Renderer code must call it on cleanup or listeners leak across reconnects. Never expose a raw `ipcRenderer.on` without the teardown wrapper. + +## Native modules are main-only and load lazily + +liblsl (via `node-labstreaminglayer`/koffi) `dlopen`s at require time. It is loaded **lazily and fail-soft** in `src/main/lsl/native.ts` (`loadLSL()` returns module-or-null, memoized). A static `import` would crash the whole app at launch on machines without liblsl. The renderer feature-gates on the `lsl:isAvailable` probe and no-ops LSL calls when unavailable. Do not move native code toward the renderer or make it eager. (See `.llms/learnings.md` → "LSL is optional".) + +## When you change anything cross-process + +Trace the full chain end to end — a half-wired channel fails silently (renderer call resolves to `undefined`, or `send` lands on no handler with no error). Use `electron-ipc-channel` for the concrete checklist. diff --git a/.claude/skills/electron-ipc-channel/SKILL.md b/.claude/skills/electron-ipc-channel/SKILL.md new file mode 100644 index 00000000..d5aa5f76 --- /dev/null +++ b/.claude/skills/electron-ipc-channel/SKILL.md @@ -0,0 +1,60 @@ +--- +name: electron-ipc-channel +description: Step-by-step procedure for adding, editing, or removing an Electron IPC channel in BrainWaves without leaving it half-wired. Use whenever the renderer needs something only the main process can do (filesystem, dialogs, native/LSL, Bluetooth) or main needs to push events to the renderer. Covers the four files that must stay in sync and how to pick invoke vs send vs webContents.send. Read electron-ipc-architecture first for the mental model. +--- + +# Adding / editing an IPC channel (BrainWaves) + +A channel is correct only when **all four touch-points agree**. Skip one and it fails silently. + +## 1. Pick the direction and primitive + +- **Renderer needs a value or an ack from main** → `invoke` ↔ `handle` (returns a Promise). Default choice for anything request/response: all `fs:*`, `dialog:*`, `lsl:discoverStreams`, `lsl:isAvailable`. +- **Renderer fires hot, high-frequency data and doesn't need a reply** → `send` ↔ `on` (void). Only for streaming paths: `eeg:writeData`, `lsl:sendEpoch`, `lsl:sendMarker`. Don't use `send` just to avoid `async` — you lose error propagation and ordering guarantees. +- **Main pushes events to the renderer** (inlet data, status, OAuth) → `webContents.send` → bridge `onX(callback)` wrapper. + +Channel naming: `namespace:verb` (e.g. `fs:readFiles`, `lsl:sendEpoch`). Match the existing groups in `src/main/index.ts`. + +## 2. The four files (keep locked) + +### a. `src/main/index.ts` — register the handler +```ts +// request/response +ipcMain.handle('fs:doThing', (_event, arg: string) => doThing(arg)); +// fire-and-forget +ipcMain.on('eeg:writeData', (_event, streamId, data) => { /* ... */ }); +``` +Wrap native/LSL handlers in try/catch and surface failures via `emitLSLStatus(...)` (don't throw across IPC for the hot paths). For main→renderer pushes, send on `mainWindow?.webContents.send('lsl:status', payload)`. + +### b. `src/preload/index.ts` — expose it on `electronAPI` +```ts +doThing: (arg: string): Promise => ipcRenderer.invoke('fs:doThing', arg), +// fire-and-forget returns void: +writeEEGData: (streamId: string, data: unknown): void => + ipcRenderer.send('eeg:writeData', streamId, data), +``` +For a main→renderer push, expose a subscription that **registers a listener and returns a teardown** — copy the shape of `onLSLInletData`: +```ts +onThing: (handler: (p: Payload) => void): (() => void) => { + const listener = (_e: unknown, p: Payload) => handler(p); + ipcRenderer.on('ns:thing', listener); + return () => ipcRenderer.removeListener('ns:thing', listener); +}, +``` + +### c. `src/renderer/types/electron.d.ts` — add the matching signature +The `ElectronAPI` interface must mirror b exactly. This is the file most often forgotten — without it the renderer call is `any`/untyped or a type error. `invoke` → `Promise`; `send` → `void`; subscription → `(handler) => () => void`. + +### d. Renderer caller — use it +Call `window.electronAPI.doThing(...)`. Shared payload types go in `src/shared/lslTypes.ts` (or a sibling shared file) so main and renderer import the same type. For subscriptions, store the returned teardown and call it on unmount/cleanup (or in the epic's teardown) — see how `lslBridge.ts` / device epics consume `onLSLInletData`. + +## 3. Verify the whole chain + +- `npm run typecheck` — catches preload↔`electron.d.ts` drift. +- Confirm the channel string is **byte-identical** in all three of: `handle`/`on`, `ipcRenderer.*`, and any `webContents.send`. A typo here is the classic silent failure (call resolves `undefined`, or `send` hits no handler with no error). +- Hot path? Confirm it's batched/throttled, not per-sample `invoke` (see `lslBridge.ts` batching, `lsl:status` 5s throttle). +- Native/LSL? Confirm the renderer side feature-gates on `isLSLAvailable()` and no-ops when false, and that the handler tolerates `loadLSL()` returning null. + +## 4. Removing a channel + +Delete from all four files. Grep the channel string repo-wide before declaring it gone — leftover `ipcRenderer.on` listeners leak; leftover `electron.d.ts` entries lie about the API. diff --git a/.claude/skills/pyodide-mne/SKILL.md b/.claude/skills/pyodide-mne/SKILL.md new file mode 100644 index 00000000..50bf7c54 --- /dev/null +++ b/.claude/skills/pyodide-mne/SKILL.md @@ -0,0 +1,58 @@ +--- +name: pyodide-mne +description: How BrainWaves runs MNE-Python analysis in-browser via Pyodide (WASM) inside a web worker. Use when adding/changing analysis, adding a Python package, debugging Pyodide load failures (esp. prod-only), routing plots back to the UI, or anything touching src/renderer/utils/webworker or InstallMNE.mjs. Covers the JS↔worker↔Python message protocol, plot routing, package install, and the pyodide:// protocol. +--- + +# Pyodide + MNE in BrainWaves + +MNE-Python runs as WASM inside a **web worker**, driven from redux-observable epics. You almost never touch Python execution directly — you `postMessage` a Python string and receive results on the worker's `message` event. + +## The pieces + +- `src/renderer/utils/webworker/webworker.js` — the worker. Boots Pyodide, then `self.onmessage` runs `pyodide.runPythonAsync(data)` and posts back `{ results, plotKey, error }`. **Do not edit `webworker/src/`** — it's generated by `InstallPyodide.mjs` / `InstallMNE.mjs`. +- `src/renderer/utils/webworker/index.ts` — the JS API. Each export wraps a Python string in `worker.postMessage({ data, plotKey?, csvArray? })`. This is where you add/edit analysis calls. +- `webworker/utils.py` — the actual analysis (`load_data`, `get_raw_epochs`, plotting). Testable against **native** MNE (see `tests/analysis/`); keep one implementation so app and tests don't drift. +- `webworker/patches.py` — monkeypatches applied at boot (`apply_patches()`). +- `src/renderer/epics/pyodideEpics.ts` — orchestrates: launch worker, send commands, receive messages, dispatch Redux actions. + +## The message protocol (the whole contract) + +Outbound (JS → worker): `worker.postMessage({ data: "", plotKey?, csvArray? })`. +- `data` is a Python string (or `\n`-joined lines). Its last expression is the return value. +- `csvArray` rides alongside as a global the Python reads (`load_data()` pulls `js.csvArray`). +- Append `;` to a statement to suppress a return you don't want marshalled back. + +Inbound (worker → JS, on the `message` event): `{ results, plotKey, error }`. `pyodideMessageEpic` switches on `plotKey`: +- `'ready'` → `SetWorkerReady` +- `'topo' | 'psd' | 'erp'` → `Set*Plot` with `results` (an SVG string) wrapped as a MIME bundle +- `error` set → toast + `ReceiveError` +- else → `ReceiveMessage` + +## Adding an analysis step + +1. Write/extend the function in `utils.py` so it's importable and **native-MNE testable** (no `js.*` globals in the core; factor those to the boundary, like `load_data(csv_strings=...)`). +2. Add a thin wrapper in `webworker/index.ts` that `postMessage`s the call. +3. Drive it from an epic in `pyodideEpics.ts` (`tap(() => wrapperFn(worker))` then `mergeMap(() => EMPTY)` for fire-and-forget; results arrive async on the message event). +4. Add a test under `tests/analysis/` (runs real MNE via pytest, CI in `analysis.yml`). + +## Plots are async and fire-and-forget + +`worker.postMessage()` returns `undefined` — you cannot `await` a plot. Pattern: +- Epic: `tap(() => plotX(worker))` then `mergeMap(() => EMPTY)` — emit nothing. +- The Python renders with `agg` (NOT WebAgg — it needs `js.document` and throws in a worker), `fig.savefig(buf, format="svg")`, returns the SVG string with a `plotKey`. +- The worker echoes `plotKey`; `pyodideMessageEpic` routes it to the right `Set*Plot` action; `PyodidePlotWidget` renders the MIME bundle. + +Always `plt.close(_fig)` after saving — leaked figures accumulate in the long-lived worker. + +## Adding a Python package + +Edit `internals/scripts/InstallMNE.mjs` (runs on `postinstall`), not the worker. Pyodide binary packages come from the Pyodide CDN; pure-Python wheels from PyPI → written to `webworker/src/packages/` with a `manifest.json`. **Pure-Python transitive deps are NOT auto-resolved** — list them explicitly (jinja2, markupsafe, decorator, requests + certifi/charset-normalizer/idna/urllib3 are already there because MNE/pooch/matplotlib need them at import). CDN version derives from `node_modules/pyodide/package.json`, not the lock file's dev label. + +## Loading gotchas (mostly prod-only — see .llms/learnings.md for full detail) + +- Assets serve over a custom `pyodide://` Electron protocol (registered in `src/main/index.ts`), because Vite's SPA fallback returns `index.html` for worker `fetch()`s. Dev root `webworker/src/`; prod root `resourcesPath/pyodide/` (the `extraResources` dest name **must** match the protocol handler). +- Prod needs **both** `indexURL` and `packageBaseUrl` set to the protocol base — `import.meta.url` resolution fails once Vite bundles `pyodide.mjs` away from its siblings. +- Worker must be `type: 'module'`; `checkIntegrity: false` (lock hashes ≠ CDN wheels); `optimizeDeps.exclude: ['pyodide']`. +- `micropip` rejects custom schemes — wheels are JS-fetched via the protocol, written to the emscripten FS, installed via `emfs:///tmp/...`. + +If Pyodide works in `npm run dev` but breaks in `npm run package`, it's almost always asset paths / protocol dest-name / `indexURL`. diff --git a/.claude/skills/redux-observable-epochs/SKILL.md b/.claude/skills/redux-observable-epochs/SKILL.md new file mode 100644 index 00000000..3526ad7d --- /dev/null +++ b/.claude/skills/redux-observable-epochs/SKILL.md @@ -0,0 +1,63 @@ +--- +name: redux-observable-epochs +description: How EEG epochs and markers flow through BrainWaves' redux-observable (RxJS epic) pipeline — from live device samples, through marker injection, to MNE epoching for ERPs. Use when adding/debugging an epic, touching the raw/signal observables, marker injection, the marker registry, or "why are my epochs empty / markers all zero / ERP has no events". Explains the epic anatomy and the numeric-code contract that links collection to analysis. +--- + +# Epochs & markers via redux-observable + +Two distinct "epoch" worlds, joined by **numeric marker codes**: +1. **Live epochs** — RxJS observables of raw EEG samples from a device, recorded to CSV and optionally forwarded to LSL. +2. **Analysis epochs** — MNE `Epochs` built by slicing the recorded raw around marker events for ERP/topo/PSD. + +Get the codes wrong and the data looks fine but ERPs are empty. That contract is the heart of this skill. + +## Epic anatomy (the pattern to copy) + +Side effects live in epics (`redux-observable`), never in reducers/components. Standard shape (see `src/renderer/epics/`): + +```ts +const fooEpic: Epic = (action$, state$) => + action$.pipe( + filter(isActionOf(SomeActions.Trigger)), // gate on a typed action + pluck('payload'), // or map to read payload + mergeMap(/* async side effect */), // do the work + map(SomeActions.Done) // dispatch result action(s) + ); +``` + +Rules: +- Gate with `filter(isActionOf(...))` (`../utils/redux`), not string compares. +- Read current state via `state$.value.*` (e.g. `state$.value.pyodide.worker!`). +- An epic **must** return an action stream. Fire-and-forget (plots, worker pushes): `tap(() => effect())` then `mergeMap(() => EMPTY)` — emit nothing rather than leak a value. +- Register the epic in the file's `combineEpics(...)` and the file in `epics/index.ts`. An unregistered epic silently never runs. +- Long-lived subscriptions (device `disconnect$`, worker `message`/`error` via `fromEvent`) are created inside an epic and live for the app session — see `pyodideMessageEpic`/`pyodideErrorEpic`. + +## Live epoch flow (device → recording → LSL) + +`deviceEpics.ts` resolves the active backend via `getDriver(deviceType)` (never branches on MUSE/NEUROSITY). Key epics: +- `setRawObservableEpic` — `from(getDriver(dt).createRawObservable())` produces the per-sample EEG stream stored in state. +- Recording writes samples to CSV via the `eeg:writeData` IPC (main holds the write stream). +- `lslForwardEpic` + `lslBridge.batchSamplesToEpoch` — batch ~32 samples and forward to the main-process LSL outlet (only when `isLSLAvailable()`). + +Markers are injected device-agnostically: the UI calls `injectMarker()`, which delegates to the active driver's `injectMarker` (set on connect via `setActiveDriver`). Every driver implements the `EEGDriver` interface (`utils/eeg/types.ts`) — a device can't ship without a marker path. (LSL inlet is intentionally out of the driver registry; its `injectMarker` no-ops.) + +## The marker-code contract (read this before debugging empty ERPs) + +Markers in the CSV `Marker` column are **numeric** EVENTS codes (`stimulus.type`, e.g. `STIMULUS_1 = 1`), 1-based — **not** strings, not array indices. MNE's `find_events` reads them off the last (`stim`) channel. + +`buildMarkerRegistry(stimuli)` (`utils/eeg/markerRegistry.ts`) is the **single source of truth**, used by BOTH: +- collection — the CSV codes + the `-events.json` sidecar (`eeg:writeEvents` IPC), and +- analysis — the MNE `event_id` map. + +In `pyodideEpics.loadEpochsEpic`, `event_id` is derived from `buildMarkerRegistry(...).eventId`, NOT from `{title: arrayIndex}`. The classic bug: a 0-based index map didn't match the 1-based codes in the data, so code-2 epochs matched no `event_id` and MNE raised "No matching events". If you add/reorder stimuli or a new device, route everything through `buildMarkerRegistry` — don't hand-build code maps anywhere. + +## Analysis epoch flow (pyodideEpics) + +`LoadEpochs` → read CSVs → `loadCSV` → `filterIIR(1, 30)` → build `event_id` from the registry → `epochEvents(worker, eventId, tmin, tmax)` (wraps `get_raw_epochs` in `utils.py`) → `GetEpochsInfo`. Plotting/cleaning epics consume `raw_epochs`/`clean_epochs`. The actual MNE work happens in the Pyodide worker — see the `pyodide-mne` skill for the worker protocol. + +## Debugging checklist + +- **Markers all zero in CSV** → the device's `injectMarker` isn't wired, or `setActiveDriver` didn't run on connect. +- **"No matching events" / empty ERP** → `event_id` not derived from `buildMarkerRegistry`; codes (1-based) vs map mismatch. +- **Epic never fires** → not in `combineEpics` / `epics/index.ts`, or the `filter(isActionOf(...))` targets the wrong action. +- **Value leaks downstream** from a side-effect epic → use `mergeMap(() => EMPTY)`, not `map`.