testing.md

Testing (vpr test + Playwright)

• Test files live in test/*.test.ts; NOT alongside source.
• To unit-test a route action directly (no browser), type the args object as Parameters<typeof action>[0]: the generated +types/<route> module only resolves via tsconfig rootDirs for relative imports inside app/, so ~/routes/+types/… does NOT resolve from test/. See test/api-inbound-push.test.ts for the pattern (mock the route's network-facing collaborators with vi.mock, keep real decryption).
• The Fastmail flows are unit-tested offline: test/fastmail-send.test.ts drives sendEmailViaJmap through its injectable JmapSendDeps (identity match → upload → import → submit; the submit step retries once on a transient failure with the same email id; failures return false), and test/api-inbound-push.test.ts forges RFC 8291 payloads encrypted to throwaway keys from a vi.mock'd ~/lib/env (generated lazily in getters, because vi.hoisted runs before node:crypto initializes), so the decrypt cases run everywhere, CI included (PushVerification echo, StateChange drain, unknown type, undecryptable body).
• Browser tests via Playwright (vitest provider); helpers in test/helpers/ (launchBrowser.ts → goto/signIn, seedTestData.ts → testPrisma + seeded constants, launchServer.ts). goto reuses one browser + signed-in context per test file (each file runs in its own vitest fork) and skips the /login flow when the expense_session cookie is present; tests that need a logged-out or multi-account session drive their own browser directly. The post-hydration settle sleep is 150ms (was 500ms); the 100ms waitForEditorSettle beat in test/expenses.test.ts is separate.
• Use testPrisma for DB assertions; seed with the account/user constants.
• Requires local Postgres (expense_test); the schema is force-reset each run.
• The suite runs on a pinned clock: 2026-07-15T12:00:00Z, ticking in real time from that base, shared by the test process, browser pages, and the test server (test/helpers/frozen-time.ts, pinned-time.ts, pinned-clock.mjs, launchBrowser.ts::freezePageClock). Consequence for test writers:
  • Date.now() / new Date() return pinned base + elapsed; NEVER use them for real-time deadlines or polling loops (they hang); use performance.now() (see launchServer.ts, auth.test.ts polling).
  • Derive webhook timestamps (svix-timestamp) from Date.now(), not absolute wall-clock values, so they stay inside the server's replay guard.
  • The browser pin is a context.addInitScript Date override, NOT Playwright's page.clock (setFixedTime never fires page timers; install() doesn't survive navigations).
  • The server pin loads via NODE_OPTIONS=--import ./test/helpers/pinned-clock.mjs in launchServer.ts; keep the pinned instant in sync across all files.
  • The test server blanks PUBLIC_URL in its env (like the live-service keys) so the OAuth metadata issuer follows the request/forwarded origin. A .env PUBLIC_URL (set for the Fastmail push URL) changed the issuer to the prod origin locally and prompted a bad hardcode in 531e814; keep OAuth test expectations origin-derived, never hardcoded.
• React Router remounts route components when a data navigation settles (~1ms after the URL changes): the create-editor mounts, unmounts, and remounts, replacing the file input in between. A Playwright setInputFiles landing in that window sets files on the instance about to be torn down and the change event is silently lost (no request ever fires; the suite caught this as a flaky 30s waitForResponse timeout). Upload tests must wait for the settle before setting files: waitForEditorSettle(page) (test/expenses.test.ts), a 100ms beat that lands well past the ~1ms remount. Drag-drop uploads are exempt (the editor's useEffect upload fires from mount #1 and draftUploadsInFlight dedupes the remount).

Property-based tests

• The unit project carries property-based suites (fast-check) alongside its hand-written cases. A property lives in the mirror-named test file for the module (app/lib/csv.ts → test/csv.test.ts), inside one appended describe("… properties") block, so the hand-written cases stay as they are and a new file still has to be registered in both vitest.unit.config.ts include and vitest.main.config.ts exclude.
• Every property goes through assertProperty from test/helpers/property.ts, so the run count and the seed live in one place: 200 runs, seed 1, endOnFailure: true. A failure prints the seed and a shrunk counterexample and replays identically from the seed alone. Drop PROPERTY_RUNS to 50 for a slow property, with a comment saying why; never raise the 10s timeout.
• Give every fc.string() an explicit maxLength. fast-check does not bound string length by default, and an unbounded input to parseCsv or parseMarkdown is how a property test eats the per-test timeout.
• Use the helper's arbitraries rather than a local one: text(n) for general input, binary(n) for the full code-point range (what breaks a naive char loop), validDate for a real calendar date, and moneyAmount for a money-shaped decimal. text mixes in ASCII on purpose — unit: "binary" alone draws uniformly from all 1.1M code points, so a digit lands in roughly 1 character in 100,000 and a property keyed on digits, quotes or commas passes vacuously. The same trap is a matter of frequency, not alphabet: a generated row matched another about 4% of the time, so the duplicate-detection properties ran mostly against empty match lists. When a property needs a specific shape to exist at all (a real link, a bracketed email, a parseable amount, a row that genuinely duplicates another), seed it into the input or derive it from the module's own output rather than hoping the generator produces one.
• Assert the contract, not the implementation. A property that reimplements the module's branch (deriving the same regex, or the same accept/reject decision, and comparing) only proves the code was copied correctly. State the guarantee instead: a fixed point, an agreement between two surfaces, a bound that holds at its edges. When a case is a table over one field with no generated input involved, write it as a plain loop rather than wrapping it in assertProperty for decoration.
• A property suite that asserts nothing is worse than none. When you add one, break a single assertion, confirm the file fails with a shrunk counterexample, and put it back.
• When a property fails, decide in this order: the counterexample violates a contract the module documents → fix the app in the same change and keep the property; the property is wrong about the code (it contradicts a documented behavior, or asserts something never promised) → fix the property and say why in a comment. Never delete or weaken a property to reach green. (parseAmount's finiteness guard is the worked example: the exponent bound could not see "NaN" or "Infinity", which decimal.js accepts with e set to NaN.)

Landing demo (pnpm demo)

• The home-page hero is a video of the app filing a receipt: public/demo-receipt.mp4, public/demo-receipt.webm, and public/demo-receipt-poster.webp (the still the page paints first and shows instead when a visitor asks for reduced motion), all committed. pnpm demo rebuilds them: it seeds the stills' dataset, drives the real flow in Chromium, and encodes with ffmpeg, so ffmpeg has to be on PATH.
• The capture is a frame sequence with explicit holds rather than a screen recording (test/helpers/demoRecord.ts): each state gets the frames it deserves and ffmpeg joins them from a concat list, which is what makes the cut reproducible. Every shot() waits two animation frames first, because a state React has committed is not necessarily painted: an earlier cut was missing the drop highlight entirely, its one held frame having beaten the paint.
• The demo asserts what it shows rather than trusting the eye: 11 list rows before the drop and 12 after, the outline class up while the file hovers, and the merchant and amount the extraction filled. The page it records is the same seeded dataset the stills use (seedScreenshotData).
• The receipt is a generated PDF with a text layer and its extraction is a warmed cache row (warmDemoExtraction). The model call is the one step the suite cannot make (no outbound network, see app/lib/env.ts), and the cache is what the app reads on a re-upload anyway. The key comes from the app's own reader, sha256("txt:" + extractPdfText(pdf)), so a drifted key would leave the demo's fields empty; the warm reads back before recording.
• The landing baseline in screenshots/landing.png is captured with reducedMotion: "reduce": a playing video is never the same twice, so the suite compares the still the page falls back to, which keeps that path covered as well.
• The demo's duration is the sum of its holds, so tune the storyboard in test/screenshot.test.ts, not the encoder. Both formats are kept because Safari still prefers the mp4; the poster is a WebP of the video's first frame, so the still and frame one are the same image.

Screenshot regression (suite screenshots)

• Every pnpm test run captures the app's important screens and the emails it sends (test/screenshot.test.ts, "suite screenshots" block) and compares each against the committed baseline in screenshots/ via matchBaseline (test/helpers/toMatchScreenshot.ts, looks-same, ΔE tolerance 2.3) — a plain function that throws, not an expect.extend matcher: a matcher name the runner does not know resolves to a native no-op, which passed without comparing anything. Any drift fails the run and leaves screenshots/<name>.new.png (the new capture) and screenshots/<name>.diff.png next to the baseline; all drifted screens are reported in one failure, not just the first.
• Baselines are committed. When a capture changes on purpose, review the drift with pnpm screenshots:review (adapted from rentail's scripts/screenshots.ts): it serves a side-by-side UI on :3456, keyboard driven (A accept / S skip / Esc close), and Accept replaces the baseline with the new capture; the server stops itself after the last item. Baselines that were edited by hand show up in the review as "modified" (compared against the Git HEAD version, Revert/Keep).
• Comparisons are skipped when CI is set, so CI never fails on environment-rendering noise; they only run locally. A shell that exports CI (some local setups do) therefore passes the whole block without comparing anything — run CI= pnpm exec vp test run test/screenshot.test.ts when you actually want the check.
• Determinism guards: the test server pins the random content on captured screens (SCREENSHOT_HIGHLIGHT_PIN, set by launchServer.ts): the home page's "Did you know?" highlight pick and the insights page's opening starter both read it, so both capture the first eligible entry. Any new random content on a captured screen must read it too; the rotation is untouched in every other environment. The matcher also waits for document.fonts.ready (a late-loading webfont otherwise flags phantom text drift), and the suite's pinned clock keeps client-rendered dates stable. If you add a screen capture, expect a baseline-creation pass first (Baseline screenshot created) and matching runs after.