Skip to content

feat: add visual regression tests for the React Native samples - #4

Draft
mika-f wants to merge 2 commits into
mainfrom
claude/vrt-module-implementation-5ad03d
Draft

mika-f wants to merge 2 commits into
mainfrom
claude/vrt-module-implementation-5ad03d

Conversation

@mika-f

@mika-f mika-f commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

What

Adds visual regression tests (VRT) for the React Native Storybook app in packages/native-samples, using @natsuneko-laboratory/react-native-visual-regression-test. Every story is rendered on a real iOS simulator and Android emulator, captured natively, and compared with the baselines in packages/native-samples/__vrt__/<platform>/.

  • pnpm vrt / vrt:ios / vrt:android run the tests while the app is running; pnpm vrt:update updates baselines.
  • A new VRT workflow runs both platforms on PRs that touch the renderer, the samples, the fixtures, or the lockfile. Failed runs upload expected | actual | diff images as artifacts. Running the workflow manually with update checked uploads baselines taken on CI instead.

Changes in the sample app

  • .rnstorybook/preview.tsx
    • Wraps each story in a VrtBoundary (withVrt), which captures only the message and not the background.
    • Prefetches the remote images in each Flex Message and holds the capture until they are cached.
    • Renders stories in a vertical ScrollView, as in a chat. Carousels (horizontal ScrollViews) now keep their natural height instead of stretching to the screen.
  • .rnstorybook/index.ts
    • Starts the VRT agent in __DEV__.
    • Ignores the Reanimated "Dependencies should only be used on the web" warning, which comes from the on-device UI. On Android, its LogBox toast would show up in screenshots.
  • main now points at .rnstorybook/index.ts. The expo-router/entry entry had no app/ routes, so Storybook never started.
  • app.json gets the iOS bundle ID and Android package name that expo run:* added. Change them if you want different identifiers.
  • The VRT module has native code, so it needs a development build (pnpm ios / pnpm android). Expo Go does not work.

Baselines

Platform Device Taken on
iOS iPhone 17 Pro, iOS 26.5 Local simulator. The same device/runtime exists on the macos-26 runner.
Android Pixel 10 Pro, Android 17 (API 37.2) Local arm64 emulator. CI uses an x86_64 image because Android emulators do not run on Apple Silicon runners.

Notes for reviewers

  • Android baselines may need to be regenerated on CI. The CI emulator is x86_64 and the local one is arm64, so rendering may differ slightly. If the Android job fails only on antialiasing-like diffs, run the workflow with update after merging (workflow_dispatch needs the file on main), then commit the vrt-android-baselines artifact.
  • Rare flake. Locally, Android passed 21 of 22 compare runs. The failure was a 31% diff on Restaurant that looked like a hero image not yet drawn, and it never reproduced, cold starts included. If it shows up on CI, raising settleMs for Android is the next step.
  • No Gradle cache. gradle/actions/setup-gradle v6 makes caching a proprietary component that requires accepting Gradle's Terms of Use. It is left out; add it if the org accepts those terms.
  • Pinned actions. All actions are pinned to SHAs with pinact run -u.
  • iOS 27. The Expo 57 prebuilt app crashes on launch on the iOS 27 simulator because it does not adopt the UIScene lifecycle. That is unrelated to VRT and is being handled separately. VRT uses iOS 26.5.
  • Existing CI is untouched. on-pull_request-ci.yml references a missing .node-version and an undefined pnpm lint.
  • Extra commit. This branch also carries the existing be832a5 fix: fix commit (a one-line change to Apparel.stories.tsx), which is not on main yet.

🤖 Generated with Claude Code

mika-f and others added 2 commits September 27, 2026 21:10
Capture every Storybook story of native-samples on an iOS simulator and an
Android emulator with @natsuneko-laboratory/react-native-visual-regression-test,
and compare them with baselines in __vrt__/<platform>.

- Start the VRT agent and wrap stories in a VrtBoundary; hold captures until
  the remote images in each Flex Message have been prefetched.
- Render stories in a vertical ScrollView so that carousels keep their natural
  height, and silence the Reanimated warning whose LogBox toast would show up
  in Android screenshots.
- Point the app entry at .rnstorybook/index.ts, since there is no app/ router.
- Add a VRT workflow for pull requests, with a manual mode that uploads new
  baselines taken on CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant