feat(vscode): report test run progress #720

claneo · 2025-11-27T02:24:19Z

Changes Summary

core
1. Support pass logger to default reporter
2. Add two new reporter hooks: onTestSuiteStart and onTestSuiteResult
3. Support match test case with testNamePattern exactly (This is still a temporary solution that cannot handle cases where the name contains >. I plan to support matching by arrays instead of strings in the future.)
4. Stop stringify actual and expected value in error object
vscode extension
1. New features, see below
2. Simplified updateFromContents, now ancestors stack is poped when exit astnode
3. Enqueued all test items before run them

New Features

Realtime test result
Pipe default reporter to test results panel
Place error message at right location
Display error stack if exist
Display expected and actual value if it is a mismatch error
Support update snapshot if it is a snapshot mismatch error

20251127-095853.mp4

Problems

Sourcemap does not work if debugger attached, causes error stack is wrong
Is it ok to import core source code from vscode extension directly?

Checklist

Tests updated (or not required).
Documentation updated (or not required).

netlify · 2025-11-27T02:24:25Z

✅ Deploy Preview for rstest-dev ready!

Name	Link
🔨 Latest commit	`618f9f5`
🔍 Latest deploy log	https://app.netlify.com/projects/rstest-dev/deploys/6929bab8731d600008b657fb
😎 Deploy Preview	https://deploy-preview-720--rstest-dev.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot

Pull request overview

This PR implements real-time test run progress reporting for the VSCode extension, enabling features like live test results, error location highlighting, and snapshot updating. The changes span both the core testing framework (adding new reporter hooks onTestSuiteStart and onTestSuiteResult) and the VSCode extension (implementing a comprehensive test run reporter with error formatting and stack trace parsing).

Key changes include:

Added suite-level reporter hooks to track test suite execution lifecycle
Implemented exact test name pattern matching with configurable delimiter
Refactored VSCode extension to report progress in real-time via new TestRunReporter class
Enhanced error reporting with actual/expected value formatting using pretty-format
Added support for updating snapshots directly from VSCode test results

Reviewed changes

Copilot reviewed 43 out of 44 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
packages/core/src/types/reporter.ts	Added `onTestSuiteStart` and `onTestSuiteResult` hooks, logger option for default reporter
packages/core/src/types/testSuite.ts	Extracted `TestSuiteInfo` type, added `expected`/`actual` fields to `FormattedError`
packages/core/src/types/config.ts	Added `exact` flag for test name pattern matching
packages/core/src/runtime/runner/runner.ts	Implemented suite result tracking and hook invocations
packages/core/src/runtime/runner/task.ts	Added exact matching logic for test name patterns
packages/core/src/utils/error.ts	Made `getSourcemap` parameter optional
packages/vscode/src/testRunReporter.ts	New class implementing real-time test progress reporting with error formatting
packages/vscode/src/worker/reporter.ts	Refactored to use new progress-based reporter architecture
packages/vscode/src/worker/index.ts	Updated to pass reporter and logger instances to rstest
packages/vscode/src/master.ts	Added `onTestProgress` method to handle worker progress events
packages/vscode/src/testTree.ts	Simplified test tree parsing with exit callbacks, removed old result handling
packages/vscode/src/extension.ts	Added snapshot update command and improved test run workflow
packages/vscode/package.json	Added `pretty-format` dependency and snapshot update command configuration
packages/vscode/tests/suite/progress.test.ts	New integration test for progress reporting functionality
e2e/reporter/rstest.customReporterConfig.ts	Added suite hooks to test reporter
pnpm-lock.yaml	Updated dependencies (@rslib/core, @rsbuild/core, @rspack versions)

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-28T14:55:28Z

packages/vscode/tsconfig.json

    "strict": true,
-    "skipLibCheck": true
+    "skipLibCheck": true,
+    "types": ["../core/src/env.d.ts"]


The types compiler option expects an array of package names or paths to type definition files, not a path to a .d.ts file. This should likely be removed and handled differently, or you should use typeRoots instead. The current configuration will likely not work as intended.

Copilot · 2025-11-28T14:55:29Z

packages/vscode/src/master.ts

 import path, { dirname } from 'node:path';
 import { createBirpc } from 'birpc';
 import vscode from 'vscode';
+import { TEST_DELIMITER } from '../../core/src/utils/constants';


This import uses a relative path '../../core/src/utils/constants' to access core package source directly, creating tight coupling between packages. Consider exporting TEST_DELIMITER from the @rstest/core package's main entry point for proper separation of concerns.

Suggested change

import { TEST_DELIMITER } from '../../core/src/utils/constants';

import { TEST_DELIMITER } from '@rstest/core';

Copilot · 2025-11-28T14:55:29Z

packages/vscode/tests/suite/progress.test.ts

+import * as vscode from 'vscode';
+import { getTestItems, waitFor } from './helpers';
+
+suite('Configuration Integration', () => {


The test suite is named 'Configuration Integration', but the test is actually testing progress/error reporting functionality, not configuration. Consider renaming to something like 'Test Progress Reporting' or 'Test Run Reporter Integration' to better reflect what's being tested.

Copilot · 2025-11-28T14:55:29Z

packages/vscode/tests/suite/progress.test.ts

+import { getTestItems, waitFor } from './helpers';
+
+suite('Configuration Integration', () => {
+  test('respects rstest.testFileGlobPattern (array-only)', async () => {


The test is named 'respects rstest.testFileGlobPattern (array-only)', but it doesn't appear to test any glob pattern or configuration behavior. It's testing test result reporting, error messages, and snapshot functionality. Consider renaming to something like 'reports test progress with error details and snapshots'.

Copilot · 2025-11-28T14:55:30Z

packages/core/src/types/config.ts

   */
  testNamePattern?: string | RegExp;
+  /**
+   * Match `testNamePattern` exactly, test suite and case names should joined by '>'


The comment states that names should be joined by '>', but based on the code in runtime/runner/task.ts and the constant TEST_DELIMITER, the actual delimiter is ' > ' (with spaces). The documentation should accurately reflect this: "test suite and case names should be joined by ' > ' (with spaces)".

Suggested change

* Match `testNamePattern` exactly, test suite and case names should joined by '>'

* Match `testNamePattern` exactly, test suite and case names should be joined by ' > ' (with spaces)

Copilot · 2025-11-28T14:55:30Z

packages/vscode/src/testRunReporter.ts

+
+    const testItem = this.findTestItem(result);
+    if (!testItem) {
+      logger.error('Cannot find testItem', test);


Variable name mismatch: The error log references test, but the parameter is named result. This should be logger.error('Cannot find testItem', result);

Copilot · 2025-11-28T14:55:30Z

packages/vscode/src/testRunReporter.ts

+import { ROOT_SUITE_NAME } from '../../core/src/utils/constants';
+import { parseErrorStacktrace } from '../../core/src/utils/error';


This import uses a relative path '../../core/src/utils/constants' to access core package source directly. While the PR description mentions this as a known concern ("Is it ok to import core source code from vscode extension directly?"), this creates tight coupling and build order dependencies. Consider either:

Exporting these constants from @rstest/core package's main entry point

Duplicating the constant in the vscode extension

Creating a shared constants package

The same applies to the import on line 12 (parseErrorStacktrace).

Copilot · 2025-11-28T14:55:31Z

packages/vscode/src/master.ts

+  async onTestProgress<T extends keyof TestRunReporter>(
+    runId: string,
+    event: T,
+    param: Parameters<NonNullable<TestRunReporter[T]>>[0],
+  ) {
+    const reporter = this.testRunReporters.get(runId);
+    // @ts-expect-error


The @ts-expect-error comment suppresses type checking without explanation. Consider adding a comment explaining why this is necessary, or better yet, fixing the type issue. The problem likely stems from TypeScript's inability to ensure param matches the expected parameter type for the specific event method.

Suggested change

async onTestProgress<T extends keyof TestRunReporter>(

runId: string,

event: T,

param: Parameters<NonNullable<TestRunReporter[T]>>[0],

) {

const reporter = this.testRunReporters.get(runId);

// @ts-expect-error

// Map each method name to its first parameter type

type TestRunReporterEventParams = {

[K in keyof TestRunReporter]: TestRunReporter[K] extends (arg: infer P, ...args: any[]) => any ? P : never;

};

async onTestProgress<T extends keyof TestRunReporter>(

runId: string,

event: T,

param: TestRunReporterEventParams[T],

) {

const reporter = this.testRunReporters.get(runId);

Copilot · 2025-11-28T14:55:31Z

e2e/rstest.config.ts

 export default defineConfig({
  setupFiles: ['../scripts/rstest.setup.ts'],
-  testTimeout: process.env.CI ? 10_000 : 5_000,
+  testTimeout: 10_000,


The change removes the conditional timeout based on CI environment (process.env.CI ? 10_000 : 5_000) and sets a fixed 10-second timeout. This may slow down local development since tests will now wait up to 10 seconds even in non-CI environments. Consider keeping the conditional logic to maintain faster feedback in local development.

Suggested change

testTimeout: 10_000,

testTimeout: process.env.CI ? 10_000 : 5_000,

github-actions bot added the change: feat label Nov 27, 2025

claneo force-pushed the vscode-progress branch 2 times, most recently from c6411a8 to eb7033a Compare November 27, 2025 15:43

feat(vscode): report test run progress

fc55505

claneo force-pushed the vscode-progress branch from eb7033a to fc55505 Compare November 28, 2025 14:44

claneo marked this pull request as ready for review November 28, 2025 14:49

Copilot AI review requested due to automatic review settings November 28, 2025 14:49

Copilot started reviewing on behalf of claneo November 28, 2025 14:50 View session

Copilot finished reviewing on behalf of claneo November 28, 2025 14:53

Copilot AI reviewed Nov 28, 2025

View reviewed changes

fix copilot review

618f9f5

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

feat(vscode): report test run progress #720

feat(vscode): report test run progress #720

Uh oh!

claneo commented Nov 27, 2025 •

edited

Loading

Uh oh!

netlify bot commented Nov 27, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Copilot AI Nov 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	import { TEST_DELIMITER } from '../../core/src/utils/constants';
	import { TEST_DELIMITER } from '@rstest/core';

	* Match `testNamePattern` exactly, test suite and case names should joined by '>'
	* Match `testNamePattern` exactly, test suite and case names should be joined by ' > ' (with spaces)

		import { ROOT_SUITE_NAME } from '../../core/src/utils/constants';
		import { parseErrorStacktrace } from '../../core/src/utils/error';

	testTimeout: 10_000,
	testTimeout: process.env.CI ? 10_000 : 5_000,

Uh oh!

feat(vscode): report test run progress #720

Are you sure you want to change the base?

feat(vscode): report test run progress #720

Uh oh!

Conversation

claneo commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes Summary

New Features

Problems

Related Links

Checklist

Uh oh!

netlify bot commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for rstest-dev ready!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

claneo commented Nov 27, 2025 •

edited

Loading

netlify bot commented Nov 27, 2025 •

edited

Loading