Skip to content

Repository files navigation

STAF.Playwright.Tests

Sample implementation of the STAF.Playwright framework — a .NET test automation framework for web and API testing using Microsoft Playwright and MSTest. This repository provides a ready-to-run project so you can clone, restore, and execute tests with minimal setup.

Using AI to write tests? Start with AGENTS.md (cross-tool entry), then AI-assisted automation (copy-paste prompts) for @/attach bundles and ready-made prompts.


STAF.Playwright framework

This project implements and demonstrates the STAF.Playwright NuGet package:

STAF (Simple Test Automation Framework) provides base classes, page object support, HTML reporting with screenshots, and configuration for multi-environment test runs. It targets .NET 10 and uses MSTest and Microsoft Playwright.


What this sample includes

  • UI testsBaseTest and BasePage (browser setup, page objects, step reporting)
  • API testsTestBaseAPI and ApiClient with a free GET API sample (JSONPlaceholder)
  • Contract tests – OpenAPI contract validation via OpenApiContractTestBase and specs in OpenAPI/
  • ExcelExcelDriver sample (create, read, write, compare workbooks)
  • HTML reports – Per-test and final aggregated report (ResultTemplateFinal.html) in TestResults, with screenshots on failure
  • Configurationtestsetting.runsettings and optional testdata.json with environment support
  • AI agents & skills – Cursor project skills, GitHub Copilot instructions, and Visual Studio custom agents (STAF UI Automation, STAF API Automation, STAF Contract Automation, STAF QA Orchestrator). See AI agents and skills.
  • AI QA Orchestrator (work items) – STLC-aligned playbook and phase markdown reports under QA/work-items/ (Azure DevOps / Jira via MCP when configured, or pasted requirements). See Work-item / PBI QA (orchestrator).
  • MCP servers – Bundled Playwright C# and Azure DevOps MCP under MCPAgent/ for AI-assisted authoring and live work-item fetch. See Using MCP servers.

Prerequisites

  • .NET 10 SDK
  • Chrome or Edge (or another Chromium-based browser) installed on your machine. This project uses the browser already on your system (via Playwright’s channel support); you do not need to run playwright.ps1 install.
    If you later switch configuration to use Playwright’s bundled Chromium, Firefox, or WebKit, see Playwright browsers for installation.

Quick start

  1. Clone and restore

    git clone https://github.com/sooraj171/STAF.Playwright.Tests.git
    cd STAF.Playwright.Tests
    dotnet restore
  2. Run tests

    dotnet test --settings STAF.Playwright.Tests/testsetting.runsettings

    Or from the test project:

    cd STAF.Playwright.Tests
    dotnet test --settings testsetting.runsettings
  3. View results
    HTML reports (including ResultTemplateFinal.html) and screenshots are written under TestResults in the test project output (e.g. STAF.Playwright.Tests/bin/Debug/net10.0/TestResults).


AI-assisted automation (copy-paste prompts)

Use this when you want Cursor, VS Code (Copilot), or Visual Studio (Copilot) to generate STAF.Playwright tests and page objects without hunting for file paths.

1. Open the right folder

Open the repository root (the folder that contains STAF.Playwright.Tests.sln, AI/, and .cursor/) as your workspace. Paths below are relative to that root.

2. Give the model the handbook (recommended)

Editor What to do
Cursor Always-on: .cursor/rules/staf-playwright-framework.mdc. Skills: .cursor/skills/MASTER.md (stubs → AI/). Attach @AI/instructions/system-prompt.md, generation-rules.md, and skill files as needed. Entry: AGENTS.md.
VS Code Repo instructions: .github/copilot-instructions.md. Attach handbook files from .vscode/staf-ai/INDEX.md. Setup: .vscode/README.md.
Visual Studio .github/copilot-instructions.md + custom agents in .github/agents/ (@staf-ui-automation, @staf-api-automation, @staf-contract-automation, @staf-qa-orchestrator). MCP: .mcp.json.

Suggested bundles

  • UI automation: VS → @staf-ui-automation · Cursor → staf-ui-testing · Attach: AI/instructions/system-prompt.md, AI/instructions/generation-rules.md, AI/skills/ui-testing.md, AI/skills/reporting.md, AI/skills/test-data.md, AI/skills/framework-rules.md
  • API automation: VS → @staf-api-automation · Cursor → staf-api-testing · same instructions + AI/skills/api-testing.md
  • Contract tests: VS → @staf-contract-automation · Attach: AI/skills/framework-rules.md + golden Tests/ContractTests.cs
  • After a failure: AI/instructions/debugging-rules.md + the skill for the layer that failed (e.g. AI/skills/ui-testing.md)
  • Work item / PBI (STLC reports): VS → @staf-qa-orchestrator · Cursor → staf-qa-orchestrator · Attach: AI/instructions/qa-orchestrator-lifecycle.md, AI/instructions/work-item-report-templates.md, AI/skills/qa-orchestrator.md. Configure Azure DevOps MCP (setup); if unavailable, paste the description and acceptance criteria.

See AI agents and skills for the full agent and skill reference.

3. Copy a sample prompt (fill in the brackets)

New UI flow (Page Object + test)

You are working in the STAF.Playwright.Tests repository. Follow the attached AI/instructions and AI/skills files exactly.

Feature / user story:
[Describe the screen, role, and what the user should achieve—e.g. login, submit form, verify message.]

Requirements:
- UI test class: inherit BaseTest from STAF.Playwright.Framework; use ConfigManager for URLs and parameters (testsetting.runsettings), no hardcoded environments.
- Page Object: under Pages/, inherit BasePage; all ILocator and interactions live there; use WaitForElementVisibleAsync, EnterTextAsync, PressAsync, and ReportResult for steps.
- The test method must NOT use raw Playwright (no Page.Locator / Click / Fill in the test class)—only call page object methods.
- Add MSTest assertions and reporting for important steps.
- If you add new URLs, keys, or test data, update testsetting.runsettings and/or testdata.json and show the code that reads them via ConfigManager.

Deliver: (1) Page class(es) as needed (2) Test class under Tests/ (3) config/test data updates (4) short comments only where logic is non-obvious.

New API tests

Repository: STAF.Playwright.Tests. Follow the attached AI handbook files.

API scenario:
[Describe endpoints, methods, auth, expected status codes, and what to assert in the body.]

Requirements:
- Test class inherits TestBaseAPI; use ApiClient and ReportResultAPI from the framework—no ad-hoc HttpClient.
- Base URL from ConfigManager GetParameter("ApiBaseUrl") / runsettings.
- Include happy path and at least one negative or edge case if applicable.
- Use async/await consistently with existing samples.

Deliver: test class(es) under Tests/, and any runsettings/testdata updates.

OpenAPI / contract coverage

STAF.Playwright.Tests repo. Follow attached AI/skills/framework-rules.md and framework conventions.

Contract goal:
[Describe which API surface or spec to validate.]

Requirements:
- Use OpenApiContractTestBase; OpenApiSpecFolder points at specs under OpenAPI/ copied to output (match existing ContractTests pattern).
- Use framework helpers RunAllContractTestsAsync / AssertAllContractTestsPassed—do not reimplement contract validation from scratch.

Deliver: test class and any new/updated OpenAPI JSON under OpenAPI/ as needed.

Debug a failed run

STAF.Playwright.Tests. Follow AI/instructions/debugging-rules.md.

Failure summary:
[Paste assertion message, stack trace, or describe flaky behavior.]

Task: identify likely root cause (config, selector, timing, data, parallel conflict), suggest a concrete fix (files/methods to change), and say what to re-run (single test command with testsetting.runsettings).

Excel / workbook scenario

STAF.Playwright.Tests. Follow AI/skills/framework-rules.md and existing Excel samples under Tests/Excel/.

Scenario:
[Describe create/read/compare or data setup with Excel files.]

Requirements:
- Use ExcelDriver from STAF.Playwright.Framework.Excel (CreateWorkbook, Save, Open, CompareFiles, etc.)—do not manipulate OpenXML or COM Excel directly unless the framework cannot cover the case.
- Keep tests under Tests/Excel/ or follow the existing namespace/folder pattern in this repo.
- Use temp paths or isolated files so parallel runs do not clash.

Deliver: test class changes and any notes on required input files (prefer generating files in-test).

Work item / PBI — full QA cycle (markdown reports)

STAF.Playwright.Tests repo. Follow the QA Orchestrator skill / AI/instructions/qa-orchestrator-lifecycle.md and work-item-report-templates.md.

Work item:
[ADO/Jira ID or URL, or paste title + description + acceptance criteria here.]

Task: Run the lifecycle — fetch (or use paste), analysis, test design, test cases with AC traceability, code review if application code is in this workspace, test execution strategy, then summary. Write one markdown file per phase under QA/work-items/{Provider}-{WorkItemId}/ (01-pbi-fetch.md through 07-summary-report.md) using the templates.

When implementing automation afterward, follow generation-rules.md and the UI/API skills as usual.

4. Run what was generated

From the repository root:

dotnet test --settings STAF.Playwright.Tests/testsetting.runsettings

From STAF.Playwright.Tests/ (test project folder):

dotnet test --settings testsetting.runsettings

Run a single test (example — replace with your test class and method):

dotnet test --settings STAF.Playwright.Tests/testsetting.runsettings --filter "FullyQualifiedName~YourTestClass.YourTestMethod"

Then open TestResults under the test project output folder for HTML and screenshots (see Quick start step 3).


AI agents and skills

This repo ships a unified AI playbook under AI/ plus editor-specific entry points. Canonical rule text lives only in AI/; Cursor skills under .cursor/skills/ are stubs that point there.

Start here: AGENTS.md · Quick tasks: AI/instructions/QUICK_START.md · Setup by editor: AI/instructions/ai-setup.md

Visual Studio custom agents (GitHub Copilot)

In Visual Studio 2026 (18.4+) with GitHub Copilot, pick a specialized agent from the agent picker or type its handle in chat:

Agent Handle Use for
STAF UI Automation @staf-ui-automation UI tests, page objects (BaseTest, BasePage, ReportResult)
STAF API Automation @staf-api-automation REST tests (TestBaseAPI, ApiClient, ReportResultAPI)
STAF Contract Automation @staf-contract-automation OpenAPI contract tests (OpenApiContractTestBase, specs in OpenAPI/)
STAF QA Orchestrator @staf-qa-orchestrator PBI / work-item STLC — phase reports under QA/work-items/

Agent definitions: .github/agents/ · Shared Copilot rules: .github/copilot-instructions.md

Cursor project skills

Cursor discovers skills from .cursor/skills/*/SKILL.md. Index: .cursor/skills/MASTER.md. Reference by name in Composer (e.g. "using staf-ui-testing, create…") or attach the matching AI/skills/*.md file.

Skill Use for Canonical file
staf-ui-testing UI tests, page objects AI/skills/ui-testing.md
staf-api-testing API tests AI/skills/api-testing.md
staf-framework-rules Layout, naming, parallel safety AI/skills/framework-rules.md
staf-reporting HTML step reports AI/skills/reporting.md
staf-test-data Runsettings, testdata.json AI/skills/test-data.md
staf-db-testing DB validation AI/skills/db-testing.md
staf-qa-orchestrator Work-item / PBI QA cycle AI/skills/qa-orchestrator.md
staf-ai-instructions Generation + debugging playbook AI/instructions/
staf-ai-context Minimize tokens — which @ files to attach AI/instructions/ai-setup.md

Always-on Cursor rules: .cursor/rules/staf-playwright-framework.mdc · .cursor/cursor.rules

VS Code (GitHub Copilot)

VS Code does not use the .github/agents/ files directly. Instead:

  1. Repo instructions load from .github/copilot-instructions.md.
  2. Attach handbook files listed in .vscode/staf-ai/INDEX.md.
  3. Setup notes: .vscode/README.md.

For work-item QA, attach AI/instructions/qa-orchestrator-lifecycle.md and AI/skills/qa-orchestrator.md (same content the STAF QA Orchestrator VS agent uses).


Work-item / PBI QA (orchestrator)

For end-to-end QA from a PBI, User Story, or Bug (not only “write one test”), use the STAF QA Orchestrator agent or skill:

Tool How to invoke
Visual Studio Copilot Agent picker or @staf-qa-orchestrator.github/agents/staf-qa-orchestrator.agent.md
Cursor Skill staf-qa-orchestrator or @.cursor/skills/staf-qa-orchestrator/SKILL.md
VS Code Copilot Attach AI/instructions/qa-orchestrator-lifecycle.md + AI/skills/qa-orchestrator.md (see .vscode/staf-ai/INDEX.md)

Playbook and templates:

Doc Purpose
AI/instructions/qa-orchestrator-lifecycle.md Phases: fetch → analysis → design → cases → optional code review → execution strategy → summary; Azure DevOps / Jira via MCP when available, or pasted requirements
AI/instructions/work-item-report-templates.md Markdown skeletons for each report file
AI/skills/qa-orchestrator.md Skill entry: when to use, checklist, STAF reminders
.cursor/skills/staf-qa-orchestrator/SKILL.md Cursor stub pointing at the files above

Outputs: phase reports under QA/work-items/{Provider}-{WorkItemId}/ (01-pbi-fetch.md07-summary-report.md). See QA/README.md. These are separate from framework HTML test reports under TestResults.

MCP: This repo ships Playwright C# and Azure DevOps MCP servers under MCPAgent/. Configure ADO with a PAT to fetch work items live; otherwise paste the item into chat (documented as a gap in 01-pbi-fetch.md). See Using MCP servers.

A ready-made prompt is in AI-assisted automation under Work item / PBI — full QA cycle.


Project layout

Path Purpose
STAF.Playwright.Tests.sln Visual Studio / dotnet solution (open this or the repo folder)
STAF.Playwright.Tests/ Test project
STAF.Playwright.Tests/AssemblyInit.cs Assembly init/cleanup; delegates to framework for final HTML report
STAF.Playwright.Tests/Tests/Test1.cs UI sample test inheriting BaseTest
STAF.Playwright.Tests/Tests/ApiTests.cs API sample (TestBaseAPI, ApiClient) – JSONPlaceholder GET
STAF.Playwright.Tests/Tests/ContractTests.cs OpenAPI contract tests (OpenApiContractTestBase)
STAF.Playwright.Tests/Tests/Excel/ExcelDriverSampleTests.cs ExcelDriver create/read/compare samples
STAF.Playwright.Tests/Pages/GooglePage.cs Sample page object inheriting BasePage
STAF.Playwright.Tests/OpenAPI/placeholder.json OpenAPI spec for contract tests (JSONPlaceholder)
STAF.Playwright.Tests/testsetting.runsettings BaseUrl, ApiBaseUrl, Browser, Headless, Environment, etc.
STAF.Playwright.Tests/testdata.json Optional test data by environment (QA, UAT, …)
AGENTS.md Cross-tool AI entry (Cursor, Copilot, VS agents, golden files, token discipline)
AI/instructions/ Canonical AI playbook: persona, generation rules, debugging, qa-orchestrator-lifecycle, ai-setup, QUICK_START
AI/skills/ Canonical per-topic skills (UI, API, DB, test data, reporting, framework, qa-orchestrator)
QA/ Orchestrator outputs: work-items/{Provider}-{Id}/ phase .md reports (see QA/README.md)
.github/agents/ Visual Studio Copilot custom agents (UI, API, contract, QA orchestrator)
.cursor/skills/ Cursor project skills (stubs → AI/) + MASTER.md index
.cursor/cursor.rules Cursor global consistency rules (aligned with Copilot instructions)
.vscode/README.md VS Code Copilot + MCP setup
.vscode/staf-ai/INDEX.md Table of contents for AI/ files (VS Code attach list)
MCPAgent/ Bundled MCP servers — Playwright C# + Azure DevOps (see Using MCP servers)
MCPAgent/AzureDevOps/ Azure DevOps MCP executable + mcp-config.example.json
.cursor/mcp.json Cursor MCP config (Playwright + Azure DevOps)
.cursor/rules/staf-playwright-framework.mdc Cursor rules for STAF Playwright (base classes, page objects, tool usage)
.github/copilot-instructions.md GitHub Copilot / agent instructions; references AI/ as single source of truth
.vscode/mcp.json VS Code MCP config (Playwright + Azure DevOps)
.mcp.json Visual Studio MCP config (solution root)

Configuration

  • Run settings – Edit STAF.Playwright.Tests/testsetting.runsettings: BaseUrl, ApiBaseUrl, Browser (Chrome, Firefox, Edge, Webkit), Headless, Environment, and other parameters.
  • Environment variables – Override any parameter with the STAF_ prefix (e.g. STAF_BaseUrl, STAF_Headless=true).
  • Test data – Use testdata.json and optional testdata.{Environment}.json; access via ConfigManager.GetTestData(environment, section, key).

For full options and CI/CD usage, see the STAF.Playwright NuGet page and the framework’s configuration documentation.


Using MCP servers

This repository includes MCP servers under MCPAgent/ so AI assistants in Cursor, VS Code, or Visual Studio can help you author STAF.Playwright tests and (optionally) fetch Azure DevOps work items for the QA Orchestrator.

Server Path Purpose Setup
Playwright C# MCPAgent/PlaywrightCSharpMcp.exe Browser inspection and AI-assisted UI/API test authoring Ready after clone — enable in your editor
Azure DevOps MCPAgent/AzureDevOps/AzureDevOps.Mcp.Server.exe Fetch PBIs, user stories, and related work-item data Requires org / project / team + PAT (below)

Config files (edit these for your environment):

Editor File
Cursor .cursor/mcp.json
VS Code .vscode/mcp.json
Visual Studio .mcp.json

Reference template: MCPAgent/AzureDevOps/mcp-config.example.json.

Prerequisites

  • .NET 10 SDK
  • One of: Cursor, VS Code (with GitHub Copilot), or Visual Studio 2022 (17.14+) / Visual Studio 2026 with GitHub Copilot
  • Open the repository root as the workspace (Cursor/VS Code: File → Open Folder; Visual Studio: open STAF.Playwright.Tests.sln)

Playwright C# MCP (test authoring)

No credentials required. After opening the repo:

  1. CursorCursor Settings → Features → MCP (or MCP panel) and ensure playwright-csharp is enabled.
  2. VS Code — Trust the MCP server when prompted. In Copilot Chat (Agent mode), enable playwrightCsharp tools.
  3. Visual Studio — Copilot Chat → Agent mode → enable playwrightCsharp. If the server does not start, set "command" in .mcp.json to the full path of MCPAgent/PlaywrightCSharpMcp.exe. See Use MCP servers in Visual Studio.

Ask the assistant to inspect a page or generate/refine tests and page objects that follow STAF patterns (BaseTest, BasePage, ReportResult).

Azure DevOps MCP (work items / QA orchestrator)

Use this when you want the QA Orchestrator (or chat) to pull live work items from Azure DevOps instead of pasting requirements.

Step 1 — Create a Personal Access Token (PAT)

  1. Sign in to Azure DevOps and open User settings → Personal access tokens → New Token.
  2. Choose the organization you will query (or all accessible organizations if required).
  3. Set an expiration you are comfortable with.
  4. Grant at least these scopes:
    • Work Items — Read (required to fetch PBIs / stories)
    • Project and Team — Read (recommended for project/team context)
    • Code — Read (optional; useful if linked PRs/repos are needed)
  5. Create the token and copy it immediately — Azure DevOps shows it only once.

Official guide: Use personal access tokens.

Security: Never commit a real PAT to git. Prefer editor input prompts (VS Code / Visual Studio) or keep the token only in your local MCP env. Rotate the token if it was ever shared or committed by mistake.

Step 2 — Set organization, project, team, and PAT

The Azure DevOps server is started with:

AzureDevOps.Mcp.Server.exe <organization> -a envvar -d all
Setting Where it goes Example
Organization First CLI argument (from https://dev.azure.com/{organization}) contoso
Auth mode -a envvar Reads the token from ADO_MCP_AUTH_TOKEN
Tool domains -d all Enables the full ADO tool set
Project Env ado_mcp_project Your ADO project name
Team Env ado_mcp_team Often {Project} Team
PAT Env ADO_MCP_AUTH_TOKEN Your Personal Access Token (raw value)

Update the values in your editor’s MCP file:

Visual Studio (.mcp.json) and VS Code (.vscode/mcp.json) — recommended pattern:

  • Organization and PAT are prompted securely via inputs (ado_org, ado_pat).
  • Map them with ${input:ado_org} and ADO_MCP_AUTH_TOKEN: ${input:ado_pat}.
  • Set ado_mcp_project and ado_mcp_team to your project and team names.

Cursor (.cursor/mcp.json):

  1. Set command to MCPAgent/AzureDevOps/AzureDevOps.Mcp.Server.exe (use a full path only if a relative path fails on your machine).
  2. Set args to ["YOUR_ORG", "-a", "envvar", "-d", "all"].
  3. Set env.ADO_MCP_AUTH_TOKEN to your PAT (local machine only).
  4. Set ado_mcp_project and ado_mcp_team.
  5. Reload MCP (Cursor Settings → MCP) and confirm azure-devops is enabled.

Step 3 — Auth alternatives (optional)

Mode Flag When to use
Environment variable (default in this repo) -a envvar PAT stored in ADO_MCP_AUTH_TOKEN
Interactive browser sign-in -a interactive Local setup without storing a PAT
Azure CLI -a azcli Machine already signed in with az login

Step 4 — Verify

  1. Restart the editor or reload MCP servers.
  2. Confirm azure-devops tools appear in the MCP / Copilot tool list.
  3. Try a prompt such as: “Fetch Azure DevOps work item 12345 and summarize the acceptance criteria.”
  4. Or run the STAF QA Orchestrator with an ADO ID or URL (@staf-qa-orchestrator / skill staf-qa-orchestrator).

If ADO MCP is not configured, paste the work item into chat; the orchestrator records that limitation in 01-pbi-fetch.md.

AI instructions and rules

So generated code follows STAF.Playwright patterns (base classes, page objects, reporting), the repo includes:

Copy-paste prompts and @-attach bundles: AI-assisted automation (above).

MCP tools complement the handbook; they do not replace the BaseTest / BasePage / ApiClient patterns in this project.


Sample tests (all STAF.Playwright features)

  • UITest1 inherits BaseTest, uses Page and ConfigManager, and GooglePage (BasePage) for search.
  • APIApiTests inherits TestBaseAPI, uses ApiClient.GetAsync against JSONPlaceholder; set ApiBaseUrl in runsettings.
  • ContractContractTests inherits OpenApiContractTestBase, validates endpoints against OpenAPI/placeholder.json (status code and optional schema).
  • ExcelExcelDriverSampleTests uses ExcelDriver to create workbooks, set/get cells, save, open, and compare files (temp directory, no external files).

You can add more tests and page objects following the same patterns.


License

This project is licensed under the MIT License.

See LICENSE.txt for the full text.


Author and repository

Contributions and issues are welcome.

About

Reusable Playwright framework with MSTest base classes and utilities.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages