Skip to content

Add Allure upload command support#50

Merged
satvik007 merged 7 commits intomainfrom
feat/allure-upload-support
Mar 13, 2026
Merged

Add Allure upload command support#50
satvik007 merged 7 commits intomainfrom
feat/allure-upload-support

Conversation

@satvik007
Copy link
Collaborator

@satvik007 satvik007 commented Feb 10, 2026

Summary

Adds a new allure-upload command that ingests Allure 2 results directories and uploads them to QA Sphere.

Allure parser (allureParser.ts)

  • Reads *-result.json files from a directory; ignores containers, XML, and raw attachments
  • Test case matching via TMS links (type: "tms") or marker in test name (e.g. BD-123)
  • Status mapping: passed → passed, failed → failed, broken → blocked, skipped → skipped, unknown → open
  • Resolves attachments by source filename within the same directory; also collects attachments from nested steps recursively
  • Strips ANSI codes and HTML-escapes messages/stack traces
  • Derives suite/folder from Allure suite/parentSuite/subSuite labels
  • Uses formatMarker() from MarkerParser for consistent marker formatting
  • allowPartialParse support: skips invalid result files instead of aborting when --force is set

Command wiring

  • Registers allure-upload as a third command in main.ts via the same ResultUploadCommandModule
  • Added allure-upload to directoryInputTypes in ResultUploadCommandHandler — the parser receives the directory path rather than file contents
  • Unmatched-test error guidance now includes allure-specific hints
  • getMarkerFromText added to MarkerParser and exposed in tests

Tests & fixtures

  • allure-parsing.spec.ts — 545-line unit test suite covering TMS links, markers, status mapping, suite labels, attachment resolution, nested step attachments, partial parse, invalid files, and edge cases
  • result-upload.spec.ts — integration tests for the full allure upload flow via MSW
  • Fixture directories: matching-tcases, missing-tcases, empty-tsuite, invalid-results, missing-attachments, without-markers

Documentation

  • README updated with allure-upload usage, options, and examples

Testing

  • npm test — all unit and integration tests pass
  • npm run check — typecheck + lint + format
  • Manual smoke test against bistro-e2e Allure reports: uploaded 6 test results with 6 PNG screenshot attachments to a live QA Sphere instance — clean run, no errors

Resolves: https://github.com/Hypersequent/tms-issues/issues/2329

@gemini-code-assist
Copy link

Summary of Changes

Hello @satvik007, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly expands the QAS CLI's capabilities by integrating native support for Allure test reports. Users can now upload Allure results directories, leveraging a new parser that handles status mapping, folder organization, and attachment resolution. The update also includes enhanced error handling for invalid Allure files and comprehensive documentation, making the CLI more versatile for diverse test automation workflows.

Highlights

  • New Allure Upload Command: Introduced a new allure-upload command to the QAS CLI, enabling direct upload of Allure test results to QA Sphere.
  • Directory-Based Allure Parser: Implemented a dedicated parser for Allure 2 JSON results (*-result.json files), supporting status mapping, folder extraction, message processing, and attachment resolution.
  • Flexible Test Case Linking: Allure results can now link to QA Sphere test cases via TMS links (recommended), TMS link names, or markers embedded directly in test names.
  • Robust Error Handling for Allure Results: Added support for the --force option with allure-upload, allowing the CLI to skip malformed or schema-invalid Allure result files and continue processing valid ones.
  • Comprehensive Documentation and CLI Guidance: Updated README.md and CLAUDE.md to reflect Allure support, including new command examples and guidance for resolving unmatched Allure test cases.
  • Expanded Test Coverage: Added new unit tests for the Allure parser and integrated Allure into existing end-to-end upload integration tests.
Changelog
  • AGENTS.md
    • Added a symlink to CLAUDE.md.
  • CLAUDE.md
    • Updated project overview to include Allure results directories as a supported input type.
    • Modified CLI framework description to list three commands, including allure-upload.
    • Added Allure parser to the core upload pipeline documentation.
    • Included allure-parsing.spec.ts in the list of test files.
  • README.md
    • Updated the tool's description to mention Allure result directories.
    • Corrected the example format for QAS_URL.
    • Updated the 'Commands' section to include allure-upload.
    • Added a description for input paths, differentiating between files and directories for various commands.
    • Added new examples for allure-upload, including uploading results from a directory and using the --force option.
    • Updated 'Test Report Requirements' to include Allure as a supported report type and detailed Allure-specific test case linking methods.
  • package-lock.json
    • Updated the package version from 0.4.4 to 0.4.5.
  • src/commands/main.ts
    • Registered the new allure-upload command with the Yargs CLI setup.
  • src/commands/resultUpload.ts
    • Extended commandTypeDisplayStrings to include 'Allure'.
    • Added commandTypeInputKinds to specify whether a command expects 'files' or 'directories'.
    • Added commandTypeExampleInputs to provide example input paths for each command type.
    • Modified the describe getter to dynamically use commandTypeInputKinds for better command descriptions.
    • Added a positional argument for 'files' that dynamically describes input as files or directories based on the command type.
    • Updated argv.example calls to use commandTypeExampleInputs for more accurate command examples.
  • src/tests/allure-parsing.spec.ts
    • Added a new test file containing unit tests for the Allure parser, covering marker extraction, status mapping, folder priority, attachment handling, and malformed file handling.
  • src/tests/fixtures/allure/empty-tsuite/001-result.json
    • Added a new Allure result fixture for testing empty test suites.
  • src/tests/fixtures/allure/empty-tsuite/002-container.json
    • Added a new Allure container fixture for testing empty test suites.
  • src/tests/fixtures/allure/invalid-results/001-result.json
    • Added a new Allure result fixture for testing partial directories.
  • src/tests/fixtures/allure/invalid-results/003-result.json
    • Added a new Allure result fixture for testing schema-invalid files.
  • src/tests/fixtures/allure/matching-tcases/001-result.json
    • Added a new Allure result fixture with TMS link and suite label.
  • src/tests/fixtures/allure/matching-tcases/002-result.json
    • Added a new Allure result fixture with TMS link name fallback and parentSuite label.
  • src/tests/fixtures/allure/matching-tcases/003-result.json
    • Added a new Allure result fixture with marker in name and feature label.
  • src/tests/fixtures/allure/matching-tcases/004-result.json
    • Added a new Allure result fixture with failed status, status details, and package label.
  • src/tests/fixtures/allure/matching-tcases/005-result.json
    • Added a new Allure result fixture with unknown status and no labels.
  • src/tests/fixtures/allure/matching-tcases/006-container.json
    • Added a new Allure container fixture for matching test cases.
  • src/tests/fixtures/allure/matching-tcases/legacy-testsuite.xml
    • Added a new legacy Allure 1 XML fixture to ensure it's ignored by the new parser.
  • src/tests/fixtures/allure/matching-tcases/shared-attachment.txt
    • Added a new shared attachment fixture for Allure tests.
  • src/tests/fixtures/allure/missing-attachments/001-result.json
    • Added a new Allure result fixture with an existing attachment.
  • src/tests/fixtures/allure/missing-attachments/002-result.json
    • Added a new Allure result fixture with an existing attachment.
  • src/tests/fixtures/allure/missing-attachments/003-result.json
    • Added a new Allure result fixture with an existing attachment.
  • src/tests/fixtures/allure/missing-attachments/004-result.json
    • Added a new Allure result fixture with an existing attachment.
  • src/tests/fixtures/allure/missing-attachments/005-result.json
    • Added a new Allure result fixture with a missing attachment.
  • src/tests/fixtures/allure/missing-attachments/existing-attachment.txt
    • Added a new existing attachment fixture for Allure tests.
  • src/tests/fixtures/allure/missing-tcases/001-result.json
    • Added a new Allure result fixture for testing missing test cases.
  • src/tests/fixtures/allure/missing-tcases/002-result.json
    • Added a new Allure result fixture for testing missing test cases.
  • src/tests/fixtures/allure/missing-tcases/003-result.json
    • Added a new Allure result fixture for testing missing test cases.
  • src/tests/fixtures/allure/missing-tcases/004-result.json
    • Added a new Allure result fixture for testing missing test cases.
  • src/tests/fixtures/allure/missing-tcases/005-result.json
    • Added a new Allure result fixture for testing missing test cases.
  • src/tests/fixtures/allure/without-markers/001-result.json
    • Added a new Allure result fixture without markers.
  • src/tests/fixtures/allure/without-markers/002-result.json
    • Added a new Allure result fixture without markers.
  • src/tests/fixtures/allure/without-markers/003-result.json
    • Added a new Allure result fixture without markers.
  • src/tests/result-upload.spec.ts
    • Introduced TestFileType interface and fixtureInputPath helper for consistent test fixture handling.
    • Extended the fileTypes array to fileTypesWithAllure to include Allure for integration tests.
    • Updated existing integration tests to dynamically use fixtureInputPath for test report paths, supporting both files and directories.
    • Added a new test suite specifically for Allure invalid result file handling, verifying behavior with and without the --force option.
  • src/utils/result-upload/ResultUploadCommandHandler.ts
    • Extended UploadCommandType to include allure-upload.
    • Imported parseAllureResults and added it to the commandTypeParsers map.
    • Added allowPartialParse to ParserOptions to support skipping invalid files.
    • Modified the Parser type definition to clarify that data can be a directory path for directory-based parsers.
    • Updated the parseFiles method to conditionally read file content or use the file path directly based on the command type (e.g., Allure uses directory path).
    • Passed allowPartialParse (derived from the --force argument) to the parser options.
    • Refactored execRegexWithPriority to use String.prototype.match for regex execution.
  • src/utils/result-upload/ResultUploader.ts
    • Added a new private method printAllureGuidance to provide specific instructions for resolving unmatched Allure test cases.
    • Integrated printAllureGuidance into printUnmatchedGuidance to display relevant help messages for allure-upload.
  • src/utils/result-upload/allureParser.ts
    • Added a new file containing the parseAllureResults function.
    • Defined Zod schemas for Allure result structures (status, status details, attachments, labels, links, parameters).
    • Implemented logic for mapping Allure statuses to QA Sphere result statuses.
    • Included functions for extracting folder information based on Allure labels (suite, parentSuite, feature, package).
    • Developed buildMessage to format status details (stdout/stderr) based on skip options.
    • Created extractMarker to prioritize marker extraction from TMS links (URL, then name) and then from the test name itself.
    • Added error handling for malformed or schema-invalid Allure JSON files, with an option to allow partial parsing.
Activity
  • The author has performed unit tests (npm test) and code style checks (npm run check).
  • Optional manual smoke tests for the allure-upload command were suggested by the author.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

claude[bot]

This comment was marked as outdated.

gemini-code-assist[bot]

This comment was marked as outdated.

satvik007 and others added 2 commits March 6, 2026 21:28
Introduce a directory-based Allure parser and wire it into the upload pipeline so users can upload Allure 2 results with TMS-link or marker-based matching. Extend CLI, tests, and docs for allure-upload, support force-aware skipping of invalid Allure files, and add AGENTS.md as a symlink to CLAUDE.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
… and expanded tests

- Switch allureParser to use formatMarker() from MarkerParser instead of
  the removed getTCaseMarker() from misc
- Replace inline this.type === 'allure-upload' checks in parseFiles() with
  a module-level directoryInputTypes Set for cleaner extensibility
- Add 7 new allure-parsing unit tests: ANSI stripping, null arrays, HTML
  escaping, container/non-result file filtering, empty directory, failed
  test skip behavior, and negative duration handling
- Update CLAUDE.md to document directoryInputTypes, allowPartialParse, and
  allureParser behavior details

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@satvik007 satvik007 force-pushed the feat/allure-upload-support branch from 232171f to 71a1ce1 Compare March 6, 2026 16:15
claude[bot]

This comment was marked as outdated.

claude[bot]

This comment was marked as outdated.

…erFromText tests

- Make links[].url optional in schema so @TMslinK without allure.link.tms.pattern works
- Guard getMarkerFromTmsLinks against undefined URL
- Add recursive AllureStep schema and collectStepAttachmentPaths for nested attachments
- Add getMarkerFromText unit tests for edge cases
- Add blank line before getErrorMessage
claude[bot]

This comment was marked as outdated.

@satvik007 satvik007 marked this pull request as ready for review March 9, 2026 17:20
@claude
Copy link

claude bot commented Mar 9, 2026

Claude finished @satvik007's task in 4m 25s —— View job


Review Complete

  • Read review instructions
  • Gather PR context and diff
  • Launch specialized review subagents (code quality, security, performance, test coverage)
  • Fetch existing review comments for dedup
  • Aggregate findings and post review
  • Clean up outdated bot threads and reviews

The latest commit (fd821bf) bumps the version to 0.5.0. All code issues from prior reviews have been addressed — marker extraction delegates to shared getMarkerFromText, start/stop optional in Zod schema, directory validation in place, and test coverage is comprehensive.

No new issues found. Submitted a review with one carried-forward inline comment: the path traversal via attachment.source in utils.ts remains the only open item before merge.

Minimized 4 older bot review summaries, deleted 2 stale progress comments.

@satvik007 satvik007 requested review from AndrianBdn and hi-rai March 9, 2026 17:22
@satvik007
Copy link
Collaborator Author

@codex review

claude[bot]

This comment was marked as outdated.

@chatgpt-codex-connector
Copy link

Codex Review: Didn't find any major issues. Hooray!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@satvik007
Copy link
Collaborator Author

@Hypersequent/core Please review this PR

}

export type Parser = (
// Primary parser input. File-based parsers receive file contents while
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: We can change it to path for file uploads as well. I had done it in xcresults PR - #40 - but that is not merged yet

Not necessary, can be skipped now, and done in future as per requirement

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where did you get the schema definition/details from ? Can you add links in this file, which might be helpful in future ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added links. I got it from docs and also generated from examples repo by using docker containers for different languages/frameworks

@claude
Copy link

claude bot commented Mar 13, 2026

Claude finished @satvik007's task in 9s —— View job


  • Read review instructions
  • Gather PR context and diff
  • Launch specialized review subagents (code quality, security, performance, test coverage)
  • Fetch existing review comments for dedup
  • Aggregate findings and post review
  • Clean up outdated bot threads and reviews

Review Complete

The latest commits (3944626, 09b1f1d) address all remaining reviewer feedback — commandDetails consolidated into a single record, schema reference links added. All prior code issues have been resolved. Submitted a review with one carried-forward inline comment: the path traversal via attachment.source in utils.ts remains the only open item before merge.

Minimized 6 older bot review summaries, resolved 1 duplicate bot thread.

@satvik007 satvik007 merged commit 4b053bd into main Mar 13, 2026
4 checks passed
@satvik007 satvik007 deleted the feat/allure-upload-support branch March 13, 2026 09:21
Copy link

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

The latest commits (3944626, 09b1f1d) address all remaining reviewer feedback — commandDetails consolidated into a single record per hi-rai's suggestion, and schema reference links added to the test file. The implementation is clean, well-tested, and follows existing codebase patterns.

Issues resolved across PR iterations:

  • Marker extraction delegates to shared getMarkerFromText from MarkerParser.ts
  • start/stop fields made optional in Zod schema
  • Directory validation with user-friendly error message
  • CLAUDE.md numbered list formatting restored
  • Comprehensive test coverage (empty dir, stop < start, ANSI stripping, HTML escaping, nested step attachments, TMS link edge cases, short markers)
  • commandDetails consolidated into single Record<UploadCommandType, CommandDetails> (latest commit)
  • Schema reference comments added to test file (latest commit)

Still open (1 inline comment, carried forward):

  • Path traversal via attachment.source — The getFile function in utils.ts uses path.join(basePath, filePath) without validating the resolved path stays within the base directory. This remains the only substantive item to address before merge.

No new issues found in the latest commits.

const attachmentPaths = [...topLevelPaths, ...stepPaths]
attachmentsPromises.push({
index,
promise: getAttachments(attachmentPaths, attachmentBaseDirectory || resultsDirectory),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Path traversal via attachment.source — still unaddressed (carried forward from prior reviews)

The attachment.source field from Allure JSON is passed directly to getAttachments()getFile() in utils.ts, which uses path.join(basePath, filePath) without validating the resolved path stays within the base directory. A malicious result file could set source to ../../../../etc/passwd, and path.join would resolve the traversal, allowing arbitrary file reads that get uploaded to QA Sphere.

This is the most relevant for Allure since attachment paths come from user-provided JSON (unlike JUnit/Playwright where paths are typically extracted from test output).

The fix belongs in getFile() in utils.ts (shared by all parsers):

const resolvedPath = path.resolve(basePath, filePath)
if (!resolvedPath.startsWith(path.resolve(basePath) + path.sep) && resolvedPath !== path.resolve(basePath)) {
  throw new Error(`Attachment path "${filePath}" escapes the base directory`)
}

This was flagged independently by both Claude and Gemini in the first review cycle and remains the only open item.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants