Skip to content

Conversation

@albinsun
Copy link
Contributor

@albinsun albinsun commented Jan 2, 2026

Which issue(s) this PR fixes:

Issue #2373

What this PR does / why we need it:

  1. Easier check test report to know corresponding issue/reason of skipped cases, e.g.
    • Known issues (with URL)
    • Skipped byVersion
    • Skipped byPriorCase (dependency)
  2. Refactor to unify pytest.mark.skip_version_* marks to the more explicit pytest.mark.skip_version_if.
  3. For known issues, first use skip_version_if() if the milestone is decided. For those still in Planning, may use xfail (expected fail) temporarily.

Special notes for your reviewer:

For known issue to leverage this enhancement, skip reason should contain URL.

Additional documentation or context

Verification (harvester-install-and-test#9)
image

@albinsun albinsun force-pushed the mark_e2e_known_issues branch from 0306396 to 1ea2e30 Compare January 2, 2026 09:22
@albinsun albinsun changed the title Add report content if links in Skipped and XFailed reason Show Link if there is URL in Skipped and XFailed reason Jan 2, 2026
@albinsun albinsun force-pushed the mark_e2e_known_issues branch 7 times, most recently from 85644f0 to 6895359 Compare January 6, 2026 00:04
@albinsun albinsun changed the title Show Link if there is URL in Skipped and XFailed reason Show info in the Link column if there is URL in reason or skipped by dependency Jan 6, 2026
@albinsun albinsun force-pushed the mark_e2e_known_issues branch 2 times, most recently from 11ea0c5 to dca47fb Compare January 6, 2026 14:08
@albinsun albinsun force-pushed the mark_e2e_known_issues branch from dca47fb to a972aa9 Compare January 7, 2026 00:54
@albinsun albinsun marked this pull request as ready for review January 7, 2026 00:55
@albinsun albinsun requested review from a team, asc5543, khushboo-rancher and lanfon72 January 7, 2026 00:55
@albinsun albinsun changed the title Show info in the Link column if there is URL in reason or skipped by dependency Show extra info for Skipped (include XFailed) cases Jan 7, 2026
Copy link
Member

@lanfon72 lanfon72 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general LGTM

reason.format(cluster_version=cluster_ver, versions=versions)
reason.format(
cluster_version=cluster_ver,
condition=cond_str.get(cond, str(cond)),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We didn’t restrict the condition parameter to only any or all; instead, we allowed any callable function. As a result, it may fail if a user provides other callable functions.

To prevent this, we can simply use callable.__name__ to retrieve the condition name, which also eliminates the need to maintain a mapping table.

delete_image(api_client, exported_image_id, wait_timeout)

@pytest.mark.skip_version_if(
">= v1.7.0", "< v1.9.0", reason="https://github.com/harvester/harvester/issues/9515")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#9515 has been rescheduled to v1.9.0.
I would suggest to add regression in reason and not limit the before version, then when then issue be fixed, we can use the keyword to search and update the skip restriction.

And it would be helpful that add the test case path when filing the regression issue. (cc @asc5543)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants