-
-
Notifications
You must be signed in to change notification settings - Fork 4.5k
fix(releases): Consistently sort by build number & code in semver #102711
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #102711 +/- ##
===========================================
+ Coverage 80.02% 80.66% +0.64%
===========================================
Files 9239 9241 +2
Lines 393532 393840 +308
Branches 25051 25051
===========================================
+ Hits 314933 317703 +2770
+ Misses 78149 75687 -2462
Partials 450 450 |
| def test_release_list_order_by_semver(self) -> None: | ||
| self.login_as(user=self.user) | ||
| release_1 = self.create_release(version="[email protected]") | ||
| release_2 = self.create_release(version="[email protected]+122") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Switch the order in which release_2 and release_7 are created so the build_number sorting doesn't coincidentally pass due to insertion order.
771af9f to
2ac040b
Compare
| class ReleaseOrderBySemverTestCase(TestCase): | ||
| def test_compare_version_sorts_by_build_number(self): | ||
| """ | ||
| Test that compare_version correctly sorts releases by build number with various formats. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't need to actually merge this but just thought I'd include it for a better picture of how compare_version actually works when detecting regressions. Here we can see that numeric builds are compared numerically and adding any letters makes the comparison lexicographic. We also see that alphanumeric builds are deemed "greater" than numeric builds, and any build is "greater" than no build.
| release_1, # [email protected] | ||
| release_3, # [email protected] | ||
| release_9, # random_junk | ||
| release_8, # test@some_thing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This order now matches what we see in test_compare_version_sorts_by_build_number below
fixes REPLAY-803
When detecting regressions via
compare_versionwe are comparing the build codes. However, we are not ordering by build code or number in any other instances of semver ordering, which leads to inconsistent behavior.For example, when determining the "next" release to resolve an issue by, we get the greatest semver version with an arbitrary build code--say the greatest semver in our project is 2.0.0 (10) and we pick 2.0.0 (8) arbitrarily. Say we later see that issue appearing in 2.0.0 (9). We would mark it as a regression because 2.0.0 (9) > 2.0.0 (8) according to
compare_version, even though the greatest semver in our project was 2.0.0 (10) and it should not be marked as a regression.In this PR we add build number and code to the semver ordering. We mimic the behavior of
compare_versionby ordering on build code and build number (aka numeric build code). This will make our semver ordering consistent when viewing sorted releases lists, resolving issues, detecting regressions, and sending alerts about regressions.Build code ordering in
compare_version, DESC (largest to smallest):For more information on the behavior of
compare_versionwhen detecting regressions, see this comment.Notes:
Before
Sorting by semver sorts the version but not the build number, which stays in the order in which it was created.
Screen.Recording.2025-11-11.at.10.55.50.AM.mov
Resolving in the next release, where the greatest semver release is 2.0.0 (10), instead arbitrarily picks 2.0.0 (8). You can also see that "The current semver release" in the dropdown has arbitrarily picked 2.0.0 (9).
Screen.Recording.2025-11-11.at.11.08.48.AM.mov
Searching for "release is latest" in Issues returns a group associated with 2.0.0 (9), not 2.0.0 (10).
Screen.Recording.2025-11-11.at.11.26.33.AM.mov
After
Sorting by semver now sorts the build numbers as well -- alphanumeric builds on top, then numeric builds, then no build.
Screen.Recording.2025-11-11.at.10.56.57.AM.mov
Resolving in the next release, where the greatest semver release is 2.0.0 (10), now correctly picks 2.0.0 (10). "The current semver release" in the dropdown also correctly references 2.0.0 (10).
Screen.Recording.2025-11-11.at.11.10.54.AM.mov
Searching for "release is latest" in Issues correctly returns a group associated with 2.0.0 (10).
Screen.Recording.2025-11-11.at.11.27.25.AM.mov