test(install): run the Windows upgrade nobody has ever run - #470
Merged
Conversation
`install.ps1` lays the tree out flat. `ix upgrade` re-lays it, and once re-nested it — it resolved `stagedRoot` to find the CLI entry point and then swapped in the outer `stagingDir`, so the first upgrade of a flat install produced `cli\ix-<version>-windows-amd64\`. `findCompassDist` and `COMPASS_DIR` read `$IX_HOME\cli\compass` and nothing else, so the symptom is `ix view` exiting "Compass UI not found". Fixed in #346, verified once by hand, and never since. It was recorded as untestable until a GA existed, because `ix upgrade` resolves /releases/latest and that excludes prereleases — so from an RC it correctly declines and never touches the layout. True of upgrading FROM a prerelease. Not true of upgrading from an older STABLE, which is also what every real user does. So: install the previous stable, run `ix upgrade`, and assert the tree, the launcher, the compass bundle and the scratch afterwards. Stable releases only, and that is the design rather than a limitation. `ix upgrade` has no version flag, so the only run in which it can land on the release under test is the one that just made that release latest. On a prerelease it would upgrade an old stable to the CURRENT stable, exercising neither the new artifact nor any code still changeable. It is skipped there — a skipped job says so, a green one would not. It follows this cannot gate a PR either: the code performing the swap is the INSTALLED release's upgrade.ts, never the branch's. That is a property of self-replacement, not a gap. The honest question is whether the upgrade users are about to run survives, and it is asked on the run that publishes the release they will run it against. Two defects found while writing it, both in the assertions: **A version check that could not fail.** `& ix --version 2>&1` merges stderr, and `printUpdateNotice` writes "Update available: <current> → <latest>" there on ordinary commands. A CLI still on the baseline emits the version under test inside that line, so the substring assertion meant to prove the swap happened would pass against an untouched install. Both version checks read stdout only and compare for equality now — confirmed against a real CLI that `--version` puts the version and nothing else on stdout, and the notice entirely on stderr. **The same merge in the `install` job**, carried over rather than left next to its fixed twin. It has been passing honestly only because a first install of the newest release cannot produce that line — a property of the fixture, not of the check. Also, pwsh runs with $ErrorActionPreference='Stop' in Actions, where a native command's stderr merged by 2>&1 can surface as a terminating NativeCommandError. The update notice guarantees stderr after the upgrade step, and the backend image pull always fails on a runner with no Linux Docker. Set to Continue where the transcript is wanted; the exit code stays the verdict.
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
install.ps1lays the tree out flat:$IX_HOME\cli\{cli,compass,core-ingestion,ix}.ix upgradere-lays that tree, and it once re-nested it — resolvingstagedRootto find the CLI entry point and then swapping in the outerstagingDir, so the first upgrade of a flat install producedcli\ix-<version>-windows-amd64\.findCompassDistandCOMPASS_DIRread$IX_HOME\cli\compassand nothing else, so the symptom isix viewexiting withCompass UI not found. Fixed in #346, verified once, by hand, and never since.#459 covered
install.ps1. This covers the other half.Why it was thought untestable, and why that was too strong
The reasoning on record:
ix upgraderesolves/releases/latest, which excludesprereleases, so from an RC it correctly declines and never touches the layout.
That holds for upgrading from a prerelease. It does not hold for upgrading from
an older stable — which is also what every real user does. So the job installs
the previous stable, runs
ix upgrade, and asserts the tree, the launcher, thecompass bundle, and the scratch left behind.
Stable releases only — the design, not a limitation
ix upgradehas no version flag; it goes wherever/releases/latestpoints. Theonly run in which it can land on the release under test is the run that just made
that release latest. On a prerelease it would install an old stable and upgrade it
to the current stable, exercising neither the new artifact nor any code that can
still be changed. It is skipped there. A skipped job says that; a green one would not.
It follows that this cannot gate a PR either. The code performing the swap is the
installed release's
upgrade.ts, never the branch's — a PR editingupgrade.tsis testing its change one release from now whatever this job does. That is a property
of self-replacement, not a gap to paper over. The honest question is "does the
upgrade users are about to run survive", and it is asked on the run that publishes
the release they will run it against.
Net effect: on the v0.10.0 GA run, this installs v0.9.3, upgrades it to v0.10.0, and
fails the run if the layout does not survive — automatically, in the window between
tagging and announcing where the check was going to be done by hand anyway.
Two defects found while writing it, both in assertions
A version check that could not fail.
& $shim --version 2>&1merges stderr, andprintUpdateNoticewritesUpdate available: <current> → <latest>there on ordinarycommands. A CLI still sitting on the baseline emits the version under test inside
that line, so the substring assertion meant to prove the swap happened would pass
against an untouched install. Both version checks now read stdout only and compare
for equality.
Confirmed against a real CLI rather than assumed:
The same merge in the existing
installjob. Fixed here rather than left standingnext to its fixed twin. It has been passing honestly only because a first install of
the newest release cannot produce that line — a property of the fixture, not of the check.
One environment note
Actions runs pwsh with
$ErrorActionPreference='Stop', under which a native command'sstderr merged by
2>&1can surface as a terminatingNativeCommandError. This jobguarantees stderr: the update notice fires on every command once
.version-check.jsonknows a newer component exists, and the backend image pull always fails on a runner
with no Linux Docker. Set to
Continuein the steps that want the transcript; the exitcode stays the verdict.
Known, and called out rather than hidden
ix upgraderesolves three releases through the unauthenticatedapi.github.com(60/hr per IP, shared across the runner pool). A 403 there returns null and the CLI
exits 1 saying it could not reach GitHub — true, but it reads as an outage. The step
detects that string and fails with the rate limit named, so the next person does not go
looking for one. There is no token plumbing in
fetchLatestReleaseto avoid it.This PR touches the workflow, so its own
installjob runs here and exercises theassertion changes to that half.