MAINT: Publish the backend log from the shared test jobs - #2520
Open
varunj-msft wants to merge 1 commit into
Open
MAINT: Publish the backend log from the shared test jobs#2520varunj-msft wants to merge 1 commit into
varunj-msft wants to merge 1 commit into
Conversation
varunj-msft
force-pushed
the
varunj-msft/v1.1.0-Release-Backend-Log-Artifact
branch
from
September 1, 2026 18:04
6806919 to
4ebf73f
Compare
hannahwestra25
approved these changes
Sep 1, 2026
The end-to-end tests launch pyrit_backend through ServerLauncher, which writes the server log to a file in the system temp directory and then discards it when the agent is torn down. A failing job therefore reports only the client side of the failure, so a backend that is alive but unresponsive is indistinguishable from one that never started. Collect that log and publish it as a pipeline artifact, following the pattern already used by adversarial-benchmark.yml. The collection step tolerates a missing file so the jobs that do not launch a backend are unaffected, and the publish step is skipped entirely when no log was found. Search every directory tempfile.gettempdir() honours. It reads TMPDIR, TEMP and TMP before falling back to /tmp, so checking only TMPDIR silently dropped the log whenever the agent set one of the other two, which is exactly the diagnostic this change exists to capture. Name the artifact per job attempt. Build.BuildId does not change when a failed job is retried and pipeline artifacts cannot be overwritten, so the retry of any failed job would fail at the publish step. System.JobAttempt keeps each attempt distinct, and continueOnError keeps a failed upload of a diagnostic log from changing the outcome of the test job itself. Report a failed collection separately from a missing log. The step decided whether a log existed by testing the destination, so a copy that failed printed the same line as a job that never launches a backend, which on the end-to-end pipeline is the opposite of what happened. Track whether a source was found, warn with the reason the copy failed, and report the size of what was collected so a zero byte log is not mistaken for a step that did nothing.
varunj-msft
force-pushed
the
varunj-msft/v1.1.0-Release-Backend-Log-Artifact
branch
from
September 1, 2026 20:37
4ebf73f to
7c94d42
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The end-to-end and integration jobs launch
pyrit_backendthroughServerLauncher, which writes the server log to a file in the system temp directory and then throws it away when the agent is torn down. A failing job therefore reports only the client side of the failure, which means a backend that is alive but unresponsive is indistinguishable from one that never started at all.That distinction is exactly what we could not make while debugging the current End to End Tests failures: every one of the failures is a client-side read timeout, and the server's own account of what it was doing is discarded along with the agent.
This collects the log and publishes it as a pipeline artifact, following the pattern
adversarial-benchmark.ymlalready uses:condition: always(), so a job that fails, or times out at the 360-minute cap, still yields the log.ServerLauncherwrites totempfile.gettempdir(), which honoursTMPDIR/TEMP/TMPbefore falling back to/tmp, so the step checks the agent temp directory as well as/tmp.Only
.azuredevops/test-job-template.ymlchanges. No product code, no test code, and no behaviour change for any job — this adds diagnostics only.Part of the v1.1.0 release wave with #2510, #2511 and #2512.
Tests and Documentation
This is pipeline YAML, so no unit test can cover it. I validated it by extracting the script body and executing it against all three paths:
backendLogFound=true, artifact published. Step exits 0.The third case is worth calling out, because the first version of this step got it wrong.
set -uo pipefaildoes not include-e, so a failingcpis silent: the step would have claimed success, setbackendLogFound=true, and published an empty directory while simultaneously printing "No pyrit_backend.log found". The step now gates both the success message and thesetvariableon the copy actually succeeding, so the flag is set only when the file has really landed.No documentation changes, since this touches only pipeline YAML. JupyText was not run and is not applicable here: no notebooks, docs or public APIs are affected by this change.