fix: flaky navigation timeouts caused by a lost wakeup in WaitForLoadState and WaitForURL - #641
Open
myrjola wants to merge 1 commit into
Open
Conversation
…so a state recorded in between is not lost Frame.WaitForLoadState read the frame's load states, then built its waiter (a timer goroutine and four listener registrations), and only then subscribed to "loadstate". Events arrive on the connection's single dispatch goroutine, which can run in that gap; a "loadstate" it delivered there was emitted to no listener and is never replayed, so the wait slept out its whole timeout with the state it wanted already recorded. WaitForURL had the same shape around "navigated": it checked the URL, then called ExpectNavigation, which subscribes several registrations later. Upstream's client/frame.ts waitForLoadState builds the waiter first, checks _loadStates, and calls waiter.dispose() when the state is already there; JavaScript's run-to-completion makes that check atomic with the subscription. This brings the Go port to the same shape: attach the listener, re-check the recorded state, and dispose of the waiter (new waiter.dispose(), after upstream's) rather than wait for an event that has already been delivered. onLoadState and onFrameNavigated both update their state before emitting, so a re-check after subscribing cannot miss an event that fired before it. Measured with a browser (six pages clicking between two pages whose stylesheet and script make `load` trail the commit by a few ms): page.WaitForURL after a click timed out 6 times in 1,800 with the page already at the wanted URL and `load` delivered 0.2-0.4 ms after the call; with this change 0 in 1,800 under heavy load. The two new unit tests aim the event at the gap with no browser and fail within a few rounds on the previous code at any GOMAXPROCS (they raise it to 2 if needed). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Thank you for this awesome project! Here's a fix to a flaky navigation timeout caused by a subtle concurrency bug in the client which causes
WaitForURL,ExpectNavigationandGototo occasionally fail even when the page is already at the wanted URL and fully loaded. This failure became quite common in the CI pipelines of my projects but they have been running fine after taking the fix into use. I published a minimal reproduction here https://github.com/myrjola/playwright-go-lost-wakeup. I've verified the reproduction and fix on my Linux server and Macbook.I did not find a prior issue for this; happy to open one if you want it tracked separately. I'm also happy to put in the work to to improve this PR.
Click to open sequence diagrams for understanding the bug and fix.
sequenceDiagram participant W as Wait goroutine<br/>(waitForLoadStateImpl) participant F as frameImpl<br/>(loadStates, listeners) participant D as Dispatch goroutine<br/>(connection) W->>F: read loadStates F-->>W: "load" absent W->>F: setNavigationWaiter(): start timeout timer,<br/>register close / crash / framedetached listeners rect rgba(220, 38, 38, 0.15) Note over F,D: gap: nobody is subscribed to "loadstate" yet D->>F: onLoadState("load") Note over F: loadStates += "load" Note over F: emit "loadstate" to zero listeners,<br/>never replayed end W->>F: On("loadstate", handler) Note over W: nothing re-reads loadStates W->>W: block on errChan W-->>W: ErrTimeout after the full timeout,<br/>page is at the URL and fully loadedAnd the same interleaving after the fix:
sequenceDiagram participant W as Wait goroutine<br/>(waitForLoadStateImpl) participant F as frameImpl<br/>(loadStates, listeners) participant D as Dispatch goroutine<br/>(connection) W->>F: read loadStates F-->>W: "load" absent W->>W: setNavigationWaiter() D->>F: onLoadState("load"): loadStates += "load",<br/>emit to zero listeners W->>F: On("loadstate", handler) W->>F: re-check loadStates F-->>W: "load" present W->>W: waiter.dispose(): stop timer, remove listeners W-->>W: return nilClick to open AI generated report below with more details.
It shows up as a rare, load-correlated timeout whose failure screenshot shows the right page. One instance from the reproduction below, with `framenavigated` and `load` listeners attached before the click:Cause.
waitForLoadStateImplreadsf.loadStates, then builds its waiter (setNavigationWaiter: a timer goroutine and four listener registrations), and only then subscribes to"loadstate". Events arrive on the connection's single dispatch goroutine, which can run between that read and the subscription. Aloadstatedelivered in the gap is emitted to no listener and is never replayed;loadStatesis updated but nothing re-reads it, so the wait sleeps out its whole timeout.WaitForURLhas the same shape around"navigated": it checksf.URL(), then callsExpectNavigation, which subscribes several registrations later.Upstream.
client/frame.tswaitForLoadStatebuilds the waiter first, checks_loadStates, and callswaiter.dispose()when the state is already there; run-to-completion makes that check atomic with the subscription. The Go port checks first and builds the waiter afterwards, and the dispatch goroutine can interleave.Fix. Bring the port to upstream's shape:
waiter.dispose()(new, after upstream'sWaiter.dispose()): marks the waiter fulfilled so a racing handler drops its event, cancels the timeout goroutine, removes every listener.waitForLoadStateImpl: keep the cheap first check (the common already-loaded case builds no waiter), subscribe, re-checkloadStates, dispose on a hit.onLoadStateadds to the set before emitting, so a re-check after subscribing cannot miss an event that fired before it.WaitForURL: the not-yet-matching path goes through an internalexpectNavigation(cb, alreadyNavigated, ...)that re-checks the URL once the"navigated"listener is attached.ExpectNavigationpassesniland is unchanged. On a hit only the load state is awaited and noResponseis returned, whichWaitForURLdiscards anyway.Evidence. Reproduction with both runs' full output: https://github.com/myrjola/playwright-go-lost-wakeup. Six Chromium pages click between two pages whose stylesheet and script make
loadtrail the commit by a few ms; a timeout is LOST if the page is at the wanted URL andloadwas delivered well inside the budget, otherwise slow. Same box, same knobs,-race:Every loss has the same shape: the navigation committed a few ms before the call,
loadarrived 0.2 to 0.4 ms after it, and the wait ran its full 3 s.Tests.
frame_wait_race_test.go(new, no browser): aframeImplfrom the existingnewTimeoutSemanticsFixture; a goroutine firesonLoadState/onFrameNavigatedthe moment the wait's"framedetached"rejection appears on the page, which is the registration just before the wait subscribes to its own event. Both tests fail within a few rounds on v0.6201.1 and pass here; they raise GOMAXPROCS to 2 when they find 1, since at 1 the waiting goroutine never yields before it subscribes, and they check no listener is left on the frame or page.TestWaiterDisposeRemovesEveryListener(new).-race, and the fulltests/suite on Chromium under-race: pass.golangci-lintwith the repo config: 0 issues. gofumpt clean. Driver version untouched.Written with Claude (Fable 5.1) in Claude Code; the measurements are from real runs on my machine.