You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The ios-tests job in Functional Tests fails intermittently on main, with no corresponding code change to explain the failure — i.e. it's CI/infra flakiness, not a product regression.
Data
Sampled the last 8 ios-tests job runs triggered by a push to main (2026-08-28 → 2026-09-11):
4 of 8 (50%) failed, alternating pass/fail across consecutive pushes with no relevant diff in between — a clear flake signature rather than a regression.
Failure signatures observed
All failures happen in test setup (OneTimeSetUp/SetUp, i.e. session creation), never in an actual assertion — confirming this is a session-start/simulator-boot race, not application logic:
WDA launch timeout (run #391) — Session.Logs.LogTests (CanCaptureCrashlogTest, CanCaptureServerTest, CanCaptureSyslogTest, CanRetrieveLogTypesTest — all 4 in the fixture) fail with:
OpenQA.Selenium.UnknownErrorException : Unable to start WebDriverAgent session. Original error: A new session could not be created. Details: Error Domain=XCTDaemonErrorDomain Code=5 "Timed out attempting to launch app."
WDA app not registered with SpringBoard (same run) — LockDeviceTest.LockTestWithSeconds fails with:
OpenQA.Selenium.UnknownErrorException : Unable to launch WebDriverAgent. ... (domain=FBSOpenApplicationServiceErrorDomain, code=1): ... "Application "com.facebook.WebDriverAgentRunner.xctrunner" is unknown to FrontBoard"
.github/workflows/functional-test.yml's ios-tests job has no retry or readiness gate around the two flakiest steps:
Setup iOS Simulator (futureware-tech/simulator-action@v5, wait_for_boot: true) waits for the simulator to report booted, but doesn't confirm the prebuilt WDA app downloaded in the previous step is actually installed/registered with SpringBoard before tests start.
Run iOS functional tests invokes dotnet test exactly once — no [Retry(n)] on session-dependent fixtures, and no step-level retry/back-off if the whole run fails.
Given prebuilt WDA is downloaded and referenced via LOCAL_PREBUILT_WDA rather than built by Xcode locally, there's a plausible race between the simulator reporting "booted" and it having finished registering the just-installed WDA app with FrontBoard/SpringBoard — matching signatures 1 and 2 above.
Suggested next steps
Add a WDA-readiness check (e.g. poll xcrun simctl / attempt a throwaway session) after Setup iOS Simulator and before Run iOS functional tests, instead of relying solely on wait_for_boot.
Increase Appium's wdaLaunchTimeout/wdaConnectionTimeout capabilities used by the iOS test setup.
Add [Retry(2)] (NUnit) to the most session-fragile fixtures, or a CI-level retry of the whole dotnet test invocation on failure.
Consider explicitly polling xcrun simctl bootstatus <udid> -b rather than trusting wait_for_boot: true alone.
Summary
The
ios-testsjob in Functional Tests fails intermittently onmain, with no corresponding code change to explain the failure — i.e. it's CI/infra flakiness, not a product regression.Data
Sampled the last 8
ios-testsjob runs triggered by a push tomain(2026-08-28 → 2026-09-11):4 of 8 (50%) failed, alternating pass/fail across consecutive pushes with no relevant diff in between — a clear flake signature rather than a regression.
Failure signatures observed
All failures happen in test setup (
OneTimeSetUp/SetUp, i.e. session creation), never in an actual assertion — confirming this is a session-start/simulator-boot race, not application logic:WDA launch timeout (run #391) —
Session.Logs.LogTests(CanCaptureCrashlogTest,CanCaptureServerTest,CanCaptureSyslogTest,CanRetrieveLogTypesTest— all 4 in the fixture) fail with:WDA app not registered with SpringBoard (same run) —
LockDeviceTest.LockTestWithSecondsfails with:Simulator boot timeout (run #385) —
AlertTests.BeforeAllfails with:Element-wait timeout downstream of the same slowness (run fix: IHidesKeyboard interface does not include 'key' and 'strategy' overloads - (#390) and (#389) #391) —
AppTests.CanActivateAppFromBackgroundTest:FindElementby accessibility ID times out waiting for the app to return to foreground.Likely root cause
.github/workflows/functional-test.yml'sios-testsjob has no retry or readiness gate around the two flakiest steps:Setup iOS Simulator(futureware-tech/simulator-action@v5,wait_for_boot: true) waits for the simulator to report booted, but doesn't confirm the prebuilt WDA app downloaded in the previous step is actually installed/registered with SpringBoard before tests start.Run iOS functional testsinvokesdotnet testexactly once — no[Retry(n)]on session-dependent fixtures, and no step-level retry/back-off if the whole run fails.Given prebuilt WDA is downloaded and referenced via
LOCAL_PREBUILT_WDArather than built by Xcode locally, there's a plausible race between the simulator reporting "booted" and it having finished registering the just-installed WDA app with FrontBoard/SpringBoard — matching signatures 1 and 2 above.Suggested next steps
xcrun simctl/ attempt a throwaway session) afterSetup iOS Simulatorand beforeRun iOS functional tests, instead of relying solely onwait_for_boot.wdaLaunchTimeout/wdaConnectionTimeoutcapabilities used by the iOS test setup.[Retry(2)](NUnit) to the most session-fragile fixtures, or a CI-level retry of the wholedotnet testinvocation on failure.xcrun simctl bootstatus <udid> -brather than trustingwait_for_boot: truealone.