verifytypes check has fragile failure modes when pyright produces no output
Summary
The verifytypes CI check for azure-ai-transcription is consistently failing with exit code 2 (unhandled exception) or exit code 1 (after the fix in #46418). The root cause is that pyright runs for ~71 seconds during the install_from_main comparison step and then exits with code 1 while producing completely empty stdout and stderr. This causes json.loads("") to throw JSONDecodeError.
Observed behavior
- The first pyright
--verifytypes run (against the PR version of the package) succeeds — 100% type completeness score.
install_from_main sparse-clones main, does uv pip install . --force-reinstall --no-sources --python <path> to replace the package with the main-branch version.- The second pyright
--verifytypes --outputjson run (against the main version) hangs for ~71 seconds and exits code 1 with empty stdout and empty stderr.
CI log evidence
From the 2026-04-20 20:01 run (after the fix in #46418):
2026-04-20 20:01:21,488 [INFO] azure-sdk-tools: Installed azure-ai-transcription/azure_ai_transcription-1.0.0b4.tar.gz
2026-04-20 20:02:32,520 [ERROR] azure-sdk-tools: pyright --verifytypes exited with code 1 but did not produce valid JSON output.
stdout:
stderr:
71 seconds, no output on either stream. This suggests pyright/node.js is either:
- Being OOM-killed on the hosted agent (exit code mapped to 1)
- Hitting some internal error that
--outputjson suppresses
- Encountering something about the main-branch version of the package that causes it to spin
Issues found and fixed in #46418
-
verifytypes.py get_type_complete_score(): json.loads(e.output) on line 202 was unguarded — when pyright exits 1 with empty stdout, JSONDecodeError propagated as an unhandled exception (exit code 2). Now wrapped in try/except with diagnostic logging.
-
Check.py install_dev_reqs(): logger.error("Failed to install dev requirements:", e) used positional args instead of f-string, causing TypeError in the logging framework and swallowing the actual error message. Fixed to logger.error(f"Failed to install dev requirements: {e}") and raise e → bare raise.
Remaining problem
Even with the fixes in #46418, we still don't know why pyright produces no output for the main-branch version of azure-ai-transcription. The --outputjson flag may be suppressing error output that would otherwise be visible. Since both stdout and stderr are empty, there's no way to diagnose the underlying pyright failure from the current logs.
Suggestions
- Consider running pyright without
--outputjson when the JSON parse fails, to capture human-readable diagnostic output.
- Or: always log stderr from the pyright invocation even when
--outputjson is used, since pyright may write errors to stderr independently of JSON output.
- Investigate whether the hosted agent has memory pressure that could cause node.js (pyright's runtime) to get killed during the 71-second run.
- Check whether this is specific to
azure-ai-transcription or affects other packages during the install_from_main comparison step.
Affected package
azure-ai-transcription (sdk/transcription/azure-ai-transcription)
Related
verifytypes check has fragile failure modes when pyright produces no output
Summary
The
verifytypesCI check forazure-ai-transcriptionis consistently failing with exit code 2 (unhandled exception) or exit code 1 (after the fix in #46418). The root cause is that pyright runs for ~71 seconds during theinstall_from_maincomparison step and then exits with code 1 while producing completely empty stdout and stderr. This causesjson.loads("")to throwJSONDecodeError.Observed behavior
--verifytypesrun (against the PR version of the package) succeeds — 100% type completeness score.install_from_mainsparse-clonesmain, doesuv pip install . --force-reinstall --no-sources --python <path>to replace the package with the main-branch version.--verifytypes --outputjsonrun (against the main version) hangs for ~71 seconds and exits code 1 with empty stdout and empty stderr.CI log evidence
From the 2026-04-20 20:01 run (after the fix in #46418):
71 seconds, no output on either stream. This suggests pyright/node.js is either:
--outputjsonsuppressesIssues found and fixed in #46418
verifytypes.pyget_type_complete_score():json.loads(e.output)on line 202 was unguarded — when pyright exits 1 with empty stdout,JSONDecodeErrorpropagated as an unhandled exception (exit code 2). Now wrapped in try/except with diagnostic logging.Check.pyinstall_dev_reqs():logger.error("Failed to install dev requirements:", e)used positional args instead of f-string, causingTypeErrorin the logging framework and swallowing the actual error message. Fixed tologger.error(f"Failed to install dev requirements: {e}")andraise e→ bareraise.Remaining problem
Even with the fixes in #46418, we still don't know why pyright produces no output for the main-branch version of
azure-ai-transcription. The--outputjsonflag may be suppressing error output that would otherwise be visible. Since both stdout and stderr are empty, there's no way to diagnose the underlying pyright failure from the current logs.Suggestions
--outputjsonwhen the JSON parse fails, to capture human-readable diagnostic output.--outputjsonis used, since pyright may write errors to stderr independently of JSON output.azure-ai-transcriptionor affects other packages during theinstall_from_maincomparison step.Affected package
azure-ai-transcription(sdk/transcription/azure-ai-transcription)Related
pyrightinvocation are properly surfaced inverifytypes#46418--no-sourcesand--pythonflags toinstall_from_main)