-
Notifications
You must be signed in to change notification settings - Fork 3.9k
Open
Labels
area:integrationIntegrations (context providers, model providers, etc.)Integrations (context providers, model providers, etc.)javascriptPull requests that update Javascript codePull requests that update Javascript codekind:enhancementIndicates a new feature request, imrovement, or extensionIndicates a new feature request, imrovement, or extension
Description
Problem
Hub integration tests in extensions/cli/src/hubLoader.test.ts can fail spuriously when the Hub API experiences 5xx errors or outages. This creates noise in CI and makes it harder to identify real test failures.
Suggested Solution
Add a health check helper that runs before integration tests and gracefully skips them if the Hub API is unavailable:
describe("Real Hub Integration Tests", () => {
let hubAvailable = false;
beforeAll(async () => {
hubAvailable = await isHubAvailable();
if (!hubAvailable) {
console.warn("Hub API unavailable - skipping integration tests");
}
});
async function isHubAvailable(): Promise<boolean> {
try {
const response = await fetch(`${env.apiBase}health`, {
signal: AbortSignal.timeout(5000),
});
return response.ok;
} catch {
return false;
}
}
it("should load rule from real hub", async () => {
if (!hubAvailable) return;
// actual test
});
});Benefits
- Prevents spurious CI failures from external service outages
- Makes tests more reliable without changing functionality
- Standard practice for integration testing
- Graceful degradation (logs warning, doesn't fail build)
Context
Discovered while working on PR #9314 - the hub integration tests failed with 5xx errors during an apparent service outage, blocking an unrelated PR.
Metadata
Metadata
Assignees
Labels
area:integrationIntegrations (context providers, model providers, etc.)Integrations (context providers, model providers, etc.)javascriptPull requests that update Javascript codePull requests that update Javascript codekind:enhancementIndicates a new feature request, imrovement, or extensionIndicates a new feature request, imrovement, or extension
Type
Projects
Status
Todo