Skip to content

Add health check gatekeeper for hub integration tests #9320

@shanevcantwell

Description

@shanevcantwell

Problem

Hub integration tests in extensions/cli/src/hubLoader.test.ts can fail spuriously when the Hub API experiences 5xx errors or outages. This creates noise in CI and makes it harder to identify real test failures.

Suggested Solution

Add a health check helper that runs before integration tests and gracefully skips them if the Hub API is unavailable:

describe("Real Hub Integration Tests", () => {
  let hubAvailable = false;
  
  beforeAll(async () => {
    hubAvailable = await isHubAvailable();
    if (!hubAvailable) {
      console.warn("Hub API unavailable - skipping integration tests");
    }
  });

  async function isHubAvailable(): Promise<boolean> {
    try {
      const response = await fetch(`${env.apiBase}health`, {
        signal: AbortSignal.timeout(5000),
      });
      return response.ok;
    } catch {
      return false;
    }
  }

  it("should load rule from real hub", async () => {
    if (!hubAvailable) return;
    // actual test
  });
});

Benefits

  • Prevents spurious CI failures from external service outages
  • Makes tests more reliable without changing functionality
  • Standard practice for integration testing
  • Graceful degradation (logs warning, doesn't fail build)

Context

Discovered while working on PR #9314 - the hub integration tests failed with 5xx errors during an apparent service outage, blocking an unrelated PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:integrationIntegrations (context providers, model providers, etc.)javascriptPull requests that update Javascript codekind:enhancementIndicates a new feature request, imrovement, or extension

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions