Introducing openai-compat provider

[neoromantic]

OpenAI-Compatible Provider

Objective

Ax should support any OpenAI-compatible endpoint (Groq, Cerebras, Vercel AI Gateway, Fireworks, custom proxies, etc.) without forcing bespoke adapters. Contributors need a reusable transport that:

• Reuses the existing OpenAI request serialization (chat + embeddings + streaming) so features such as tools and thinking budgets behave consistently.
• Lets users set an arbitrary base_url, API key, and optional extra headers/query params required by gateways.
• Documents provider-specific caveats (e.g., Groq rejecting logit_bias, Cerebras disallowing certain penalties) so callers can adjust prompts/config.

References

• Cerebras OpenAI compatibility docs: https://inference-docs.cerebras.ai/resources/openai
• Groq OpenAI compatibility docs: https://console.groq.com/docs/openai
• Existing adapters: src/ax/ai/openai/api.ts (serializer), src/ax/ai/openrouter/api.ts (shows how to reuse OpenAI base), src/ax/ai/wrap.ts (factory).

Implementation Plan

1. New Adapter (AxAIOpenAICompatible)
   • File: src/ax/ai/openai-compatible/api.ts.
   • Extends AxAIOpenAIBase but accepts endpoint, apiKey, optional extraHeaders, optional supportFor override, and optional providerName.
   • Defaults config via axAIOpenAIDefaultConfig() but allow overrides.
   • Provide a helper to merge user-defined headers and fall back to Authorization: Bearer <apiKey> plus optional HTTP-Referer, etc.
2. Factory Wiring
   • Update AxAIArgs union + constructor switch (src/ax/ai/wrap.ts) to recognize name: 'openai-compatible'.
   • Export the new args/class from src/ax/ai/index.ts.
3. Documentation & Examples
   • README.md (and llm.txt via docs) gets a “Using OpenAI-Compatible Providers” section with Groq + Cerebras + Vercel snippets (ai({ name: 'openai-compatible', endpoint: 'https://api.groq.com/openai/v1', config: { model: 'groq/llama3-70b-8192' } })).
   • Mention known unsupported parameters per provider and recommend disabling them via config.
   • Optional new example file under src/examples/openai-compatible.ts showing environment variable usage (AI_COMPAT_API_URL, AI_COMPAT_API_KEY).
4. Tests
   • Unit tests in src/ax/ai/openai-compatible/api.test.ts verifying:
     • Custom endpoint + headers propagate to apiCall.
     • Serializer still matches OpenAI baseline.
     • supportFor override works.
   • (Optional) integration smoke test keyed off env vars; skip when undefined.
5. Release Notes & Changelog

• Add entry in CHANGELOG.md under “Unreleased” or upcoming version.

Open Questions / Follow-ups

• Should we surface provider-specific validation (e.g., warn if user sets logit_bias while targeting Groq)? For scope we’ll document the limitations but not enforce them, while leaving room for per-provider presets via models[].
• Need to validate streaming usage with gateways; if any require stream_options, confirm compatibility.

Next steps: implement adapter + tests + docs, then update this file with outcomes and verification notes.

---

Implementation Status

✅ OpenAI-Compatible Adapter

• Added AxAIOpenAICompatible in src/ax/ai/openai-compatible/api.ts, reusing the OpenAI serializer but allowing arbitrary endpoint, headers, providerName, and capability overrides.
• Wired the adapter into the ai() factory + AxAIArgs union (src/ax/ai/wrap.ts) so ai({ name: 'openai-compatible', ... }) just works.
• Generated exports via npm run build:index -w src/ax so AxAIOpenAICompatible is publicly available for direct imports.

✅ Tooling & Config

• Added explicit vitest.config.ts files under src/ax/ and src/tools/ to pin Vitest to the workspace root and avoid leaking user-level vite.config.ts.

✅ Docs & Examples

• README now has an “OpenAI-Compatible Providers” section (Groq, Cerebras, Vercel AI Gateway guidance + env vars) and the example list references a new src/examples/openai-compatible.ts.
• New example demonstrates configuring env vars (AI_COMPAT_API_URL, AI_COMPAT_API_KEY, optional AI_COMPAT_PROVIDER_HEADER) and logging the response.

✅ Tests

• New unit tests in src/ax/ai/openai-compatible/api.test.ts ensure endpoint/header wiring + capability overrides.
• Updated src/ax/ai/integration.test.ts to cover the openai-compatible branch of the factory.
• Vitest now runs cleanly for both src/ax and src/tools thanks to the pinned configs.

✅ Changelog

• Added an “Unreleased” entry noting the new provider + configs (see CHANGELOG.md).

Testing Evidence

Commands executed in repo root unless noted:

1. npm run test:type-check -w src/examples – ensured new example type-checks.
2. npm run test:unit -w src/ax – ran the full Ax package Vitest suite (69 files / 808 tests).
3. npm run test:unit -w src/tools – ran tools package Vitest suite (2 files / 4 tests).

Note: the top-level npm install triggered workspace tests automatically; with the new Vitest configs in place the suites now pass locally. If another developer has a parent-level vite.config.ts (e.g., in ~/Developer), these new per-workspace configs prevent Vitest from picking it up.

Follow-ups

• Documentation explicitly states provider-specific parameter gaps, but there’s no runtime validation. Consider logging warnings when disallowed params (logit_bias, presence_penalty, etc.) are set while targeting a custom model preset.
• Evaluate adding helpers/presets for Groq/Cerebras inside models[] to simplify multi-model routing (not required for this PR).