ci: dispatchable model-onboarding check (one model, one product) - #128
Open
denispetre wants to merge 2 commits into
Open
ci: dispatchable model-onboarding check (one model, one product)#128denispetre wants to merge 2 commits into
denispetre wants to merge 2 commits into
Conversation
… product Runs the langchain integration suite for one model that is not in the committed matrix (injected for the run only) as agents/agenthub/agent-gym/testproduct, against the LLM Gateway env held in a GitHub environment (LLMGW_SETTINGS = alpha; a prod-tier environment can be selected once provisioned). Dispatched by hand or by UiPath/llm-gateway-config when a model is routed for agents/agenthub in prod. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
denispetre
requested review from
DragosBobolea,
cosminacho,
cristipufu,
dragosvelcea,
ionmincu,
ionut-mihalache-uipath,
radu-mocanu and
vldcmp-uipath
as code owners
August 27, 2026 08:39
…dApiFlavors Each vendor client speaks one gateway API flavor; run a model only through the clients whose flavor it declares (Gemini generateContent, Anthropic Messages, Bedrock Invoke vs Converse, OpenAI chat-completions vs responses), composing the bedrock/openai/litellm entries per flavor instead of guessing by family. New api_flavors + operations inputs. One documented exception: Vertex-hosted Claude entries declare GeminiGenerateContent as a stand-in for Vertex passthrough, which for Claude is Anthropic Messages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
model-onboarding.yml(workflow_dispatch): run this repo's langchain integration suite for a single model not in the committed matrix — typically one the LLM Gateway just onboarded — as a chosen requesting product (agents/agenthub/agent-gym/testproduct), against the gateway environment held in a GitHub environment of this repo (LLMGW_SETTINGS= alpha today; a prod-tier one likeLLMGW_SETTINGS_PRODonce provisioned).Client selection is driven by the gateway's declared
SupportedApiFlavors(inputapi_flavors), not by model family — each vendor client speaks one flavor, so a model is only exercised through clients whose flavor it declares:GeminiGenerateContentUiPathChatGoogleGenerativeAI), litellmGEMINI_CONFIGSAnthropicMessagesUiPathChatAnthropic, vendor_type by vendor) + vertexai / bedrock Anthropic clients, litellm Vertex ClaudeAwsBedrockInvoke/AwsBedrockConverseUiPathChatBedrock/UiPathChatBedrockConverse, litellm invoke / converseOpenAiChatCompletions/OpenAiResponsesUiPathAzureChatOpenAI(plain /use_responses_api), litellm chat / responsesNormalizedChat(operation)UiPathChat, thinking/reasoning config by familyMixed lists (bedrock, openai, litellm) are composed per flavor so e.g. a Converse-only Bedrock model never runs the Invoke client. One documented exception: the gateway declares Vertex-hosted Claude as
GeminiGenerateContent(its flavor enum has no Vertex-Anthropic value; it means "Vertex passthrough"), which the script maps to the Anthropic/Vertex clients.scripts/inject_model_into_matrix.pyinserts the model intoCOMPLETIONS_MODELS_WITH_CONFIGSfor the run only. Nothing is committed; the cassette DB is neither uploaded nor committed.ci.yml's test job. Product-specific feature/operation-code headers derive from the product input.Verified: gemini-3.7-flash ran 110/110 green through this injection against alpha; the flavor composer was exercised on Gemini, Bedrock (Invoke+Converse+Messages), GPT (chat+responses) and Vertex Claude cases.
🤖 Generated with Claude Code