Skip to content

Conversation

@george-mountain
Copy link

This PR adds support for using Azure OpenAI, which is currently not supported as of the time of this PR. Users can now provide their Azure credentials and use any deployed model from their Azure OpenAI instance.

It also closes issue #45.

Configuration Instructions:

Update your conf.yaml:

BASIC_MODEL:
  base_url: $AZURE_API_BASE
  model: $MODEL
  api_key: $AZURE_API_KEY
  api_version: $API_VERSION

Define credentials in your .env file:

AZURE_API_BASE=https://azure-xxxxxx-ai-services.cognitiveservices.azure.com/  # your Azure API base URL
AZURE_API_KEY=xxxx  # your API key
API_VERSION=2024-02-15-preview  # API version
MODEL=gpt-4o  # your deployed model (change if not using gpt-4o)

raise ValueError(f"Invalid LLM Conf: {llm_type}")

is_azure = "api_version" in llm_conf or (
llm_conf.get("base_url") and "azure.com" in llm_conf.get("base_url", "")
Copy link

@JounQin JounQin May 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to use a customm Azure OpenAPI gateway then? Why not just read whether azure_endpoint is available directly or add a new azure_openai?: boolean config instead?

Kind of related to nextai-translator/bob-plugin-openai-translator#157.


It seems #107 would be better.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JounQin
I understand your point about using a boolean for the check, which is simpler and straight to the point. However, I wanted to ensure consistency with the BASIC_MODEL configuration, just like with other BASIC_MODEL llm configs. Also, with custom azure_endpoint, it will still work provided the user provided the api_version key on the config.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#107 is using a new AZURE config entry which is more suitable IMO.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Can't support Azure OpenAI api as configuration guide introduction

2 participants