LLM Proxy for Azure OpenAI

A Flask-based proxy server that forwards requests to Azure OpenAI services while maintaining OpenAI API compatibility.

Features

• HTTPS support with self-signed certificates
• CORS enabled for cross-origin requests
• OpenAI API compatible endpoints
• Azure OpenAI authentication handling
• Streaming responses support
• Dynamic model listing

Prerequisites

• Python 3.8+
• pip
• Azure OpenAI service subscription

Installation

1. Clone the repository

2. Install dependencies:

   pip install flask flask-cors requests pyOpenSSL

SSL Certificate Generation

Generate self-signed certificates for HTTPS:

openssl req -x509 -newkey rsa:4096 -nodes -out localhost.pem -keyout localhost-key.pem -days 365 -subj "/CN=localhost"

Configuration

Set your Azure OpenAI credentials as environment variables:

export AZURE_ENDPOINT="https://<your-instance>.openai.azure.com"
export AZURE_API_KEY="your-api-key"

Or modify these values directly in the code:

AZURE_MODEL_NAME = "DeepSeek-R1"  # Your deployed model name
AZURE_BASE_URL = os.getenv("AZURE_ENDPOINT", "https://<SERVER_NAME>.openai.azure.com")
AZURE_API_KEY = os.getenv("AZURE_API_KEY", "<API_KEY>")

Running the Server

Start the Flask server:

python main.py

The server will run on https://localhost:8085

API Endpoints

Chat Completions

• POST /v1/chat/completions/deployments//chat/completions
• POST /openai/deployments//chat/completions

Model Listing

• GET /v1/models
• GET /v1/chat/completions/models
• GET /openai/deployments//models

Testing with curl

Test the chat completion endpoint (skip SSL verification with -k):

    curl -k -X POST "https://localhost:8085/v1/chat/completions/deployments/DeepSeek-R1/chat/completions" \
    -H "Content-Type: application/json" \
    -d '{
      "messages": [
        {
          "role": "user",
          "content": "Hello, how are you?"
        }
      ],
      "temperature": 0.7,
      "max_tokens": 150
    }'

Error Handling

The server handles various error cases:

• Invalid JSON body
• Azure API authentication errors
• Connection issues
• Model fetch failures

CORS Configuration

CORS is enabled for all origins with the following configuration:

• Allowed methods: GET, POST, OPTIONS
• Allowed headers: Content-Type, Authorization, Accept, Origin, Api-Key

Security Notes

• The server uses self-signed certificates for HTTPS
• In production, replace with proper SSL certificates
• API keys should be properly secured and not hardcoded
• Consider implementing rate limiting for production use

Contributing

Feel free to submit issues and pull requests.

License

MIT License