Azure Responses API

Azure OpenAI의 Responses API를 LiteLLM의 azure/ 라우트로 호출해요.

출처: 문서

본문

개요 (Overview)

속성	설명
설명	Azure OpenAI Responses API
LiteLLM의 custom_llm_provider	`azure/`
지원 작업	`/v1/responses`
공급자 문서	Azure OpenAI Responses API
비용/로깅 지원	✅ LiteLLM이 Responses API 요청 로깅·비용 추적
지원 OpenAI 파라미터	✅ 모든 OpenAI 파라미터 지원

사용법 (Usage)

모델 응답 생성 (Non-streaming)

LiteLLM SDK:

import litellm

# Non-streaming response
response = litellm.responses(
    model="azure/o1-pro",
    input="Tell me a three sentence bedtime story about a unicorn.",
    max_output_tokens=100,
    api_key=os.getenv("AZURE_RESPONSES_OPENAI_API_KEY"),
    api_base="https://litellm8397336933.openai.azure.com/",
    api_version="2023-03-15-preview",
)
print(response)

Streaming:

import litellm

# Streaming response
response = litellm.responses(
    model="azure/o1-pro",
    input="Tell me a three sentence bedtime story about a unicorn.",
    stream=True,
    api_key=os.getenv("AZURE_RESPONSES_OPENAI_API_KEY"),
    api_base="https://litellm8397336933.openai.azure.com/",
    api_version="2023-03-15-preview",
)

for event in response:
    print(event)

OpenAI SDK with LiteLLM Proxy

먼저 proxy config.yaml에 추가:

model_list:
  - model_name: o1-pro
    litellm_params:
      model: azure/o1-pro
      api_key: os.environ/AZURE_RESPONSES_OPENAI_API_KEY
      api_base: https://litellm8397336933.openai.azure.com/
      api_version: 2023-03-15-preview

LiteLLM proxy 시작:

litellm --config /path/to/config.yaml
# RUNNING on http://0.0.0.0:4000

그 다음 proxy를 가리키는 OpenAI SDK 사용:

from openai import OpenAI

# Initialize client with your proxy URL
client = OpenAI(
    base_url="http://localhost:4000",  # Your proxy URL
    api_key="your-api-key"  # Your proxy API key
)

# Non-streaming response
response = client.responses.create(
    model="o1-pro",
    input="Tell me a three sentence bedtime story about a unicorn.",
)
print(response)

Streaming:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:4000",  # Your proxy URL
    api_key="your-api-key"  # Your proxy API key
)

# Streaming response
response = client.responses.create(
    model="o1-pro",
    input="Tell me a three sentence bedtime story about a unicorn.",
    stream=True,
)

for event in response:
    print(event)

Azure Codex 모델

Codex 모델은 Azure의 새 /v1/preview API를 사용하는데, 최신 기능에 지속적으로 접근할 수 있어 매월 api-version을 갱신할 필요가 없어요. LiteLLM은 api_version="preview"로 설정하면 요청을 /v1/preview 엔드포인트로 보내요.

LiteLLM SDK (Non-streaming):

import litellm

# Non-streaming response with Codex models
response = litellm.responses(
    model="azure/codex-mini",
    input="Tell me a three sentence bedtime story about a unicorn.",
    max_output_tokens=100,
    api_key=os.getenv("AZURE_RESPONSES_OPENAI_API_KEY"),
    api_base="https://litellm8397336933.openai.azure.com",
    api_version="preview",  # 👈 key difference
)
print(response)

Streaming:

import litellm

# Streaming response with Codex models
response = litellm.responses(
    model="azure/codex-mini",
    input="Tell me a three sentence bedtime story about a unicorn.",
    stream=True,
    api_key=os.getenv("AZURE_RESPONSES_OPENAI_API_KEY"),
    api_base="https://litellm8397336933.openai.azure.com",
    api_version="preview",  # 👈 key difference
)

for event in response:
    print(event)

Proxy config.yaml:

model_list:
  - model_name: codex-mini
    litellm_params:
      model: azure/codex-mini
      api_key: os.environ/AZURE_RESPONSES_OPENAI_API_KEY
      api_base: https://litellm8397336933.openai.azure.com
      api_version: preview  # 👈 key difference

OpenAI SDK with LiteLLM Proxy:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:4000",  # Your proxy URL
    api_key="your-api-key"  # Your proxy API key
)

# Non-streaming response
response = client.responses.create(
    model="codex-mini",
    input="Tell me a three sentence bedtime story about a unicorn.",
)
print(response)

Responses 모델을 completion으로 사용하기

model="azure/responses/<deployment>" 패턴으로 completion 호출에도 Responses 모델을 쓸 수 있어요.

LiteLLM SDK:

from litellm import completion
import os

os.environ["AZURE_API_BASE"] = "https://my-azure-endpoint.openai.azure.com/"
os.environ["AZURE_API_VERSION"] = "2023-03-15-preview"
os.environ["AZURE_API_KEY"] = "my-api-key"

response = completion(
    model="azure/responses/my-custom-o1-pro",
    messages=[{"role": "user", "content": "Hello world"}],
)
print(response)

Proxy config.yaml:

model_list:
  - model_name: my-custom-o1-pro
    litellm_params:
      model: azure/responses/my-custom-o1-pro
      api_key: os.environ/AZURE_API_KEY
      api_base: https://my-azure-endpoint.openai.azure.com/
      api_version: 2023-03-15-preview

cURL:

curl http://localhost:4000/v1/chat/completions \
  -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "my-custom-o1-pro",
    "messages": [{"role": "user", "content": "Hello world"}]
  }'

Azure Responses API