제공자 특정 Wildcard 라우팅

제공자 특정 Wildcard 라우팅

제공자의 모든 모델 프록시

config.yaml에 정의하지 않고 특정 제공자의 모든 모델을 프록시하고 싶을 때 사용하세요.

1단계. 제공자 특정 라우팅 정의

• SDK
• PROXY

from litellm import Routerrouter = Router(
    model_list=[
        {
            "model_name": "anthropic/*",
            "litellm_params": {
                "model": "anthropic/*",
                "api_key": os.environ["ANTHROPIC_API_KEY"]
            }
        },
        {
            "model_name": "groq/*",
            "litellm_params": {
                "model": "groq/*",
                "api_key": os.environ["GROQ_API_KEY"]
            }
        },
        {
            "model_name": "fo::*:static::*", # all requests matching this pattern will be routed to this deployment, example: model="fo::hi::static::hi" will be routed to deployment: "openai/fo::*:static::*"
            "litellm_params": {
                "model": "openai/fo::*:static::*",
                "api_key": os.environ["OPENAI_API_KEY"]
            }
        }
    ])

1단계 - config.yaml에서 제공자 특정 라우팅 정의

model_list:
  # provider specific wildcard routing
  - model_name: "anthropic/*"
    litellm_params:
      model: "anthropic/*"
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: "groq/*"
    litellm_params:
      model: "groq/*"
      api_key: os.environ/GROQ_API_KEY
  - model_name: "fo::*:static::*" # all requests matching this pattern will be routed to this deployment, example: model="fo::hi::static::hi" will be routed to deployment: "openai/fo::*:static::*"
    litellm_params:
      model: "openai/fo::*:static::*"
      api_key: os.environ/OPENAI_API_KEY

[PROXY용] 2단계 - litellm proxy 실행

$ litellm --config /path/to/config.yaml

3단계 - 테스트

• SDK
• PROXY

from litellm import Routerrouter = Router(model_list=...)# Test with `anthropic/` - all models with `anthropic/` prefix will get routed to `anthropic/*`resp = completion(model="anthropic/claude-sonnet-5", messages=[{"role": "user", "content": "Hello, Claude!"}])print(resp)# Test with `groq/` - all models with `groq/` prefix will get routed to `groq/*`resp = completion(model="groq/llama3-8b-8192", messages=[{"role": "user", "content": "Hello, Groq!"}])print(resp)# Test with `fo::*::static::*` - all requests matching this pattern will be routed to `openai/fo::*:static::*`resp = completion(model="fo::hi::static::hi", messages=[{"role": "user", "content": "Hello, Claude!"}])print(resp)

anthropic/로 테스트 - anthropic/ 프리픽스가 있는 모든 모델은 anthropic/*로 라우팅됩니다.

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "anthropic/claude-sonnet-5",
    "messages": [
      {"role": "user", "content": "Hello, Claude!"}
    ]
  }'

groq/로 테스트 - groq/ 프리픽스가 있는 모든 모델은 groq/*로 라우팅됩니다.

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "groq/llama3-8b-8192",
    "messages": [
      {"role": "user", "content": "Hello, Groq!"}
    ]
  }'

fo::*::static::*로 테스트 - 이 패턴과 일치하는 모든 요청은 openai/fo::*:static::*로 라우팅됩니다.

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "fo::hi::static::hi",
    "messages": [
      {"role": "user", "content": "Hello, Claude!"}
    ]
  }'

[PROXY용] Wildcard 모델 접근 제어

더 알아보기 (Learn more)

• LiteLLM Router