Vertex AI - Anthropic, DeepSeek, Model Garden, xAI

Vertex AI - Anthropic, DeepSeek, Model Garden, xAI

Vertex AI의 파트너 모델(Anthropic, DeepSeek, GLM, Llama, Mistral, AI21, Qwen, GPT-OSS, xAI)을 LiteLLM에서 사용하는 방법을 알아봐요.

출처: 문서

본문

지원 파트너 제공사

제공사	LiteLLM 라우트	Vertex 문서
Anthropic (Claude)	vertex_ai/claude-*	Vertex AI - Anthropic Models
DeepSeek	vertex_ai/deepseek-ai/{MODEL}	Vertex AI - DeepSeek Models
ZAI (GLM)	vertex_ai/zai-org/{MODEL}	Vertex AI - GLM Models
Meta/Llama	vertex_ai/meta/{MODEL}	Vertex AI - Meta Models
Mistral	vertex_ai/mistral-*	Vertex AI - Mistral Models
AI21 (Jamba)	vertex_ai/jamba-*	Vertex AI - AI21 Models
Qwen	vertex_ai/qwen/*	Vertex AI - Qwen Models
OpenAI (GPT-OSS)	vertex_ai/openai/gpt-oss-*	Vertex AI - GPT-OSS Models
xAI (Grok)	vertex_ai/xai/{MODEL}	xAI models (incl. Vertex), Vertex AI Model Garden

Vertex AI - Anthropic (Claude)

모델명	함수 호출
claude-sonnet-4-5@20250929	completion('vertex_ai/claude-sonnet-4-5@20250929', messages)
claude-3-opus@20240229	completion('vertex_ai/claude-3-opus@20240229', messages)
claude-3-5-sonnet@20240620	completion('vertex_ai/claude-3-5-sonnet@20240620', messages)
claude-3-sonnet@20240229	completion('vertex_ai/claude-3-sonnet@20240229', messages)
claude-3-haiku@20240307	completion('vertex_ai/claude-3-haiku@20240307', messages)
claude-3-7-sonnet@20250219	completion('vertex_ai/claude-3-7-sonnet@20250219', messages)

사용법

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "claude-sonnet-5"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    temperature=0.7,
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

1. config에 추가:

model_list:
    - model_name: anthropic-vertex
      litellm_params:
        model: vertex_ai/claude-sonnet-5
        vertex_ai_project: "my-test-project"
        vertex_ai_location: "us-east-1"
    - model_name: anthropic-vertex
      litellm_params:
        model: vertex_ai/claude-sonnet-5
        vertex_ai_project: "my-test-project"
        vertex_ai_location: "us-west-1"

2. Proxy 시작:

litellm --config /path/to/config.yaml

# RUNNING at http://0.0.0.0:4000

3. 테스트:

curl --location 'http://0.0.0.0:4000/chat/completions' \
      --header "Authorization: Bearer ***" \
      --header 'Content-Type: application/json' \
      --data '{
            "model": "anthropic-vertex", # 👈 the 'model_name' in config
            "messages": [
                {
                "role": "user",
                "content": "what llm are you"
                }
            ],
        }'

thinking / reasoning_content 사용

from litellm import completion

resp = completion(
    model="vertex_ai/claude-sonnet-5",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    thinking={"type": "enabled", "budget_tokens": 1024},
)

config.yaml:

- model_name: claude-sonnet-5
  litellm_params:
    model: vertex_ai/claude-sonnet-5
    vertex_ai_project: "my-test-project"
    vertex_ai_location: "us-west-1"

Proxy 시작:

litellm --config /path/to/config.yaml

테스트:

curl http://0.0.0.0:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: *** " \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [{"role": "user", "content": "What is the capital of France?"}],
    "thinking": {"type": "enabled", "budget_tokens": 1024}
  }'

예상 응답:

ModelResponse(
    id='chatcmpl-c542d76d-f675-4e87-8e5f-05855f5d0f5e',
    created=1740470510,
    model='claude-sonnet-5',
    object='chat.completion',
    system_fingerprint=None,
    choices=[
        Choices(
            finish_reason='stop',
            index=0,
            message=Message(
                content="The capital of France is Paris.",
                role='assistant',
                tool_calls=None,
                function_call=None,
                provider_specific_fields={
                    'citations': None,
                    'thinking_blocks': [
                        {
                            'type': 'thinking',
                            'thinking': 'The capital of France is Paris. This is a very straightforward factual question.',
                            'signature': 'EuYBCkQYAiJAy6...'
                        }
                    ]
                }
            ),
            thinking_blocks=[
                {
                    'type': 'thinking',
                    'thinking': 'The capital of France is Paris. This is a very straightforward factual question.',
                    'signature': 'EuYBCkQYAiJAy6AGB...'
                }
            ],
            reasoning_content='The capital of France is Paris. This is a very straightforward factual question.'
        )
    ],
    usage=Usage(
        completion_tokens=68,
        prompt_tokens=42,
        total_tokens=110,
        completion_tokens_details=None,
        prompt_tokens_details=PromptTokensDetailsWrapper(
            audio_tokens=None,
            cached_tokens=0,
            text_tokens=None,
            image_tokens=None
        ),
        cache_creation_input_tokens=0,
        cache_read_input_tokens=0
    )
)

VertexAI DeepSeek

속성	내용
라우트	vertex_ai/deepseek-ai/{MODEL}
Vertex 문서	Vertex AI - DeepSeek Models

모든 Vertex AI DeepSeek 모델을 지원해요. 모든 Vertex AI DeepSeek 모델에는 vertex_ai/deepseek-ai/ 접두사를 사용하세요.

모델명	사용법
vertex_ai/deepseek-ai/deepseek-r1-0528-maas	completion('vertex_ai/deepseek-ai/deepseek-r1-0528-maas', messages)

VertexAI ZAI (GLM)

속성	내용
라우트	vertex_ai/zai-org/{MODEL}
Vertex 문서	Vertex AI - GLM Models

모든 Vertex AI GLM 모델을 지원해요. 모든 Vertex AI GLM 모델에는 vertex_ai/zai-org/ 접두사를 사용하세요.

모델명	사용법
vertex_ai/zai-org/glm-4.7-maas	completion('vertex_ai/zai-org/glm-4.7-maas', messages)

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

response = completion(
    model="vertex_ai/zai-org/glm-4.7-maas",
    messages=[{"role": "user", "content": "hi"}],
    vertex_project="your-vertex-project",
    # vertex_location routes to "global"
)
print("\nModel Response", response)

1. config에 추가:

model_list:
  - model_name: glm-4.7
    litellm_params:
      model: vertex_ai/zai-org/glm-4.7-maas
      vertex_project: "my-project"
      # vertex_location routes to "global"

VertexAI Meta/Llama API

모델명	함수 호출
meta/llama-3.2-90b-vision-instruct-maas	completion('vertex_ai/meta/llama-3.2-90b-vision-instruct-maas', messages)
meta/llama3-8b-instruct-maas	completion('vertex_ai/meta/llama3-8b-instruct-maas', messages)
meta/llama3-70b-instruct-maas	completion('vertex_ai/meta/llama3-70b-instruct-maas', messages)
meta/llama3-405b-instruct-maas	completion('vertex_ai/meta/llama3-405b-instruct-maas', messages)
meta/llama-4-scout-17b-16e-instruct-maas	completion('vertex_ai/meta/llama-4-scout-17b-16e-instruct-maas', messages)
meta/llama-4-scout-17-128e-instruct-maas	completion('vertex_ai/meta/llama-4-scout-128b-16e-instruct-maas', messages)
meta/llama-4-maverick-17b-128e-instruct-maas	completion('vertex_ai/meta/llama-4-maverick-17b-128e-instruct-maas',messages)
meta/llama-4-maverick-17b-16e-instruct-maas	completion('vertex_ai/meta/llama-4-maverick-17b-16e-instruct-maas',messages)

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "meta/llama3-405b-instruct-maas"

vertex_ai_project = "your-vertex-project"
vertex_ai_location = "your-vertex-location"

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

VertexAI Mistral API

모든 Vertex AI Mistral 모델을 지원해요. 모든 Vertex AI Mistral 모델에는 vertex_ai/mistral- 접두사를 사용하세요.

속성	내용
라우트	vertex_ai/mistral-{MODEL}
Vertex 문서	Vertex AI - Mistral Models

모델명	함수 호출
mistral-large@latest	completion('vertex_ai/mistral-large@latest', messages)
mistral-large@2407	completion('vertex_ai/mistral-large@2407', messages)
mistral-small-2503	completion('vertex_ai/mistral-small-2503', messages)
mistral-large-2411	completion('vertex_ai/mistral-large-2411', messages)
mistral-nemo@latest	completion('vertex_ai/mistral-nemo@latest', messages)
codestral@latest	completion('vertex_ai/codestral@latest', messages)
codestral@@2405	completion('vertex_ai/codestral@2405', messages)

Codestral FIM 사용

FIM 작업을 위해 OpenAI /v1/completion 엔드포인트로 VertexAI의 Codestral을 호출해요.

from litellm import text_completion
import os

# os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""
# OR run `!gcloud auth print-access-token` in your terminal

model = "codestral@2405"

vertex_ai_project = "your-vertex-project"
vertex_ai_location = "your-vertex-location"

response = text_completion(
    model="vertex_ai/" + model,
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
    prompt="def is_odd(n): \n return n % 2 == 1 \ndef test_is_odd():",
    suffix="return True",                                              # optional
    temperature=0,                                                     # optional
    top_p=1,                                                           # optional
    max_tokens=10,                                                     # optional
    min_tokens=10,                                                     # optional
    seed=10,                                                           # optional
    stop=["return"],                                                   # optional
)

print("\nModel Response", response)

1. config에 추가:

model_list:
    - model_name: vertex-codestral
      litellm_params:
        model: vertex_ai/codestral@2405
        vertex_ai_project: "my-test-project"
        vertex_ai_location: "us-east-1"
    - model_name: vertex-codestral
      litellm_params:
        model: vertex_ai/codestral@2405
        vertex_ai_project: "my-test-project"
        vertex_ai_location: "us-west-1"

3. 테스트:

curl -X POST 'http://0.0.0.0:4000/completions' \
      -H "Authorization: Bearer ***" \
      -H 'Content-Type: application/json' \
      -d '{
            "model": "vertex-codestral", # 👈 the 'model_name' in config
            "prompt": "def is_odd(n): \n return n % 2 == 1 \ndef test_is_odd():",
            "suffix":"return True",
            "temperature":0,
            "top_p":1,
            "max_tokens":10,
            "min_tokens":10,
            "seed":10,
            "stop":["return"],
        }'

VertexAI AI21 모델

모델명	함수 호출
jamba-1.5-mini@001	completion(model='vertex_ai/jamba-1.5-mini@001', messages)
jamba-1.5-large@001	completion(model='vertex_ai/jamba-1.5-large@001', messages)

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "meta/jamba-1.5-mini@001"

vertex_ai_project = "your-vertex-project"
vertex_ai_location = "your-vertex-location"

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

VertexAI Qwen API

속성	내용
라우트	vertex_ai/qwen/{MODEL}
Vertex 문서	Vertex AI - Qwen Models

모든 Vertex AI Qwen 모델을 지원해요. 모든 Vertex AI Qwen 모델에는 vertex_ai/qwen/ 접두사를 사용하세요.

모델명	사용법
vertex_ai/qwen/qwen3-coder-480b-a35b-instruct-maas	completion('vertex_ai/qwen/qwen3-coder-480b-a35b-instruct-maas', messages)
vertex_ai/qwen/qwen3-235b-a22b-instruct-2507-maas	completion('vertex_ai/qwen/qwen3-235b-a22b-instruct-2507-maas', messages)

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "qwen/qwen3-coder-480b-a35b-instruct-maas"

vertex_ai_project = "your-vertex-project"
vertex_ai_location = "your-vertex-location"

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

VertexAI GPT-OSS 모델

속성	내용
라우트	vertex_ai/openai/{MODEL}
Vertex 문서	Vertex AI - GPT-OSS Models

모든 Vertex AI GPT-OSS 모델을 지원해요. 모든 Vertex AI GPT-OSS 모델에는 vertex_ai/openai/ 접두사를 사용하세요.

모델명	사용법
vertex_ai/openai/gpt-oss-20b-maas	completion('vertex_ai/openai/gpt-oss-20b-maas', messages)

reasoning_effort 사용

GPT-OSS 모델은 향상된 추론 능력을 위한 reasoning_effort 파라미터를 지원해요.

from litellm import completion

response = completion(
    model="vertex_ai/openai/gpt-oss-20b-maas",
    messages=[{"role": "user", "content": "Solve this complex problem step by step"}],
    reasoning_effort="low",  # Options: "minimal", "low", "medium", "high"
    vertex_ai_project="your-vertex-project",
    vertex_ai_location="us-central1",
)

config.yaml:

model_list:
- model_name: gpt-oss
  litellm_params:
    model: vertex_ai/openai/gpt-oss-20b-maas
    vertex_ai_project: "my-test-project"
    vertex_ai_location: "us-central1"

테스트:

curl http://0.0.0.0:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: *** " \
  -d '{
    "model": "gpt-oss",
    "messages": [{"role": "user", "content": "Solve this complex problem step by step"}],
    "reasoning_effort": "low"
  }'

VertexAI xAI (Grok)

Vertex AI Model Garden의 xAI Grok 모델은 다른 Model Garden 발행자 모델과 동일한 OpenAI 호환 chat-completions 경로를 사용해요. vertex_ai/xai/ 접두사를 사용하세요(xai/는 XAI_API_KEY를 쓰는 직접 xAI API이므로 사용하지 마세요).

속성	내용
라우트	vertex_ai/xai/{MODEL}
Vertex / xAI 문서	xAI models, Model Garden

모델명	사용법
vertex_ai/xai/grok-4.1-fast-non-reasoning	completion('vertex_ai/xai/grok-4.1-fast-non-reasoning', messages)
vertex_ai/xai/grok-4.1-fast-reasoning	completion('vertex_ai/xai/grok-4.1-fast-reasoning', messages)
vertex_ai/xai/grok-4.20-non-reasoning	completion('vertex_ai/xai/grok-4.20-non-reasoning', messages)
vertex_ai/xai/grok-4.20-reasoning	completion('vertex_ai/xai/grok-4.20-reasoning', messages)

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

vertex_ai_project = "your-vertex-project"  # or os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location"  # or os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/xai/grok-4.1-fast-non-reasoning",
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

1. config에 추가:

model_list:
    - model_name: grok-vertex
      litellm_params:
        model: vertex_ai/xai/grok-4.1-fast-non-reasoning
        vertex_ai_project: "my-test-project"
        vertex_ai_location: "us-central1"

더 알아보기 (Learn more)

• Vertex AI Model Garden 문서
• xAI 제공사 문서