Vertex AI 자체 배포 모델

Vertex AI 자체 배포 모델 (Self Deployed Models)

Model Garden이나 사용자 정의 엔드포인트를 통해 자신만의 모델을 Vertex AI에 배포하고 사용하는 방법을 알아봐요.

출처: 문서

본문

Model Garden

팁: Vertex Model Garden의 모든 OpenAI 호환 모델을 지원해요.

Model Garden 사용

거의 모든 Vertex Model Garden 모델은 OpenAI 호환 모델이에요.

SDK:

from litellm import completion
import os

## set ENV variables
os.environ["VERTEXAI_PROJECT"] = "hardy-device-38811"
os.environ["VERTEXAI_LOCATION"] = "us-central1"

response = completion(
  model="vertex_ai/openai/",
  messages=[{ "content": "Hello, how are you?","role": "user"}]
)

Proxy:

1. config에 추가:

model_list:
    - model_name: llama3-1-8b-instruct
      litellm_params:
        model: vertex_ai/openai/5464397967697903616
        vertex_ai_project: "my-test-project"
        vertex_ai_location: "us-east-1"

2. Proxy 시작:

litellm --config /path/to/config.yaml

# RUNNING at http://0.0.0.0:4000

3. 테스트:

curl --location 'http://0.0.0.0:4000/chat/completions' \
      --header "Authorization: Bearer ***" \
      --header 'Content-Type: application/json' \
      --data '{
            "model": "llama3-1-8b-instruct", # 👈 the 'model_name' in config
            "messages": [
                {
                "role": "user",
                "content": "what llm are you"
                }
            ],
        }'

비-OpenAI 호환 모델:

from litellm import completion
import os

## set ENV variables
os.environ["VERTEXAI_PROJECT"] = "hardy-device-38811"
os.environ["VERTEXAI_LOCATION"] = "us-central1"

response = completion(
  model="vertex_ai/",
  messages=[{ "content": "Hello, how are you?","role": "user"}]
)

Gemma 모델 (사용자 정의 엔드포인트)

Gemma 모델을 OpenAI 호환 포맷의 사용자 정의 Vertex AI 예측 엔드포인트에 배포해요.

Proxy 사용법:

1. config.yaml에 추가:

model_list:
  - model_name: gemma-model
    litellm_params:
      model: vertex_ai/gemma/gemma-3-12b-it-1222199011122
      api_base: https://ENDPOINT.us-central1-PROJECT.prediction.vertexai.goog/v1/projects/PROJECT_ID/locations/us-central1/endpoints/ENDPOINT_ID:predict
      vertex_project: "my-project-id"
      vertex_location: "us-central1"

2. Proxy 시작:

litellm --config /path/to/config.yaml

3. 테스트:

curl http://0.0.0.0:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "gemma-model",
    "messages": [{"role": "user", "content": "What is machine learning?"}],
    "max_tokens": 100
  }'

SDK 사용법:

from litellm import completion

response = completion(
    model="vertex_ai/gemma/gemma-3-12b-it-1222199011122",
    messages=[{"role": "user", "content": "What is machine learning?"}],
    api_base="https://ENDPOINT.us-central1-PROJECT.prediction.vertexai.goog/v1/projects/PROJECT_ID/locations/us-central1/endpoints/ENDPOINT_ID:predict",
    vertex_project="my-project-id",
    vertex_location="us-central1",
)

MedGemma 모델 (사용자 정의 엔드포인트)

MedGemma 모델을 OpenAI 호환 포맷의 사용자 정의 Vertex AI 예측 엔드포인트에 배포해요. MedGemma 모델은 동일한 vertex_ai/gemma/ 라우트를 사용해요.

Proxy 사용법:

1. config.yaml에 추가:

model_list:
  - model_name: medgemma-model
    litellm_params:
      model: vertex_ai/gemma/medgemma-2b-v1
      api_base: https://ENDPOINT.us-central1-PROJECT.prediction.vertexai.goog/v1/projects/PROJECT_ID/locations/us-central1/endpoints/ENDPOINT_ID:predict
      vertex_project: "my-project-id"
      vertex_location: "us-central1"

2. Proxy 시작:

litellm --config /path/to/config.yaml

3. 테스트:

curl http://0.0.0.0:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "medgemma-model",
    "messages": [{"role": "user", "content": "What are the symptoms of hypertension?"}],
    "max_tokens": 100
  }'

SDK 사용법:

from litellm import completion

response = completion(
    model="vertex_ai/gemma/medgemma-2b-v1",
    messages=[{"role": "user", "content": "What are the symptoms of hypertension?"}],
    api_base="https://ENDPOINT.us-central1-PROJECT.prediction.vertexai.goog/v1/projects/PROJECT_ID/locations/us-central1/endpoints/ENDPOINT_ID:predict",
    vertex_project="my-project-id",
    vertex_location="us-central1",
)

더 알아보기 (Learn more)

• Vertex AI Model Garden 문서
• Vertex AI 예측 문서