Vertex AI 임베딩

Vertex AI 임베딩 (Embedding)

LiteLLM에서 Vertex AI 임베딩 모델을 사용하는 방법을 알아봐요. 텍스트 임베딩, BGE, 멀티모달 임베딩까지 지원해요.

출처: 문서

본문

사용법 - 임베딩

import litellm
from litellm import embedding
litellm.vertex_project = "hardy-device-38811" # Your Project ID
litellm.vertex_location = "us-central1"  # proj location

response = embedding(
    model="vertex_ai/textembedding-gecko",
    input=["good morning from litellm"],
)
print(response)

config.yaml에 모델 추가:

model_list:
  - model_name: snowflake-arctic-embed-m-long-1731622468876
    litellm_params:
      model: vertex_ai/
      vertex_project: "adroit-crow-413218"
      vertex_location: "us-central1"
      vertex_credentials: adroit-crow-413218-a956eef1a2a8.json

litellm_settings:
  drop_params: True

Proxy 시작:

$ litellm --config /path/to/config.yaml

OpenAI Python SDK로 요청:

import openai

client = openai.OpenAI(api_key="sk-", base_url="http://0.0.0.0:4000")

response = client.embeddings.create(
    model="snowflake-arctic-embed-m-long-1731622468876",
    input = ["good morning from litellm", "this is another item"],
)

print(response)

지원되는 임베딩 모델

여기에 나열된 모든 모델이 지원돼요.

모델명	함수 호출
text-embedding-004	`embedding(model="vertex_ai/text-embedding-004", input)`
text-multilingual-embedding-002	`embedding(model="vertex_ai/text-multilingual-embedding-002", input)`
textembedding-gecko	`embedding(model="vertex_ai/textembedding-gecko", input)`
textembedding-gecko-multilingual	`embedding(model="vertex_ai/textembedding-gecko-multilingual", input)`
textembedding-gecko-multilingual@001	`embedding(model="vertex_ai/textembedding-gecko-multilingual@001", input)`
textembedding-gecko@001	`embedding(model="vertex_ai/textembedding-gecko@001", input)`
textembedding-gecko@003	`embedding(model="vertex_ai/textembedding-gecko@003", input)`
text-embedding-preview-0409	`embedding(model="vertex_ai/text-embedding-preview-0409", input)`
text-multilingual-embedding-preview-0409	`embedding(model="vertex_ai/text-multilingual-embedding-preview-0409", input)`
gemini-embedding-2-preview	`embedding(model="vertex_ai/gemini-embedding-2-preview", input)`
gemini-embedding-2 (GA)	`embedding(model="vertex_ai/gemini-embedding-2", input)`
Fine-tuned OR Custom Embedding models	`embedding(model="vertex_ai/", input)`

지원되는 OpenAI (통합) 파라미터

param	type	vertex 대응
`input`	string 또는 List[string]	`instances`
`dimensions`	int	`output_dimensionality`
`input_type`	Literal["RETRIEVAL_QUERY","RETRIEVAL_DOCUMENT", "SEMANTIC_SIMILARITY", "CLASSIFICATION", "CLUSTERING", "QUESTION_ANSWERING", "FACT_VERIFICATION"]	`task_type`

OpenAI (통합) 파라미터와 함께 사용

response = litellm.embedding(
    model="vertex_ai/text-embedding-004",
    input=["good morning from litellm", "gm"],
    input_type = "RETRIEVAL_DOCUMENT",
    dimensions=1,
)

import openai

client = openai.OpenAI(api_key="sk-", base_url="http://0.0.0.0:4000")

response = client.embeddings.create(
    model="text-embedding-004",
    input = ["good morning from litellm", "gm"],
    dimensions=1,
    extra_body = {
        "input_type": "RETRIEVAL_QUERY",
    }
)

print(response)

지원되는 Vertex 전용 파라미터

param	type
`auto_truncate`	bool
`task_type`	Literal["RETRIEVAL_QUERY","RETRIEVAL_DOCUMENT", "SEMANTIC_SIMILARITY", "CLASSIFICATION", "CLUSTERING", "QUESTION_ANSWERING", "FACT_VERIFICATION"]
`title`	str

Vertex 전용 파라미터 (`task_type` 및 `title`)와 함께 사용

모든 vertex 전용 파라미터를 임베딩 함수에 그대로 전달할 수 있어요:

response = litellm.embedding(
    model="vertex_ai/text-embedding-004",
    input=["good morning from litellm", "gm"],
    task_type = "RETRIEVAL_DOCUMENT",
    title = "test",
    dimensions=1,
    auto_truncate=True,
)

import openai

client = openai.OpenAI(api_key="sk-", base_url="http://0.0.0.0:4000")

response = client.embeddings.create(
    model="text-embedding-004",
    input = ["good morning from litellm", "gm"],
    dimensions=1,
    extra_body = {
        "task_type": "RETRIEVAL_QUERY",
        "auto_truncate": True,
        "title": "test",
    }
)

print(response)

BGE 임베딩

Vertex AI에 배포된 BGE(Baidu General Embedding) 모델을 사용해요.

import litellm

response = litellm.embedding(
    model="vertex_ai/bge/",
    input=["Hello", "World"],
    vertex_project="your-project-id",
    vertex_location="your-location"
)

print(response)

config.yaml:

model_list:
  - model_name: bge-embedding
    litellm_params:
      model: vertex_ai/bge/
      vertex_project: "your-project-id"
      vertex_location: "us-central1"
      vertex_credentials: your-credentials.json

litellm_settings:
  drop_params: True

Proxy 시작:

$ litellm --config /path/to/config.yaml

OpenAI Python SDK로 요청:

import openai

client = openai.OpenAI(api_key="sk-", base_url="http://0.0.0.0:4000")

response = client.embeddings.create(
    model="bge-embedding",
    input=["good morning from litellm", "this is another item"]
)

print(response)

Private Service Connect(PSC) 엔드포인트 사용 config.yaml (PSC):

model_list:
  - model_name: bge-small-en-v1.5
    litellm_params:
      model: vertex_ai/bge/1234567890
      api_base: http://10.96.32.8  # Your PSC IP
      vertex_project: my-project-id  #optional
      vertex_location: us-central1 #optional

멀티모달 임베딩

Gemini Embedding 2 Preview (Multimodal)

gemini-embedding-2-preview는 텍스트·이미지·오디오·비디오·PDF를 단일 요청으로 처리하는 통합 멀티모달 임베딩을 지원해요. GA 모델 id gemini-embedding-2도 같은 동작을 노출하므로 아래 예시에서 모델명만 바꾸면 돼요.

응답 형태: Vertex는 하나의 결합 벡터를 반환해요.

Vertex AI의 Gemini 임베딩 엔드포인트는 단일 콘텐츠 embedContent만 노출하므로(batchEmbedContents 없음), input=[...]에 N개 항목을 전달하면 N개의 별도 벡터가 아니라 모든 부분을 융합한 1개의 통합 임베딩이 반환돼요. 항목당 벡터 하나를 얻으려면 입력별로 embedding(...)을 한 번씩 호출해요. 이는 입력 요소당 임베딩 하나를 반환하는 Gemini API 경로(gemini/gemini-embedding-2-preview)와 다르다는 점에 유의해요.

입력 형식:

Data URIs: data:image/png;base64,
GCS URLs: gs://bucket/path/to/file.png (MIME 타입은 확장자에서 유추)

지원 MIME 타입: image/png, image/jpeg, audio/mpeg, audio/wav, video/mp4, video/quicktime, application/pdf

import litellm
from litellm import embedding

litellm.vertex_project = "your-project-id"
litellm.vertex_location = "us-central1"

# Text + Image (GCS URL)
response = embedding(
    model="vertex_ai/gemini-embedding-2-preview",
    input=[
        "Describe this image",
        "gs://my-bucket/images/photo.png"
    ],
)

# Text + Image (base64)
response = embedding(
    model="vertex_ai/gemini-embedding-2-preview",
    input=[
        "The food was delicious",
        "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAgAAAAIAQMAAAD+wSzIAAAABlBMVEX///+/v7+jQ3Y5AAAADklEQVQI12P4AIX8EAgALgAD/aNpbtEAAAAASUVORK5CYII"
    ],
)

model_list:
  - model_name: vertex-gemini-embedding-2-preview
    litellm_params:
      model: vertex_ai/gemini-embedding-2-preview
      vertex_project: "your-project-id"
      vertex_location: "us-central1"

curl -X POST http://localhost:4000/embeddings \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "vertex-gemini-embedding-2-preview",
    "input": ["Describe this", "gs://bucket/image.png"]
  }'

multimodalembedding@001 (Legacy)

알려진 제한:

요청당 이미지/비디오 1개만 지원
GCS 또는 base64 인코딩 이미지/비디오만 지원

사용법:

# Using GCS Images
response = await litellm.aembedding(
    model="vertex_ai/multimodalembedding@001",
    input="gs://cloud-samples-data/vertex-ai/llm/prompts/landmark1.png" # will be sent as a gcs image
)

# Using base2 encoded images
response = await litellm.aembedding(
    model="vertex_ai/multimodalembedding@001",
    input="data:image/jpeg;base64,..." # will be sent as a base64 encoded image
)

config.yaml:

model_list:
  - model_name: multimodalembedding@001
    litellm_params:
      model: vertex_ai/multimodalembedding@001
      vertex_project: "adroit-crow-413218"
      vertex_location: "us-central1"
      vertex_credentials: adroit-crow-413218-a956eef1a2a8.json

litellm_settings:
  drop_params: True

Proxy 시작:

$ litellm --config /path/to/config.yaml

OpenAI SDK:

import openai

client = openai.OpenAI(api_key="sk-", base_url="http://0.0.0.0:4000")

response = client.embeddings.create(
    model="multimodalembedding@001",
    input = "gs://cloud-samples-data/vertex-ai/llm/prompts/landmark1.png",
)

print(response)

텍스트 + 이미지 + 비디오 임베딩

# Text + Image
response = await litellm.aembedding(
    model="vertex_ai/multimodalembedding@001",
    input=["hey", "gs://cloud-samples-data/vertex-ai/llm/prompts/landmark1.png"] # will be sent as a gcs image
)

# Text + Video
response = await litellm.aembedding(
    model="vertex_ai/multimodalembedding@001",
    input=["hey", "gs://my-bucket/embeddings/supermarket-video.mp4"] # will be sent as a gcs image
)

# Image + Video
response = await litellm.aembedding(
    model="vertex_ai/multimodalembedding@001",
    input=["gs://cloud-samples-data/vertex-ai/llm/prompts/landmark1.png", "gs://my-bucket/embeddings/supermarket-video.mp4"] # will be sent as a gcs image
)

config.yaml:

model_list:
  - model_name: multimodalembedding@001
    litellm_params:
      model: vertex_ai/multimodalembedding@001
      vertex_project: "adroit-crow-413218"
      vertex_location: "us-central1"
      vertex_credentials: adroit-crow-413218-a956eef1a2a8.json

litellm_settings:
  drop_params: True

OpenAI SDK로 요청:

import openai

client = openai.OpenAI(api_key="sk-", base_url="http://0.0.0.0:4000")

# Text + Image
response = client.embeddings.create(
    model="multimodalembedding@001",
    input = ["hey", "gs://cloud-samples-data/vertex-ai/llm/prompts/landmark1.png"],
)

Vertex SDK 직접 사용도 가능해요. default_vertex_config:

default_vertex_config:
  vertex_project: "adroit-crow-413218"
  vertex_location: "us-central1"
  vertex_credentials: adroit-crow-413218-a956eef1a2a8.json

더 알아보기 (Learn more)

Vertex AI 임베딩 문서
Gemini 임베딩 문서

Vertex AI 임베딩