Volcano Engine
Volcano Engine (Volcengine)
https://www.volcengine.com/docs/82379/1263482
Chat과 Embeddings를 포함한 모든 Volcengine 모델을 지원해요. LiteLLM 요청 시
model=volcengine/<any-model-on-volcengine>처럼 접두어로volcengine/를 붙이면 돼요.
API 키 (API Key)
# env variable
os.environ['VOLCENGINE_API_KEY']
# or
os.environ['ARK_API_KEY']
샘플 사용법 (Sample Usage)
from litellm import completion
import os
os.environ['VOLCENGINE_API_KEY'] = ""
response = completion(
model="volcengine/<OUR_ENDPOINT_ID>",
messages=[
{
"role": "user",
"content": "What's the weather like in Boston today in Fahrenheit?",
}
],
temperature=0.2, # optional
top_p=0.9, # optional
frequency_penalty=0.1, # optional
presence_penalty=0.1, # optional
max_tokens=10, # optional
stop=["\n\n"], # optional
)
print(response)
샘플 사용법 - 스트리밍 (Sample Usage - Streaming)
from litellm import completion
import os
os.environ['VOLCENGINE_API_KEY'] = ""
response = completion(
model="volcengine/<OUR_ENDPOINT_ID>",
messages=[
{
"role": "user",
"content": "What's the weather like in Boston today in Fahrenheit?",
}
],
stream=True,
temperature=0.2, # optional
top_p=0.9, # optional
frequency_penalty=0.1, # optional
presence_penalty=0.1, # optional
max_tokens=10, # optional
stop=["\n\n"], # optional
)
for chunk in response:
print(chunk)
샘플 사용법 - Embedding
from litellm import embedding
import os
os.environ['VOLCENGINE_API_KEY'] = ""
response = embedding(
model="volcengine/doubao-embedding-text-240715",
input=["hello world", "good morning"]
)
print(response)
지원 Embedding 모델 (Supported Embedding Models)
doubao-embedding-large(2048 dimensions)doubao-embedding-large-text-250515(2048 dimensions)doubao-embedding-large-text-240915(4096 dimensions)doubao-embedding(2560 dimensions)doubao-embedding-text-240715(2560 dimensions)
Embedding 파라미터 (Embedding Parameters)
from litellm import embedding
response = embedding(
model="volcengine/doubao-embedding-text-240715",
input=["sample text"],
encoding_format="float", # optional: "float" (default), "base64"
user="user-123", # optional: user identifier for tracking
)
지원 모델 - 💥 모든 Volcengine 모델 지원!
chat completions와 embeddings 모두에 대해 모든 volcengine 모델을 지원해요.
- Chat 모델: completion 요청 시
volcengine/<OUR_ENDPOINT_ID>접두어 설정 - Embedding 모델: 위에 나열된 특정 모델명 사용 (예:
volcengine/doubao-embedding-text-240715)
샘플 사용법 - LiteLLM Proxy
Config.yaml 설정
model_list:
# Chat model
- model_name: volcengine-model
litellm_params:
model: volcengine/<OUR_ENDPOINT_ID>
api_key: os.environ/VOLCENGINE_API_KEY
# Embedding model
- model_name: volcengine-embedding
litellm_params:
model: volcengine/doubao-embedding-text-240715
api_key: os.environ/VOLCENGINE_API_KEY
요청 보내기
Chat Completion
curl --location 'http://localhost:4000/chat/completions' \
--header "Authorization: Bearer ***" \
--header 'Content-Type: application/json' \
--data '{
"model": "volcengine-model",
"messages": [
{
"role": "user",
"content": "here is my api key. openai_api_key=sk-<your-api-key>"
}
]
}'
Embedding
curl --location 'http://localhost:4000/embeddings' \
--header "Authorization: Bearer ***" \
--header 'Content-Type: application/json' \
--data '{
"model": "volcengine-embedding",
"input": ["hello world", "good morning"]
}'
출처: 문서