Bytez
Bytez
LiteLLM은 Bytez의 모든 chat 모델을 지원해요. 멀티모달 모델도 지원돼요 🔥
지원 태스크: chat, image-text-to-text, audio-text-to-text, video-text-to-text
출처: 문서
본문
사용법 (Usage)
API 키
import os
os.environ["BYTEZ_API_KEY"] = "YOUR_BYTEZ_KEY_GOES_HERE"
SDK
from litellm import completion
import os
## set ENV variables
os.environ["BYTEZ_API_KEY"] = "YOUR_BYTEZ_KEY_GOES_HERE"
response = completion(
model="bytez/google/gemma-3-4b-it",
messages = [{ "content": "Hello, how are you?","role": "user"}]
)
Proxy
config.yaml에 모델 추가:
model_list:
- model_name: gemma-3
litellm_params:
model: bytez/google/gemma-3-4b-it
api_key: os.environ/BYTEZ_API_KEY
Proxy 시작:
$ BYTEZ_API_KEY=YOUR_BYTEZ_API_KEY_HERE litellm --config /path/to/config.yaml --debug
LiteLLM Proxy Server로 요청:
OpenAI Python v1.0.0+:
import openai
client = openai.OpenAI(
api_key="sk-<your-litellm-api-key>", # pass litellm proxy key, if you're using virtual keys
base_url="http://0.0.0.0:4000" # litellm-proxy-base url
)
response = client.chat.completions.create(
model="gemma-3",
messages = [
{
"role": "system",
"content": "Be a good human!"
},
{
"role": "user",
"content": "What do you know about earth?"
}
]
)
print(response)
curl:
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header "Authorization: Bearer ***" \
--header 'Content-Type: application/json' \
--data '{
"model": "gemma-3",
"messages": [
{
"role": "system",
"content": "Be a good human!"
},
{
"role": "user",
"content": "What do you know about earth?"
}
],
}'
자동 프롬프트 템플릿 처리
messages 목록을 보내면 모든 프롬프트 포맷팅이 API에서 자동으로 처리돼요. 사용자 지정 포맷팅을 원하면 [email protected] 또는 Discord로 연락하면 제공해 드려요.
추가 파라미터 전달 - max_tokens, temperature
SDK:
# !uv add litellm
from litellm import completion
import os
## set ENV variables
os.environ["BYTEZ_API_KEY"] = "YOUR_BYTEZ_KEY_HERE"
# bytez gemma-3 call
response = completion(
model="bytez/google/gemma-3-4b-it",
messages = [{ "content": "Hello, how are you?","role": "user"}],
max_tokens=20,
temperature=0.5,
)
Proxy:
model_list:
- model_name: gemma-3
litellm_params:
model: bytez/google/gemma-3-4b-it
api_key: os.environ/BYTEZ_API_KEY
max_tokens: 20
temperature: 0.5
추가 파라미터 - repetition_penalty
SDK:
# !uv add litellm
from litellm import completion
import os
## set ENV variables
os.environ["BYTEZ_API_KEY"] = "YOUR_BYTEZ_KEY_HERE"
# bytez llama3 call with additional params
response = completion(
model="bytez/google/gemma-3-4b-it",
messages = [{ "content": "Hello, how are you?","role": "user"}],
repetition_penalty=1.2,
)
Proxy:
model_list:
- model_name: gemma-3
litellm_params:
model: bytez/google/gemma-3-4b-it
api_key: os.environ/BYTEZ_API_KEY
repetition_penalty: 1.2