Sarvam.ai
Sarvam.ai
LiteLLM은 Sarvam ai의 모든 텍스트 모델을 지원해요.
사용법 (Usage)
import os
from litellm import completion
# Set your Sarvam API key
os.environ["SARVAM_API_KEY"] = ""
messages = [{"role": "user", "content": "Hello"}]
response = completion(
model="sarvam/sarvam-m",
messages=messages,
)
print(response)
LiteLLM Proxy Server와 함께 사용하기 (Usage with LiteLLM Proxy Server)
LiteLLM Proxy Server로 Sarvam.ai 모델을 호출하는 방법이에요.
- config.yaml 수정:
model_list:
- model_name: my-model
litellm_params:
model: sarvam/<your-model-name> # add sarvam/ prefix to route as Sarvam provider
api_key: api-key # api key to send your model
- 프록시 시작:
$ litellm --config /path/to/config.yaml
-
LiteLLM Proxy Server에 요청 보내기:
-
OpenAI Python v1.0.0+
-
curl
import openai
client = openai.OpenAI(
api_key="sk-<your-litellm-api-key>", # pass litellm proxy key, if you're using virtual keys
base_url="http://0.0.0.0:4000" # litellm-proxy-base url
)
response = client.chat.completions.create(
model="my-model",
messages=[
{
"role": "user",
"content": "what llm are you"
}
],
)
print(response)
curl --location 'http://0.0.0.0:4000/chat/completions' \
--header "Authorization: Bearer ***" \
--header 'Content-Type: application/json' \
--data '{
"model": "my-model",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}'
출처: 문서