NLP Cloud
NLP Cloud
LiteLLM은 NLP Cloud의 모든 LLM을 지원해요.
API 키 (API Keys)
import os
os.environ["NLP_CLOUD_API_KEY"] = "your-api-key"
샘플 사용법 (Sample Usage)
import os
from litellm import completion
# set env
os.environ["NLP_CLOUD_API_KEY"] = "your-api-key"
messages = [{"role": "user", "content": "Hey! how's it going?"}]
response = completion(model="dolphin", messages=messages)
print(response)
스트리밍 (streaming)
completion 호출 시 stream=True만 설정하면 돼요.
import os
from litellm import completion
# set env
os.environ["NLP_CLOUD_API_KEY"] = "your-api-key"
messages = [{"role": "user", "content": "Hey! how's it going?"}]
response = completion(model="dolphin", messages=messages, stream=True)
for chunk in response:
print(chunk["choices"][0]["delta"]["content"]) # same as openai format
dolphin이 아닌 모델 (non-dolphin models)
기본적으로 LiteLLM은 dolphin과 chatdolphin을 nlp cloud로 매핑해요.
다른 모델(예: GPT-J, Llama-2 등)을 nlp cloud로 호출하려면 그 모델을 사용자 지정 llm provider로 설정하면 돼요.
import os
from litellm import completion
# set env - [OPTIONAL] replace with your nlp cloud key
os.environ["NLP_CLOUD_API_KEY"] = "your-api-key"
messages = [{"role": "user", "content": "Hey! how's it going?"}]
# e.g. to call Llama2 on NLP Cloud
response = completion(model="nlp_cloud/finetuned-llama-2-70b", messages=messages, stream=True)
for chunk in response:
print(chunk["choices"][0]["delta"]["content"]) # same as openai format
출처: 문서