Petals
Petals
Petals: https://github.com/bigscience-workshop/petals
사전 준비 (Pre-Requisites)
petals가 설치되어 있는지 확인해 주세요.
uv add git+https://github.com/bigscience-workshop/petals
사용법 (Usage)
모든 Petals LLM에는 petals/ 접두어를 붙여야 해요. 이렇게 하면 custom_llm_provider가 petals로 설정돼요.
from litellm import completion
response = completion(
model="petals/petals-team/StableBeluga2",
messages=[{ "content": "Hello, how are you?","role": "user"}]
)
print(response)
스트리밍과 함께 사용하기 (Usage with Streaming)
response = completion(
model="petals/petals-team/StableBeluga2",
messages=[{ "content": "Hello, how are you?","role": "user"}],
stream=True
)
print(response)
for chunk in response:
print(chunk)
모델 정보 (Model Details)
| 모델명 | 함수 호출 |
|---|---|
petals-team/StableBeluga |
completion('petals/petals-team/StableBeluga2', messages) |
huggyllama/llama-65b |
completion('petals/huggyllama/llama-65b', messages) |
출처: 문서