Cloudflare Workers AI
Cloudflare Workers AI
Cloudflare Workers AI 텍스트 생성 모델을 LiteLLM의 cloudflare/ 라우트로 호출해요.
출처: 문서
본문
API 키
# env variable
os.environ['CLOUDFLARE_API_KEY'] = "3dnSGlxxxx"
os.environ['CLOUDFLARE_ACCOUNT_ID'] = "03xxxxx"
샘플 사용법 (Sample Usage)
from litellm import completion
import os
os.environ['CLOUDFLARE_API_KEY'] = "3dnSGlxxxx"
os.environ['CLOUDFLARE_ACCOUNT_ID'] = "03xxxxx"
response = completion(
model="cloudflare/@cf/meta/llama-2-7b-chat-int8",
messages=[
{"role": "user", "content": "hello from litellm"}
],
)
print(response)
샘플 사용법 - 스트리밍 (Streaming)
from litellm import completion
import os
os.environ['CLOUDFLARE_API_KEY'] = "3dnSGlxxxx"
os.environ['CLOUDFLARE_ACCOUNT_ID'] = "03xxxxx"
response = completion(
model="cloudflare/@hf/thebloke/codellama-7b-instruct-awq",
messages=[
{"role": "user", "content": "hello from litellm"}
],
stream=True,
)
for chunk in response:
print(chunk)