CopilotKit SDK with LiteLLM
CopilotKit SDK with LiteLLM
LiteLLM Proxy를 통해 CopilotKit SDK를 모든 LLM 제공자와 함께 사용하는 방법을 알려드릴게요.
출처: 문서
본문
참고: CopilotKit SDK와 LiteLLM Proxy의 통합은 LiteLLM v1.81.7-nightly 이상에서 동작해요.
빠른 시작
1. Config에 모델 추가
config.yaml
model_list: - model_name: claude-sonnet-5 litellm_params: model: "anthropic/claude-sonnet-5" api_key: "os.environ/ANTHROPIC_API_KEY"
2. LiteLLM Proxy 시작
litellm --config config.yaml
3. CopilotKit SDK 사용
import OpenAI from "openai";import { CopilotRuntime, OpenAIAdapter, copilotRuntimeNextJSAppRouterEndpoint,} from "@copilotkit/runtime";import { NextRequest } from "next/server";const model = "claude-sonnet-5";const openai = new OpenAI({ apiKey: proces..._KEY || "sk-", baseURL: process.env.OPENAI_BASE_URL || "http://localhost:4000/v1",});const serviceAdapter = new OpenAIAdapter({ openai, model });const runtime = new CopilotRuntime();export const POST = async (req: NextRequest) => { const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({ runtime, serviceAdapter, endpoint: "/api/copilotkit", }); return handleRequest(req);};
4. 테스트
curl -X POST http://localhost:3000/api/copilotkit \ -H "Content-Type: application/json" \ -d '{ "method": "agent/run", "params": { "agentId": "default" }, "runId": "your_run_id", "threadId": "your_thread_id", "runId": ""your_run_id"", "tools": [], "context": [], "forwardedProps": {}, "state": {}, "messages": [ { "id": "166e573e-f7c6-4c0f-8685-04dbefec18be", "content": "Hi", "role": "user" } ] }}'
환경 변수
| 변수 | 값 | 설명 |
| OPENAI_API_KEY | sk- | LiteLLM API 키 |
| OPENAI_BASE_URL | http://localhost:4000/v1 | LiteLLM 프록시 URL |