/assistants

/assistants

OpenAI가 Assistants API를 폐기했어요. 2026년 8월 26일에 종료됩니다. 대신 Responses API로 마이그레이션하는 것을 고려하세요. 자세한 내용은 OpenAI의 마이그레이션 가이드를 참고하세요.

Threads, Messages, Assistants를 다룹니다.

LiteLLM은 현재 다음을 지원해요:

• Assistants 생성 (Create Assistants)
• Assistants 삭제 (Delete Assistants)
• Assistants 가져오기 (Get Assistants)
• Thread 생성 (Create Thread)
• Thread 가져오기 (Get Thread)
• Messages 추가 (Add Messages)
• Messages 가져오기 (Get Messages)
• Thread 실행 (Run Thread)

지원 프로바이더:

• OpenAI
• Azure OpenAI
• OpenAI 호환 API

빠른 시작

기존 Assistant를 호출해 보세요.

• Assistant 가져오기
• 사용자가 대화를 시작하면 Thread 만들기
• 사용자가 질문할 때 Thread에 Messages 추가
• 모델과 도구를 호출해 응답을 생성하려면 Thread에서 Assistant 실행

SDK + PROXY

• SDK
• PROXY

Assistant 만들기

import litellm
import os

# setup env
os.environ["OPENAI_API_KEY"] = "sk-.."

assistant = litellm.create_assistants(
    custom_llm_provider="openai",
    model="gpt-5.6-terra",
    instructions="You are a personal math tutor. When asked a question, write and run Python code to answer the question.",
    name="Math Tutor",
    tools=[{"type": "code_interpreter"}],
)

### ASYNC USAGE ###
# assistant = await litellm.acreate_assistants(
#             custom_llm_provider="openai",
#             model="gpt-5.6-terra",
#             instructions="You are a personal math tutor. When asked a question, write and run Python code to answer the question.",
#             name="Math Tutor",
#             tools=[{"type": "code_interpreter"}],
# )

Assistant 가져오기

from litellm import get_assistants, aget_assistants
import os

# setup env
os.environ["OPENAI_API_KEY"] = "sk-.."

assistants = get_assistants(custom_llm_provider="openai")

### ASYNC USAGE ###
# assistants = await aget_assistants(custom_llm_provider="openai")

Thread 만들기

from litellm import create_thread, acreate_thread
import os

os.environ["OPENAI_API_KEY"] = "sk-.."

new_thread = create_thread(
    custom_llm_provider="openai",
    messages=[
        {"role": "user", "content": "Hey, how's it going?"}
    ],  # type: ignore
)

### ASYNC USAGE ###
# new_thread = await acreate_thread(custom_llm_provider="openai",messages=[{"role": "user", "content": "Hey, how's it going?"}])

Thread에 Messages 추가

from litellm import create_thread, get_thread, aget_thread, add_message, a_add_message
import os

os.environ["OPENAI_API_KEY"] = "sk-.."

## CREATE A THREAD
_new_thread = create_thread(
    custom_llm_provider="openai",
    messages=[
        {"role": "user", "content": "Hey, how's it going?"}
    ],  # type: ignore
)

## OR retrieve existing thread
received_thread = get_thread(
    custom_llm_provider="openai",
    thread_id=_new_thread.id,
)

### ASYNC USAGE ###
# received_thread = await aget_thread(custom_llm_provider="openai", thread_id=_new_thread.id)

## ADD A MESSAGE TO A THREAD
message = add_message(
    custom_llm_provider="openai",
    thread_id=_new_thread.id,
    role="user",
    content="Hey, how's it going?",
)

### ASYNC USAGE ###
# message = await a_add_message(custom_llm_provider="openai", thread_id=_new_thread.id, role="user", content="Hey, how's it going?")

Thread에서 Assistant 실행

from litellm import run_thread
import os

os.environ["OPENAI_API_KEY"] = "sk-.."

run = run_thread(
    custom_llm_provider="openai",
    thread_id=_new_thread.id,
    assistant_id=assistant.id,
)

프록시 설정 (Azure)

assistant_settings:
  custom_llm_provider: azure
  litellm_params: 
    api_key: os.environ/AZURE_API_KEY
    api_base: os.environ/AZURE_API_BASE
    api_version: os.environ/AZURE_API_VERSION

$ litellm --config /path/to/config.yaml

# RUNNING on http://0.0.0.0:4000

프록시로 Assistant 생성

curl "http://localhost:4000/v1/assistants" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "instructions": "You are a personal math tutor. When asked a question, write and run Python code to answer the question.",
    "name": "Math Tutor",
    "tools": [{"type": "code_interpreter"}],
    "model": "gpt-5.6-terra"
  }'

프록시로 Assistant 가져오기

curl "http://0.0.0.0:4000/v1/assistants?order=desc&limit=20" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***"

프록시로 Thread 만들기

curl http://0.0.0.0:4000/v1/threads \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d ''

프록시로 Thread 가져오기

curl http://0.0.0.0:4000/v1/threads/{thread_id} \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***"

프록시로 Thread에 Messages 추가

curl http://0.0.0.0:4000/v1/threads/{thread_id}/messages \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "role": "user",
    "content": "How does AI work? Explain it in simple terms."
  }'

프록시로 Thread에서 Assistant 실행

curl http://0.0.0.0:4000/v1/threads/thread_abc123/runs \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "assistant_id": "asst_abc123"
  }'

출처: 문서

본문

스트리밍

• SDK
• PROXY

from litellm import run_thread_stream 
import os

os.environ["OPENAI_API_KEY"] = "sk-.."

message = {"role": "user", "content": "Hey, how's it going?"}  

data = {"custom_llm_provider": "openai", "thread_id": _new_thread.id, "assistant_id": assistant_id, **message}

run = run_thread_stream(**data)
with run as run:
    assert isinstance(run, AssistantEventHandler)
    for chunk in run: 
      print(f"chunk: {chunk}")
    run.until_done()

curl -X POST 'http://0.0.0.0:4000/threads/{thread_id}/runs' \
-H "Authorization: Bearer ***" \
-H 'Content-Type: application/json' \
-d '{
      "assistant_id": "asst_6xVZQFFy1Kw87NbnYeNebxTf",
      "stream": true
}'

👉 Proxy API Reference

Azure OpenAI

config

assistant_settings:
  custom_llm_provider: azure
  litellm_params: 
    api_key: os.environ/AZURE_API_KEY
    api_base: os.environ/AZURE_API_BASE

curl

curl -X POST "http://localhost:4000/v1/assistants" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "instructions": "You are a personal math tutor. When asked a question, write and run Python code to answer the question.",
    "name": "Math Tutor",
    "tools": [{"type": "code_interpreter"}],
    "model": "<my-azure-deployment-name>"
  }'

OpenAI 호환 API

OpenAI 호환 Assistants API(예: Astra Assistants API)를 호출하려면 모델 이름에 openai/를 추가하세요:

config

assistant_settings:
  custom_llm_provider: openai
  litellm_params: 
    api_key: os.environ/ASTRA_API_KEY
    api_base: os.environ/ASTRA_API_BASE

curl

curl -X POST "http://localhost:4000/v1/assistants" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "instructions": "You are a personal math tutor. When asked a question, write and run Python code to answer the question.",
    "name": "Math Tutor",
    "tools": [{"type": "code_interpreter"}],
    "model": "openai/<my-astra-model-name>"
  }'

더 알아보기 (Learn more)

• Responses API
• Chat Completions