/v1/messages/count_tokens

/v1/messages/count_tokens

개요

Anthropic 호환 토큰 카운팅 엔드포인트예요. 모델로 보내기 전에 메시지의 토큰 수를 셀 수 있어요.

빠른 시작

1. LiteLLM Proxy 시작

litellm --config /path/to/config.yaml

# RUNNING on http://0.0.0.0:4000

2. 토큰 세기

• curl
• Python (httpx)

curl -X POST "http://localhost:4000/v1/messages/count_tokens" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [
      {"role": "user", "content": "Hello, how are you?"}
    ]
  }'

import httpx

response = httpx.post(
    "http://localhost:4000/v1/messages/count_tokens",
    headers={
        "Content-Type": "application/json",
        "Authorization": "Bearer sk-<your-litellm-api-key>"
    },
    json={
        "model": "claude-sonnet-5",
        "messages": [
            {"role": "user", "content": "Hello, how are you?"}
        ]
    }
)

print(response.json())
# {"input_tokens": 14}

기대 응답:

{
  "input_tokens": 14
}

LiteLLM Proxy 설정

config.yaml 에 모델을 추가하세요:

model_list:
  - model_name: claude-sonnet-5
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: os.environ/ANTHROPIC_API_KEY

  - model_name: claude-vertex
    litellm_params:
      model: vertex_ai/claude-sonnet-5
      vertex_project: my-project
      vertex_location: us-east5
      vertex_count_tokens_location: us-east5 # Optional: Override location for token counting (count_tokens not available on global location)

  - model_name: claude-bedrock
    litellm_params:
      model: bedrock/us.anthropic.claude-sonnet-5
      aws_region_name: us-west-2

요청 파라미터

Messages 형식

{
  "messages": [
    {"role": "user", "content": "Hello!"},
    {"role": "assistant", "content": "Hi there!"},
    {"role": "user", "content": "How are you?"}
  ]
}

응답 형식

{
  "input_tokens": "<number>"
}

지원 프로바이더

/v1/messages/count_tokens 엔드포인트는 적절한 프로바이더별 토큰 카운팅 API로 자동 라우팅됩니다:

예시

System 메시지로 토큰 세기

curl -X POST "http://localhost:4000/v1/messages/count_tokens" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [
      {"role": "user", "content": "You are a helpful assistant. Please help me write a haiku about programming."}
    ]
  }'

다중 턴 대화 토큰 세기

curl -X POST "http://localhost:4000/v1/messages/count_tokens" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [
      {"role": "user", "content": "What is the capital of France?"},
      {"role": "assistant", "content": "The capital of France is Paris."},
      {"role": "user", "content": "What is its population?"}
    ]
  }'

Vertex AI Claude와 함께 사용

curl -X POST "http://localhost:4000/v1/messages/count_tokens" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "claude-vertex",
    "messages": [
      {"role": "user", "content": "Hello, world!"}
    ]
  }'

Bedrock Claude와 함께 사용

curl -X POST "http://localhost:4000/v1/messages/count_tokens" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "claude-bedrock",
    "messages": [
      {"role": "user", "content": "Hello, world!"}
    ]
  }'

Anthropic Passthrough와의 비교

LiteLLM은 토큰을 세는 두 가지 방법을 제공해요:

Passthrough 예시

네이티브 헤더를 모두 가진 직접 Anthropic API 접근:

curl --request POST \
    --url http://0.0.0.0:4000/anthropic/v1/messages/count_tokens \
    --header "x-api-key: *** \
    --header "anthropic-version: 2023-06-01" \
    --header "anthropic-beta: token-counting-2024-11-01" \
    --header "content-type: application/json" \
    --data '{
        "model": "claude-sonnet-5",
        "messages": [
            {"role": "user", "content": "Hello, world"}
        ]
    }'

출처: 문서

더 알아보기 (Learn more)

• Token Counting
• /v1/messages