Cohere

Cohere

Cohere의 chat, 임베딩, rerank, OCR(Parse) 모델을 LiteLLM으로 호출해요.

출처: 문서

본문

API 키

import os

os.environ["COHERE_API_KEY"] = ""

사용법 (Usage)

LiteLLM Python SDK

Cohere v2 API (기본):

from litellm import completion

## set ENV variables
os.environ["COHERE_API_KEY"] = "cohere key"

# cohere v2 call
response = completion(
    model="cohere_chat/command-a-03-2025",
    messages = [{ "content": "Hello, how are you?","role": "user"}]
)

Cohere v1 API:

Cohere v1/chat API를 사용하려면 모델 이름에 cohere_chat/v1/ 접두사를 붙이세요.

from litellm import completion

## set ENV variables
os.environ["COHERE_API_KEY"] = "cohere key"

# cohere v1 call
response = completion(
    model="cohere_chat/v1/command-a-03-2025",
    messages = [{ "content": "Hello, how are you?","role": "user"}]
)

스트리밍

Cohere v2 스트리밍:

from litellm import completion

## set ENV variables
os.environ["COHERE_API_KEY"] = "cohere key"

# cohere v2 streaming
response = completion(
    model="cohere_chat/command-a-03-2025",
    messages = [{ "content": "Hello, how are you?","role": "user"}],
    stream=True
)

for chunk in response:
    print(chunk)

Cohere v1 스트리밍:

from litellm import completion

## set ENV variables
os.environ["COHERE_API_KEY"] = "cohere key"

# cohere v1 streaming
response = completion(
    model="cohere_chat/v1/command-a-03-2025",
    messages = [{ "content": "Hello, how are you?","role": "user"}],
    stream=True
)

for chunk in response:
    print(chunk)

LiteLLM Proxy 사용법

1. 환경에 키 저장

export COHERE_API_KEY="your-api-key"

2. Proxy 시작

config.yaml에서 사용할 cohere 모델을 정의해요.

Cohere v1 모델용:

model_list:
  - model_name: command-a-03-2025
    litellm_params:
      model: cohere_chat/v1/command-a-03-2025
      api_key: "os.environ/COHERE_API_KEY"

Cohere v2 모델용:

model_list:
  - model_name: command-a-03-2025-v2
    litellm_params:
      model: cohere_chat/command-a-03-2025
      api_key: "os.environ/COHERE_API_KEY"

litellm --config /path/to/config.yaml

3. 테스트

Cohere v1 - curl:

curl --location 'http://0.0.0.0:4000/chat/completions' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <your-...key>' \
  --data ' {
    "model": "command-a-03-2025",
    "messages": [
      {
        "role": "user",
        "content": "what llm are you"
      }
    ]
  }'

Cohere v2 - curl:

curl --location 'http://0.0.0.0:4000/chat/completions' \
  --header 'Content-Type: application/json' \
  --header 'Authorization: Bearer <your-...key>' \
  --data ' {
    "model": "command-a-03-2025-v2",
    "messages": [
      {
        "role": "user",
        "content": "what llm are you"
      }
    ]
  }'

Cohere v1 - OpenAI SDK:

import openai

client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

# request sent to cohere v1 model
response = client.chat.completions.create(
    model="command-a-03-2025",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ]
)
print(response)

Cohere v2 - OpenAI SDK:

import openai

client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

# request sent to cohere v2 model
response = client.chat.completions.create(
    model="command-a-03-2025-v2",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ]
)
print(response)

지원 모델 (Supported Models)

모델 이름	함수 호출
command-a-03-2025	litellm.completion('command-a-03-2025', messages)
command-r-plus-08-2024	litellm.completion('command-r-plus-08-2024', messages)
command-r-08-2024	litellm.completion('command-r-08-2024', messages)
command-r-plus	litellm.completion('command-r-plus', messages)
command-r	litellm.completion('command-r', messages)
command-light	litellm.completion('command-light', messages)
command-nightly	litellm.completion('command-nightly', messages)

임베딩 (Embedding)

from litellm import embedding

os.environ["COHERE_API_KEY"] = "cohere key"

# cohere call
response = embedding(
    model="embed-english-v3.0",
    input=["good morning from litellm", "this is another item"],
)

v3 모델 Input Type 설정

v3 모델에는 필수 파라미터인 input_type이 있어요. LiteLLM은 기본적으로 search_document를 사용해요. 다음 네 가지 값 중 하나가 될 수 있어요:

• input_type="search_document": (기본값) 벡터 데이터베이스에 저장할 텍스트(문서)에 사용
• input_type="search_query": 벡터 DB에서 가장 관련 문서를 찾는 검색 쿼리에 사용
• input_type="classification": 임베딩을 분류 시스템 입력으로 사용할 때
• input_type="clustering": 텍스트 클러스터링에 임베딩을 사용할 때

from litellm import embedding

os.environ["COHERE_API_KEY"] = "cohere key"

# cohere call
response = embedding(
    model="embed-english-v3.0",
    input=["good morning from litellm", "this is another item"],
    input_type="search_document"
)

지원 임베딩 모델

모델 이름	함수 호출
embed-english-v3.0	embedding(model="embed-english-v3.0", input=["good morning from litellm", "this is another item"])
embed-english-light-v3.0	embedding(model="embed-english-light-v3.0", input=[...])
embed-multilingual-v3.0	embedding(model="embed-multilingual-v3.0", input=[...])
embed-multilingual-light-v3.0	embedding(model="embed-multilingual-light-v3.0", input=[...])
embed-english-v2.0	embedding(model="embed-english-v2.0", input=[...])
embed-english-light-v2.0	embedding(model="embed-english-light-v2.0", input=[...])
embed-multilingual-v2.0	embedding(model="embed-multilingual-v2.0", input=[...])

Rerank

LiteLLM은 Cohere rerank의 v1과 v2 클라이언트를 모두 지원해요. 기본적으로 rerank 엔드포인트는 v2 클라이언트를 사용하며, v1/rerank를 명시적으로 호출하면 v1 클라이언트를 지정할 수 있어요.

LiteLLM SDK 사용법:

from litellm import rerank
import os

os.environ["COHERE_API_KEY"] = "sk-.."
query = "What is the capital of the United States?"
documents = [
    "Carson City is the capital city of the American state of Nevada.",
    "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
    "Washington, D.C. is the capital of the United States.",
    "Capital punishment has existed in the United States since before it was a country.",
]

response = rerank(
    model="cohere/rerank-english-v3.0",
    query=query,
    documents=documents,
    top_n=3,
)
print(response)

LiteLLM Proxy 사용법:

config.yaml에 추가:

model_list:
  - model_name: Salesforce/Llama-Rank-V1
    litellm_params:
      model: together_ai/Salesforce/Llama-Rank-V1
      api_key: os.environ/TOGETHERAI_API_KEY
  - model_name: rerank-english-v3.0
    litellm_params:
      model: cohere/rerank-english-v3.0
      api_key: os.environ/COHERE_API_KEY

Proxy 시작 후 테스트:

litellm --config /path/to/config.yaml
# RUNNING on http://0.0.0.0:4000

curl http://0.0.0.0:4000/rerank \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rerank-english-v3.0",
    "query": "What is the capital of the United States?",
    "documents": [
      "Carson City is the capital city of the American state of Nevada.",
      "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
      "Washington, D.C. is the capital of the United States.",
      "Capital punishment has existed in the United States since before it was a country."
    ],
    "top_n": 3
  }'

Parse (OCR)

Cohere Parse는 문서 이미지를 markdown으로 변환해요. LiteLLM은 이를 /ocr 엔드포인트로 서빙하므로 요청과 응답이 다른 OCR 공급자와 같은 형태를 사용하며, 각 호출이 청구 페이지당 비용으로 추적돼요.

Parse는 image_url 문서만 받아요: 이미지 URL 또는 base64 data:image/... URI. PDF와 document_url 입력은 Cohere로 아무것도 보내기 전에 400으로 거부돼요.

LiteLLM SDK 사용법:

import os
from litellm import ocr

os.environ["COHERE_API_KEY"] = ""

response = ocr(
    model="cohere/parse-v5.0",
    document={
        "type": "image_url",
        "image_url": "https://raw.githubusercontent.com/mistralai/cookbook/refs/heads/main/mistral/ocr/receipt.png",
    },
)

for page in response.pages:
    print(page.markdown)
print(response.usage_info.pages_processed)

LiteLLM Proxy 사용법:

config.yaml에 추가:

model_list:
  - model_name: cohere-parse
    litellm_params:
      model: cohere/parse-v5.0
      api_key: os.environ/COHERE_API_KEY

Proxy 시작 후 테스트:

litellm --config /path/to/config.yaml
# RUNNING on http://0.0.0.0:4000

curl http://0.0.0.0:4000/v1/ocr \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "cohere-parse",
    "document": {
      "type": "image_url",
      "image_url": "https://raw.githubusercontent.com/mistralai/cookbook/refs/heads/main/mistral/ocr/receipt.png"
    }
  }'

더 알아보기 (Learn more)

• LiteLLM 개요
• LiteLLM Python SDK 사용법
• LiteLLM Proxy Server 설정
• 지원되는 공급업체 전체 목록