Cohere SDK

Cohere SDK

Cohere의 패스스루 엔드포인트를 소개할게요. Cohere의 프로바이더 고유 엔드포인트를 네이티브 형식 그대로(변환 없이) 호출할 수 있는 기능이에요. /v1/chat, /v2/chat 같은 Cohere 원본 엔드포인트를 그대로 사용할 수 있어요.

출처: 문서

본문

연결 대상 호스트는 https://api.cohere.com이고, 프록시를 통한 주소는 이렇게 구성돼요.

LITELLM_PROXY_BASE_URL/cohere

사용 예시 (Example Usage)

/v1/chat 엔드포인트를 Cohere 네이티브 형식으로 호출하는 예시예요.

curl --request POST \
  --url http://0.0.0.0:4000/cohere/v1/chat \
  --header 'accept: application/json' \
  --header 'content-type: application/json' \
  --header "Authorization: bearer ***" \
  --data '{
    "chat_history": [
      {"role": "USER", "message": "Who discovered gravity?"},
      {"role": "CHATBOT", "message": "The man who is widely credited with discovering gravity is Sir Isaac Newton"}
    ],
    "message": "What year was he born?",
    "connectors": [{"id": "web-search"}]
  }'

빠른 시작 (Quick Start)

Cohere의 /rerank 엔드포인트도 그대로 호출할 수 있어요. 먼저 API 키를 환경 변수로 설정해요.

export COHERE_API_KEY=""

그다음 LiteLLM 프록시를 실행해요.

litellm
# RUNNING on http://0.0.0.0:4000

rerank 호출 예시예요.

curl --request POST \
  --url http://0.0.0.0:4000/cohere/v1/rerank \
  --header 'accept: application/json' \
  --header 'content-type: application/json' \
  --header "Authorization: bearer ***" \
  --data '{
    "model": "rerank-english-v3.0",
    "query": "What is the capital of the United States?",
    "top_n": 3,
    "documents": ["Carson City is the capital city of the American state of Nevada.",
                  "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
                  "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.",
                  "Capitalization or capitalisation in English grammar is the use of a capital letter at the start of a word. English usage varies from capitalization in other languages.",
                  "Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."]
  }'

예시 (Examples)

핵심 아이디어는 단순해요. Cohere SDK의 베이스 URL을 http://0.0.0.0:4000/cohere로 바꾸면 돼요. 원래 https://api.cohere.com을 호출하던 것을 http://0.0.0.0:4000/cohere로 바꾸고, 인증 헤더를 bearer $CO_API_KEY 대신 LiteLLM 키(bearer anything 또는 가상 키 bearer LITELLM_VIRTUAL_KEY)로 바꾸면 돼요. 이렇게 하면 비용 추적·로깅·접근 제어가 모두 LiteLLM을 통해 처리돼요.

예시 1: rerank 엔드포인트

LiteLLM 프록시 호출

curl --request POST \
  --url http://0.0.0.0:4000/cohere/v1/rerank \
  --header 'accept: application/json' \
  --header 'content-type: application/json' \
  --header "Authorization: bearer ***" \
  --data '{
    "model": "rerank-english-v3.0",
    "query": "What is the capital of the United States?",
    "top_n": 3,
    "documents": ["Carson City is the capital city of the American state of Nevada.",
                  "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
                  "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.",
                  "Capitalization or capitalisation in English grammar is the use of a capital letter at the start of a word. English usage varies from capitalization in other languages.",
                  "Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."]
  }'

Cohere 직접 API 호출 (변환 전)

curl --request POST \
  --url https://api.cohere.com/v1/rerank \
  --header 'accept: application/json' \
  --header 'content-type: application/json' \
  --header "Authorization: bearer ***" \
  --data '{
    "model": "rerank-english-v3.0",
    "query": "What is the capital of the United States?",
    "top_n": 3,
    "documents": ["Carson City is the capital city of the American state of Nevada.",
                  "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
                  "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.",
                  "Capitalization or capitalisation in English grammar is the use of a capital letter at the start of a word. English usage varies from capitalization in other languages.",
                  "Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."]
  }'

예시 2: chat API

LiteLLM 프록시 호출

curl --request POST \
  --url http://0.0.0.0:4000/cohere/v1/chat \
  --header 'accept: application/json' \
  --header 'content-type: application/json' \
  --header "Authorization: bearer ***" \
  --data '{
    "chat_history": [
      {"role": "USER", "message": "Who discovered gravity?"},
      {"role": "CHATBOT", "message": "The man who is widely credited with discovering gravity is Sir Isaac Newton"}
    ],
    "message": "What year was he born?",
    "connectors": [{"id": "web-search"}]
  }'

Cohere 직접 API 호출 (변환 전)

curl --request POST \
  --url https://api.cohere.com/v1/chat \
  --header 'accept: application/json' \
  --header 'content-type: application/json' \
  --header "Authorization: bearer ***" \
  --data '{
    "chat_history": [
      {"role": "USER", "message": "Who discovered gravity?"},
      {"role": "CHATBOT", "message": "The man who is widely credited with discovering gravity is Sir Isaac Newton"}
    ],
    "message": "What year was he born?",
    "connectors": [{"id": "web-search"}]
  }'

예시 3: embedding

Cohere 직접 API 호출 (변환 전)

curl --request POST \
  --url https://api.cohere.com/v1/embed \
  --header 'accept: application/json' \
  --header 'content-type: application/json' \
  --header "Authorization: bearer ***" \
  --data '{
    "model": "embed-english-v3.0",
    "texts": ["hello", "goodbye"],
    "input_type": "classification"
  }'

고급: 가상 키(Virtual Keys)와 함께 사용하기

가상 키는 LiteLLM 프록시에 데이터베이스가 설정된 경우에 사용할 수 있어요. 가상 키 설정 문서를 참고해 주세요. 가상 키를 쓰면 각 사용자/팀별로 별도 키를 발급해 비용을 추적할 수 있어요.

사용법 (Usage)

먼저 환경 변수를 설정해요.

export DATABASE_URL=""
export LITELLM_MASTER_KEY=""
export COHERE_API_KEY=""

프록시를 실행해요.

litellm
# RUNNING on http://0.0.0.0:4000

가상 키를 생성해요.

curl -X POST 'http://0.0.0.0:4000/key/generate' \
-H "Authorization: Bearer ***" \
-H 'Content-Type: application/json' \
-d '{}'

응답에서 키를 받아요.

{
    ...
    "key": "sk-<virtual-key>"
}

이제 rerank 엔드포인트를 가상 키로 호출해요.

curl --request POST \
  --url http://0.0.0.0:4000/cohere/v1/rerank \
  --header 'accept: application/json' \
  --header 'content-type: application/json' \
  --header "Authorization: bearer ***" \
  --data '{
    "model": "rerank-english-v3.0",
    "query": "What is the capital of the United States?",
    "top_n": 3,
    "documents": ["Carson City is the capital city of the American state of Nevada.",
                  "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
                  "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.",
                  "Capitalization or capitalisation in English grammar is the use of a capital letter at the start of a word. English usage varies from capitalization in other languages.",
                  "Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."]
  }'

더 알아보기 (Learn more)

• Cohere Chat API Reference
• Cohere Rerank API Reference
• 가상 키 설정