/rerank

/rerank

검색 결과를 재정렬(리랭킹)하는 데 쓰는 엔드포인트예요. LiteLLM은 리랭킹 API에 대해 cohere API 요청/응답 형식을 따르며, Cohere, Together AI, Azure AI 등 여러 제공자를 지원해요.

출처: 문서

본문

tip

LiteLLM은 리랭킹 API에 대해 cohere api 요청/응답 형식을 따르는 것을 참고하세요.

개요

LiteLLM Python SDK 사용법

Quick Start

from litellm import rerank
import os

os.environ["COHERE_API_KEY"] = "sk-.."

query = "What is the capital of the United States?"
documents = [
    "Carson City is the capital city of the American state of Nevada.",
    "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
    "Washington, D.C. is the capital of the United States.",
    "Capital punishment has existed in the United States since before it was a country.",
]

response = rerank(
    model="cohere/rerank-english-v3.0",
    query=query,
    documents=documents,
    top_n=3,
)
print(response)

비동기 사용법

from litellm import arerank
import os, asyncio

os.environ["COHERE_API_KEY"] = "sk-.."

async def test_async_rerank(): 
    query = "What is the capital of the United States?"
    documents = [
        "Carson City is the capital city of the American state of Nevada.",
        "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
        "Washington, D.C. is the capital of the United States.",
        "Capital punishment has existed in the United States since before it was a country.",
    ]

    response = await arerank(
        model="cohere/rerank-english-v3.0",
        query=query,
        documents=documents,
        top_n=3,
    )
    print(response)

asyncio.run(test_async_rerank())

LiteLLM 프록시 사용법

LiteLLM은 Rerank 호출을 위한 cohere api 호환 /rerank 엔드포인트를 제공해요.

설정 이것을 litellm 프록시 config.yaml에 추가해요.

model_list:
  - model_name: Salesforce/Llama-Rank-V1
    litellm_params:
      model: together_ai/Salesforce/Llama-Rank-V1
      api_key: os.environ/TOGETHERAI_API_KEY
  - model_name: rerank-english-v3.0
    litellm_params:
      model: cohere/rerank-english-v3.0
      api_key: os.environ/COHERE_API_KEY

litellm 시작

litellm --config /path/to/config.yaml

# RUNNING on http://0.0.0.0:4000

테스트 요청

curl http://0.0.0.0:4000/rerank \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "rerank-english-v3.0",
    "query": "What is the capital of the United States?",
    "documents": [
        "Carson City is the capital city of the American state of Nevada.",
        "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
        "Washington, D.C. is the capital of the United States.",
        "Capital punishment has existed in the United States since before it was a country."
    ],
    "top_n": 3
  }'

지원 제공자

⚡️모든 지원 모델과 제공자는 models.litellm.ai에서 확인하세요

더 알아보기 (Learn more)

• /responses