PDF 입력 사용하기

PDF 입력 사용하기 (Using PDF Input)

/chat/completions 엔드포인트에 PDF(및 기타 문서 타입)를 보내고 받는 방법을 다뤄요.

다음에서 동작해요:

  • Vertex AI 모델 (Gemini + Anthropic)
  • Bedrock 모델
  • Anthropic API 모델
  • OpenAI API 모델
  • Mistral (이미 업로드된 파일의 file ID 사용만 — OpenAI file_id 입력과 유사)

빠른 시작

url

  • SDK
  • PROXY
from litellm import completion
from litellm.utils import supports_pdf_input

# set aws credentials
os.environ["AWS_ACCESS_KEY_ID"] = ""
os.environ["AWS_SECRET_ACCESS_KEY"] = ""
os.environ["AWS_REGION_NAME"] = ""

# pdf url
file_url = "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"

# model
model = "bedrock/us.anthropic.claude-sonnet-5"

file_content = [
    {"type": "text", "text": "What's this file about?"},
    {
        "type": "file",
        "file": {
            "file_id": file_url,
        }
    },
]

if not supports_pdf_input(model, None):
    print("Model does not support image input")

response = completion(
    model=model,
    messages=[{"role": "user", "content": file_content}],
)
assert response is not None
  1. config.yaml 설정
model_list:
  - model_name: bedrock-model
    litellm_params:
      model: bedrock/us.anthropic.claude-sonnet-5
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: os.environ/AWS_REGION_NAME
  1. 프록시 시작
litellm --config /path/to/config.yaml
  1. 테스트!
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer ***" \
-d '{
    "model": "bedrock-model",
    "messages": [
        {"role": "user", "content": [
            {"type": "text", "text": "What's this file about?"},
            {
                "type": "file",
                "file": {
                    "file_id": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
                }
            }
        ]},
    ]
}'

base64

  • SDK
  • PROXY
from litellm import completion
from litellm.utils import supports_pdf_input

# set aws credentials
os.environ["AWS_ACCESS_KEY_ID"] = ""
os.environ["AWS_SECRET_ACCESS_KEY"] = ""
os.environ["AWS_REGION_NAME"] = ""

# pdf url
image_url = "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"
response = requests.get(url)
file_data = response.content

encoded_file = base64.b64encode(file_data).decode("utf-8")
base64_url = f"data:application/pdf;base64,{encoded_file}"

# model
model = "bedrock/us.anthropic.claude-sonnet-5"

file_content = [
    {"type": "text", "text": "What's this file about?"},
    {
        "type": "file",
        "file": {
            "file_data": base64_url,
        }
    },
]

if not supports_pdf_input(model, None):
    print("Model does not support image input")

response = completion(
    model=model,
    messages=[{"role": "user", "content": file_content}],
)
assert response is not None
  1. config.yaml 설정
model_list:
  - model_name: bedrock-model
    litellm_params:
      model: bedrock/us.anthropic.claude-sonnet-5
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: os.environ/AWS_REGION_NAME
  1. 프록시 시작
litellm --config /path/to/config.yaml
  1. 테스트!
curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer ***" \
-d '{
    "model": "bedrock-model",
    "messages": [
        {"role": "user", "content": [
            {"type": "text", "text": "What's this file about?"},
            {
                "type": "file",
                "file": {
                    "file_data": "data:application/pdf;base64...",
                }
            }
        ]},
    ]
}'

출처: 문서

본문

형식 지정

문서의 형식을 지정하려면 format 파라미터를 사용할 수 있어요.

  • SDK
  • PROXY
from litellm import completion
from litellm.utils import supports_pdf_input

# set aws credentials
os.environ["AWS_ACCESS_KEY_ID"] = ""
os.environ["AWS_SECRET_ACCESS_KEY"] = ""
os.environ["AWS_REGION_NAME"] = ""

# pdf url
file_url = "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf"

# model
model = "bedrock/us.anthropic.claude-sonnet-5"

file_content = [
    {"type": "text", "text": "What's this file about?"},
    {
        "type": "file",
        "file": {
            "file_id": file_url,
            "format": "application/pdf",
        }
    },
]

if not supports_pdf_input(model, None):
    print("Model does not support image input")

response = completion(
    model=model,
    messages=[{"role": "user", "content": file_content}],
)
assert response is not None

Proxy 사용법:

curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer ***" \
-d '{
    "model": "bedrock-model",
    "messages": [
        {"role": "user", "content": [
            {"type": "text", "text": "What's this file about?"},
            {
                "type": "file",
                "file": {
                    "file_id": "https://www.w3.org/WAI/ER/tests/xhtml/testfiles/resources/pdf/dummy.pdf",
                    "format": "application/pdf",
                }
            }
        ]},
    ]
}'

Mistral 예시

문서 이해에 Mistral 모델을 사용하는 샘플 페이로드예요.

  • SDK
  • PROXY
from litellm import completion

# pdf file_id received from files endpoint
file_id = "fa778e5e-46ec-4562-8418-36623fe25a71"

# model
model = "mistral/mistral-large-latest"

file_content = [
    {"type": "text", "text": "What's this file about?"},
    {
        "type": "file",
        "file": {
            "file_id": file_id,
        }
    },
]

response = completion(
    model=model,
    messages=[{"role": "user", "content": file_content}],
)
assert response is not None

curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer ***" \
-d '{
    "model": "mistral/mistral-large-latest",
    "messages": [
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What is the content of the file?"
                },
                {
                    "type": "file",
                    "file": {
                        "file_id": "fa778e5e-46ec-4562-8418-36623fe25a71"
                    }
                }
            ]
        }
    ]
}'

모델이 pdf 입력을 지원하는지 확인

  • SDK
  • PROXY

litellm.supports_pdf_input(model="bedrock/us.anthropic.claude-sonnet-5") -> 모델이 pdf 입력을 받을 수 있으면 True 반환

assert litellm.supports_pdf_input(model="bedrock/us.anthropic.claude-sonnet-5") == True
  1. config.yaml에 bedrock 모델 정의
model_list:
  - model_name: bedrock-model # model group name
    litellm_params:
      model: bedrock/us.anthropic.claude-sonnet-5
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: os.environ/AWS_REGION_NAME
    model_info: # OPTIONAL - set manually
      supports_pdf_input: True
  1. proxy server 실행
litellm --config config.yaml
  1. /model_group/info 호출로 모델이 pdf 입력을 지원하는지 확인
curl -X 'GET' \
  'http://localhost:4000/model_group/info' \
  -H 'accept: application/json' \
  -H "x-api-key: ***"

기대 응답:

{
  "data": [
    {
      "model_group": "bedrock-model",
      "providers": ["bedrock"],
      "max_input_tokens": 128000,
      "max_output_tokens": 16384,
      "mode": "chat",
      ...,
      "supports_pdf_input": true   # 👈 supports_pdf_input is true
    }
  ]
}

더 알아보기 (Learn more)