추론 모델 채팅 컴플리션

추론 모델 채팅 컴플리션 (Chat Completion with Reasoning)

추론(사고) 모델을 OpenAI 호환 채팅 컴플리션 API로 호출하는 예제입니다. 응답에서 reasoning(사고 과정)과 content(최종 답)를 분리해 출력하는 방법을 보여줍니다.

출처: 문서

본문

vllm serve ... --reasoning-parser로 추론 파서를 지정하면 /v1/chat/completions 응답에 reasoning 필드가 담깁니다. 아래 예제가 그 호출과 응답 파싱을 보여줍니다.

openai_chat_completion_with_reasoning.py

# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
"""An example shows how to generate chat completions from reasoning models
like DeepSeekR1.

To run this example, you need to start the vLLM server
with the reasoning parser:

```bash
vllm serve deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B \
    --reasoning-parser deepseek_r1

This example demonstrates how to generate chat completions from reasoning models using the OpenAI Python client library. """

from openai import OpenAI

Modify OpenAI's API key and API base to use vLLM's API server.

openai_api_key = "EMPTY" openai_api_base = "http://localhost:8000/v1"

def main(): client = OpenAI( api_key=openai_api_key, base_url=openai_api_base, )

models = client.models.list()
model = models.data[0].id

# Round 1
messages = [{"role": "user", "content": "9.11 and 9.8, which is greater?"}]
# ruff: noqa: E501
# For granite, add: `extra_body={"chat_template_kwargs": {"thinking": True}}`
response = client.chat.completions.create(model=model, messages=messages)

reasoning = response.choices[0].message.reasoning
content = response.choices[0].message.content

print("reasoning for Round 1:", reasoning)
print("content for Round 1:", content)

# Round 2
messages.append({"role": "assistant", "content": content})
messages.append(
    {
        "role": "user",
        "content": "How many Rs are there in the word 'strawberry'?",
    }
)
response = client.chat.completions.create(model=model, messages=messages)

reasoning = response.choices[0].message.reasoning
content = response.choices[0].message.content

print("reasoning for Round 2:", reasoning)
print("content for Round 2:", content)

if name == "main": main()

## 더 알아보기 (Learn more)

- [Reasoning Outputs](https://docs.vllm.ai/en/latest/features/reasoning_outputs/) — 추론 출력
- [Structured Reasoning](https://docs.vllm.ai/en/latest/features/structured_outputs/) — 구조화 출력