예측 출력

예측 출력 (Predicted Outputs)

Predicted Outputs 사용하기

• LiteLLM Python SDK
• LiteLLM Proxy Server

이 예시에서는 C# 코드 일부를 리팩터링하고 Username 속성을 Email로 바꾸려고 해요:

import litellm
os.environ["OPENAI_API_KEY"] = "your-api-key"

code = """
/// <summary>
/// Represents a user with a first name, last name, and username.
/// </summary>
public class User
{
    /// <summary>
    /// Gets or sets the user's first name.
    /// </summary>
    public string FirstName { get; set; }

    /// <summary>
    /// Gets or sets the user's last name.
    /// </summary>
    public string LastName { get; set; }

    /// <summary>
    /// Gets or sets the user's username.
    /// </summary>
    public string Username { get; set; }
}
"""

completion = litellm.completion(
    model="gpt-5.6-luna",
    messages=[
        {
            "role": "user",
            "content": "Replace the Username property with an Email property. Respond only with code, and with no markdown formatting.",
        },
        {"role": "user", "content": code},
    ],
    prediction={"type": "content", "content": code},
)

print(completion)

출처: 문서

본문

Proxy Server 사용

1. config.yaml에 모델 정의

model_list:
  - model_name: gpt-5.6-luna # OpenAI gpt-5.6-luna
    litellm_params:
      model: openai/gpt-5.6-luna
      api_key: os.environ/OPENAI_API_KEY 

2. proxy server 실행

litellm --config config.yaml

3. OpenAI Python SDK로 테스트

from openai import OpenAI

client = OpenAI(
    api_key="LITELLM_PROXY_KEY", # sk-<your-litellm-api-key>
    base_url="LITELLM_PROXY_BASE" # http://0.0.0.0:4000
)

completion = client.chat.completions.create(
    model="gpt-5.6-luna",
    messages=[
        {
            "role": "user",
            "content": "Replace the Username property with an Email property. Respond only with code, and with no markdown formatting.",
        },
        {"role": "user", "content": code},
    ],
    prediction={"type": "content", "content": code},
)

print(completion)

더 알아보기 (Learn more)

• Chat Completions
• Reliability - Retries, Fallbacks