Inception

Inception

Inception은 Mercury 계열의 diffusion LLM(dLLM)을 제공해요. API는 OpenAI 호환성이에요.

출처: 문서

본문

개요 (Overview)

사용 가능한 모델 (Available Models)

필수 변수

os.environ["INCEPTION_API_KEY"] = ""  # your Inception API key

LiteLLM Python SDK 사용법

비스트리밍 (Non-streaming)

import os
import litellm
from litellm import completion

os.environ["INCEPTION_API_KEY"] = ""  # your Inception API key
messages = [{"content": "Hello, how are you?", "role": "user"}]

# Inception call
response = completion(
    model="inception/mercury-2",
    messages=messages
)
print(response)

스트리밍 (Streaming)

import os
import litellm
from litellm import completion

os.environ["INCEPTION_API_KEY"] = ""  # your Inception API key
messages = [{"content": "Write a short story about AI", "role": "user"}]

# Inception call with streaming
response = completion(
    model="inception/mercury-2",
    messages=messages,
    stream=True
)

for chunk in response:
    print(chunk)

Reasoning Effort와 Reasoning Summary

Mercury는 표준 low, medium, high와 함께 근실시간 응답을 위한 Inception 전용 instant 값을 제공하는 reasoning_effort 컨트롤을 노출해요. reasoning_summary=True로 설정하면 응답에 모델 reasoning의 요약을 받을 수 있어요.

import os
from litellm import completion

os.environ["INCEPTION_API_KEY"] = ""  # your Inception API key

response = completion(
    model="inception/mercury-2",
    messages=[{"role": "user", "content": "If a bat and ball cost $1.10 and the bat is $1 more than the ball, how much is the ball?"}],
    reasoning_effort="high",
    reasoning_summary=True,
)
print(response.choices[0].message.content)
print(response.reasoning_summary)  # {"content": "...", "status": "complete"}

Function Calling

import os
from litellm import completion

os.environ["INCEPTION_API_KEY"] = ""  # your Inception API key

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather in a location",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "The city and state, e.g. San Francisco, CA"
                }
            },
            "required": ["location"]
        }
    }
}]

messages = [{"role": "user", "content": "What's the weather in Boston?"}]

response = completion(
    model="inception/mercury-2",
    messages=messages,
    tools=tools,
    tool_choice="auto"
)
print(response)

Fill-in-the-Middle (FIM)

mercury-edit-2는 Inception의 /v1/fim/completions 엔드포인트를 통해 코드 자동완성을 제공해요. text-completion-inception/ 라우트와 함께 text_completion을 사용하고 프롬프트(접두사)와 선택적 suffix를 전달하세요.

import os
from litellm import text_completion

os.environ["INCEPTION_API_KEY"] = ""  # your Inception API key

response = text_completion(
    model="text-completion-inception/mercury-edit-2",
    prompt="def add(a, b):\n return ",
    suffix="\n",
    max_tokens=64,
)
print(response.choices[0].text)

LiteLLM Proxy Server 사용법

config.yaml:

model_list:
  - model_name: mercury-2
    litellm_params:
      model: inception/mercury-2
      api_key: os.environ/INCEPTION_API_KEY
  - model_name: mercury-edit-2
    litellm_params:
      model: text-completion-inception/mercury-edit-2
      api_key: os.environ/INCEPTION_API_KEY

지원되는 OpenAI 파라미터

• max_tokens
• max_completion_tokens
• temperature
• stop
• tools
• tool_choice
• stream
• stream_options
• response_format

더 알아보기 (Learn more)

• LiteLLM 개요
• LiteLLM Python SDK 사용법
• LiteLLM Proxy Server 설정
• 지원되는 공급업체 전체 목록