Fireworks AI

Fireworks AI

Fireworks AI의 모든 모델을 지원해요. completion 요청 시 fireworks_ai/ 접두사로 설정하기만 하면 돼요.

출처: 문서

본문

개요 (Overview)

속성	설명
설명	프로덕션 준비된 복합 AI 시스템을 구축하는 가장 빠르고 효율적인 추론 엔진
LiteLLM 라우트	fireworks_ai/
공급자 문서	Fireworks AI
지원 OpenAI 엔드포인트	/chat/completions, /responses, /embeddings, /completions, /audio/transcriptions, /rerank

이 가이드는 LiteLLM을 Fireworks AI와 통합하는 방법을 설명해요. 세 가지 주요 방식이 있어요:

• Fireworks AI serverless 모델 사용 – Fireworks 관리 모델에 쉽게 연결
• 자체 Fireworks 계정의 모델 연결 – Fireworks 계정에 호스팅된 모델 접근
• 직접 라우트 배포로 연결 – 특정 Fireworks 인스턴스에 더 유연하고 커스터마이즈 가능한 연결

API 키

# env variable
os.environ['FIREWORKS_AI_API_KEY']

샘플 사용법 - Serverless 모델

from litellm import completion
import os

os.environ['FIREWORKS_AI_API_KEY'] = ""

response = completion(
    model="fireworks_ai/glm-5p2",
    messages=[
        {"role": "user", "content": "hello from litellm"}
    ],
)
print(response)

glm-5p2 같은 bare serverless slug는 accounts/fireworks/models/glm-5p2로 자동 확장되므로 짧은 slug나 전체 resource id를 전달할 수 있어요.

샘플 사용법 - Serverless 모델 스트리밍

from litellm import completion
import os

os.environ['FIREWORKS_AI_API_KEY'] = ""

response = completion(
    model="fireworks_ai/glm-5p2",
    messages=[
        {"role": "user", "content": "hello from litellm"}
    ],
    stream=True
)

for chunk in response:
    print(chunk)

샘플 사용법 - 자체 Fireworks 계정 모델

from litellm import completion
import os

os.environ['FIREWORKS_AI_API_KEY'] = ""

response = completion(
    model="fireworks_ai/accounts/fireworks/models/YOUR_MODEL_ID",
    messages=[
        {"role": "user", "content": "hello from litellm"}
    ],
)
print(response)

샘플 사용법 - 직접 라우트 배포

from litellm import completion
import os

os.environ['FIREWORKS_AI_API_KEY'] = "YOUR_DIRECT_API_KEY"

response = completion(
    model="fireworks_ai/accounts/fireworks/models/qwen2p5-coder-7b#accounts/gitlab/deployments/2fb7764c",
    messages=[
        {"role": "user", "content": "hello from litellm"}
    ],
    api_base="https://gitlab-2fb7764c.direct.fireworks.ai/v1"
)
print(response)

참고: 위는 chat 인터페이스용이에요. text completion 인터페이스를 사용하려면 model="text-completion-openai/accounts/fireworks/models/qwen2p5-coder-7b#accounts/gitlab/deployments/2fb7764c"를 사용하세요.

샘플 사용법 - Routers

Fireworks 라우터는 accounts/fireworks/models/<model-id>가 아니라 accounts/fireworks/routers/<router-id>에서 서빙되므로, bare slug만으로는 LiteLLM이 어느 것을 의미하는지 알 수 없어요. 라우터를 대상으로 하려면 slug 앞에 routers/를 붙이세요. LiteLLM은 routers/<id>를 accounts/fireworks/routers/<id>로 확장해요. 라우터에 대한 자세한 내용은 Fireworks routers 문서를 참고하세요.

from litellm import completion
import os

os.environ['FIREWORKS_AI_API_KEY'] = ""

response = completion(
    model="fireworks_ai/routers/glm-latest",
    messages=[
        {"role": "user", "content": "hello from litellm"}
    ],
)
print(response)

전체 resource id(fireworks_ai/accounts/fireworks/routers/glm-latest)도 명시적으로 원하면 받아들여져요. -fast로 끝나는 slug(예: fireworks_ai/glm-5p2-fast)는 routers/ 접두사 없이도 라우터로 처리돼요.

LiteLLM Proxy 사용법

1. config.yaml에 Fireworks AI 모델 설정

model_list:
  - model_name: fireworks-glm-5p2
    litellm_params:
      model: fireworks_ai/glm-5p2
      api_key: "os.environ/FIREWORKS_AI_API_KEY"

2. Proxy 시작

litellm --config config.yaml

3. 테스트

curl:

curl --location 'http://0.0.0.0:4000/chat/completions' \
  --header 'Content-Type: application/json' \
  --data ' {
    "model": "fireworks-glm-5p2",
    "messages": [
      {
        "role": "user",
        "content": "what llm are you"
      }
    ]
  }'

OpenAI v1.0.0+:

import openai

client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(
    model="fireworks-glm-5p2",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ]
)
print(response)

Langchain:

from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
    SystemMessagePromptTemplate,
)
from langchain.schema import HumanMessage, SystemMessage

chat = ChatOpenAI(
    openai_api_base="http://0.0.0.0:4000",  # set openai_api_base to the LiteLLM Proxy
    model = "fireworks-glm-5p2",
    temperature=0.1
)

messages = [
    SystemMessage(
        content="You are a helpful assistant that im using to make a test request to."
    ),
    HumanMessage(
        content="test from litellm. tell me why it's amazing in 1 sentence"
    ),
]
response = chat(messages)
print(response)

Responses API

/v1/responses의 fireworks_ai/ 모델은 Fireworks의 네이티브 https://api.fireworks.ai/inference/v1/responses 엔드포인트로 바로 가므로, MCP tools("type": "mcp"), previous_response_id, reasoning output items 같은 서버 측 기능이 Fireworks에 직접 호출할 때와 동일하게 동작해요.

SDK:

import os
from litellm import responses

os.environ["FIREWORKS_AI_API_KEY"] = "YOUR_API_KEY"

response = responses(
    model="fireworks_ai/accounts/fireworks/models/kimi-k3",
    input="Use the deepwiki MCP server to tell me in one sentence what the BerriAI/litellm repository is.",
    tools=[
        {
            "type": "mcp",
            "server_label": "deepwiki",
            "server_url": "https://mcp.deepwiki.com/mcp",
            "require_approval": "never",
        }
    ],
)
print(response.output)

Proxy:

model_list:
  - model_name: fireworks-kimi-k3
    litellm_params:
      model: fireworks_ai/accounts/fireworks/models/kimi-k3
      api_key: "os.environ/FIREWORKS_AI_API_KEY"

litellm --config /path/to/config.yaml

curl http://0.0.0.0:4000/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "fireworks-kimi-k3",
    "input": "Use the deepwiki MCP server to tell me in one sentence what the BerriAI/litellm repository is.",
    "tools": [
      {
        "type": "mcp",
        "server_label": "deepwiki",
        "server_url": "https://mcp.deepwiki.com/mcp",
        "require_approval": "never"
      }
    ]
  }'

다중 턴 tool calling은 Fireworks에 직접 호출할 때와 동일하게 동작해요: function_call_output 항목을 Fireworks가 반환한 previous_response_id와 함께 보내면 Fireworks가 서버 측에서 대화를 계속해요. developer input 항목은 Fireworks의 Responses API가 kimi-k3, qwen3.8 같은 모델에 developer 역할이 없으므로 system 메시지로 Fireworks에 전송돼요.

문서 인라인 (Document Inlining)

LiteLLM은 Fireworks AI 모델용 문서 인라인을 지원해요. 비전 모델이 아니지만 문서/이미지 등을 파싱해야 하는 모델에 유용해요. 모델이 비전 모델이 아니면 LiteLLM이 image_url의 url에 #transform=inline을 추가해요.

SDK:

from litellm import completion
import os

os.environ["FIREWORKS_AI_API_KEY"] = "YOUR_API_KEY"
os.environ["FIREWORKS_AI_API_BASE"] = "https://audio-prod.api.fireworks.ai/v1"

completion = litellm.completion(
    model="fireworks_ai/accounts/fireworks/models/llama-v3p3-70b-instruct",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://storage.googleapis.com/fireworks-public/test/sample_resume.pdf"
                    },
                },
                {
                    "type": "text",
                    "text": "What are the candidate's BA and MBA GPAs?",
                },
            ],
        }
    ],
)
print(completion)

Proxy:

model_list:
  - model_name: llama-v3p3-70b-instruct
    litellm_params:
      model: fireworks_ai/accounts/fireworks/models/llama-v3p3-70b-instruct
      api_key: os.environ/FIREWORKS_AI_API_KEY
      # api_base: os.environ/FIREWORKS_AI_API_BASE [OPTIONAL], defaults to "https://api.fireworks.ai/inference/v1"

litellm --config config.yaml

curl -L -X POST 'http://0.0.0.0:4000/chat/completions' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer ***' \
  -d '{
    "model": "llama-v3p3-70b-instruct",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "image_url",
            "image_url": {
              "url": "https://storage.googleapis.com/fireworks-public/test/sample_resume.pdf"
            },
          },
          {
            "type": "text",
            "text": "What are the candidate's BA and MBA GPAs?",
          },
        ],
      }
    ]
  }'

자동 추가 비활성화

image_url의 url에 #transform=inline 자동 추가를 비활성화하려면 disable_add_transform_inline_image_block을 True로 설정하세요.

SDK:

litellm.disable_add_transform_inline_image_block = True

Proxy:

litellm_settings:
  disable_add_transform_inline_image_block: true

Reasoning Effort

reasoning_effort 파라미터는 선택된 Fireworks AI 모델에서 지원돼요.

SDK:

from litellm import completion
import os

os.environ["FIREWORKS_AI_API_KEY"] = "YOUR_API_KEY"

response = completion(
    model="fireworks_ai/accounts/fireworks/models/qwen3-8b",
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ],
    reasoning_effort="low",
)
print(response)

Proxy:

curl http://0.0.0.0:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "fireworks_ai/accounts/fireworks/models/qwen3-8b",
    "messages": [
      {
        "role": "user",
        "content": "What is the capital of France?"
      }
    ],
    "reasoning_effort": "low"
  }'

지원 모델 - 모든 Fireworks AI 모델 지원!

모델 이름	함수 호출
glm-5p2	completion(model="fireworks_ai/glm-5p2", messages)
deepseek-v4-pro	completion(model="fireworks_ai/deepseek-v4-pro", messages)
kimi-k3	completion(model="fireworks_ai/kimi-k3", messages)
qwen3p8-max	completion(model="fireworks_ai/qwen3p8-max", messages)
minimax-m3	completion(model="fireworks_ai/minimax-m3", messages)
gpt-oss-120b	completion(model="fireworks_ai/gpt-oss-120b", messages)

위 표는 인기 모델의 작은 선택이에요. 전체 최신 모델/라우터 목록은 Fireworks model library를 참고하세요.

지원 임베딩 모델

모델 이름	함수 호출
fireworks_ai/nomic-ai/nomic-embed-text-v1.5	response = litellm.embedding(model="fireworks_ai/nomic-ai/nomic-embed-text-v1.5", input=input_text)
fireworks_ai/nomic-ai/nomic-embed-text-v1	response = litellm.embedding(model="fireworks_ai/nomic-ai/nomic-embed-text-v1", input=input_text)
fireworks_ai/WhereIsAI/UAE-Large-V1	response = litellm.embedding(model="fireworks_ai/WhereIsAI/UAE-Large-V1", input=input_text)
fireworks_ai/thenlper/gte-large	response = litellm.embedding(model="fireworks_ai/thenlper/gte-large", input=input_text)
fireworks_ai/thenlper/gte-base	response = litellm.embedding(model="fireworks_ai/thenlper/gte-base", input=input_text)

오디오 전사 (Audio Transcription)

SDK:

from litellm import transcription
import os

os.environ["FIREWORKS_AI_API_KEY"] = "YOUR_API_KEY"
os.environ["FIREWORKS_AI_API_BASE"] = "https://audio-prod.api.fireworks.ai/v1"

response = transcription(
    model="fireworks_ai/whisper-v3",
    audio=audio_file,
)

Proxy:

model_list:
  - model_name: whisper-v3
    litellm_params:
      model: fireworks_ai/whisper-v3
      api_base: https://audio-prod.api.fireworks.ai/v1
      api_key: os.environ/FIREWORKS_API_KEY
      model_info:
        mode: audio_transcription

litellm --config config.yaml

curl -L -X POST 'http://0.0.0.0:4000/v1/audio/transcriptions' \
  -H "Authorization: Bearer ***" \
  -F 'file=@"/Users/krrishdholakia/Downloads/gettysburg.wav"' \
  -F 'model="whisper-v3"' \
  -F 'response_format="verbose_json"' \

Rerank

SDK:

from litellm import rerank
import os

os.environ["FIREWORKS_AI_API_KEY"] = "YOUR_API_KEY"
query = "What is the capital of France?"
documents = [
    "Paris is the capital and largest city of France, home to the Eiffel Tower and the Louvre Museum.",
    "France is a country in Western Europe known for its wine, cuisine, and rich history.",
    "The weather in Europe varies significantly between northern and southern regions.",
    "Python is a popular programming language used for web development and data science.",
]

response = rerank(
    model="fireworks_ai/fireworks/qwen3-reranker-8b",
    query=query,
    documents=documents,
    top_n=3,
    return_documents=True,
)
print(response)

Proxy:

model_list:
  - model_name: qwen3-reranker-8b
    litellm_params:
      model: fireworks_ai/fireworks/qwen3-reranker-8b
      api_key: os.environ/FIREWORKS_API_KEY
      model_info:
        mode: rerank

litellm --config config.yaml

curl http://0.0.0.0:4000/rerank \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-reranker-8b",
    "query": "What is the capital of France?",
    "documents": [
      "Paris is the capital and largest city of France, home to the Eiffel Tower and the Louvre Museum.",
      "France is a country in Western Europe known for its wine, cuisine, and rich history.",
      "The weather in Europe varies significantly between northern and southern regions.",
      "Python is a popular programming language used for web development and data science."
    ],
    "top_n": 3,
    "return_documents": true
  }'

더 알아보기 (Learn more)

• LiteLLM 개요
• LiteLLM Python SDK 사용법
• LiteLLM Proxy Server 설정
• 지원되는 공급업체 전체 목록