Databricks

Databricks

LiteLLM은 Databricks의 모든 모델을 지원해요. litellm 요청 시 model=databricks/<any-model-on-databricks> 접두사로 설정하기만 하면 돼요.

출처: 문서

본문

인증 (Authentication)

LiteLLM은 Databricks에 대해 선호 순서대로 여러 인증 방법을 지원해요.

OAuth M2M (프로덕션 권장)

Databricks 파트너 요구사항에 따라 프로덕션 배포에는 Service Principal 자격 증명을 사용하는 OAuth Machine-to-Machine 인증이 권장돼요.

import os
from litellm import completion

# Set OAuth credentials (Service Principal)
os.environ["DATABRICKS_CLIENT_ID"] = "your-service-principal-application-id"
os.environ["DATABRICKS_CLIENT_SECRET"] = "your-service-principal-secret"
os.environ["DATABRICKS_API_BASE"] = "https://adb-xxx.azuredatabricks.net/serving-endpoints"

response = completion(
    model="databricks/databricks-dbrx-instruct",
    messages=[{"role": "user", "content": "Hello!"}],
)

Personal Access Token (PAT)

PAT 인증은 개발·테스트 시나리오에서 지원돼요.

import os
from litellm import completion

os.environ["DATABRICKS_API_KEY"] = "dapi..."  # Your Personal Access Token
os.environ["DATABRICKS_API_BASE"] = "https://adb-xxx.azuredatabricks.net/serving-endpoints"

response = completion(
    model="databricks/databricks-dbrx-instruct",
    messages=[{"role": "user", "content": "Hello!"}],
)

Databricks SDK 인증 (자동)

자격 증명이 제공되지 않으면 LiteLLM이 Databricks SDK로 자동 인증을 사용해요. 이는 환경에 구성된 OAuth, Azure AD 및 기타 통합 인증 방법을 지원해요.

from litellm import completion

# No environment variables needed - uses Databricks SDK unified auth
# Requires: uv add databricks-sdk
response = completion(
    model="databricks/databricks-dbrx-instruct",
    messages=[{"role": "user", "content": "Hello!"}],
)

파트너 귀속용 사용자 지정 User-Agent

LiteLLM 위에 Databricks와 통합되는 제품을 만든다면, Databricks 텔레메트리에서 올바른 귀속을 위해 자체 파트너 식별자를 전달할 수 있어요. 파트너 이름은 LiteLLM 사용자 에이전트 앞에 붙어요:

입력	결과 User-Agent
(none)	litellm/1.79.1
mycompany/1.0.0	mycompany_litellm/1.79.1
partner_product/2.5.0	partner_product_litellm/1.79.1
acme	acme_litellm/1.79.1

참고: 사용자 지정 user agent의 버전은 무시되며 LiteLLM의 버전이 항상 사용돼요.

# Via parameter
response = completion(
    model="databricks/databricks-dbrx-instruct",
    messages=[{"role": "user", "content": "Hello!"}],
    user_agent="mycompany/1.0.0",
)
# Resulting User-Agent: mycompany_litellm/1.79.1

# Via environment variable
os.environ["DATABRICKS_USER_AGENT"] = "mycompany/1.0.0"
# Resulting User-Agent: mycompany_litellm/1.79.1

보안 (Security)

LiteLLM은 자격 증명 유출을 방지하기 위해 모든 디버그 로그에서 민감한 정보(토큰, 시크릿, API 키)를 자동으로 redact해요. 여기에는 다음이 포함돼요:

• Authorization 헤더
• API 키와 토큰
• Client secrets
• Personal access tokens (PATs)

사용법 (Usage)

ENV VAR:

import os

os.environ["DATABRICKS_API_KEY"] = ""
os.environ["DATABRICKS_API_BASE"] = ""

SDK:

from litellm import completion
import os

## set ENV variables
os.environ["DATABRICKS_API_KEY"] = "databricks key"
os.environ["DATABRICKS_API_BASE"] = "databricks base url"  # e.g.: https://adb-3064715882934586.6.azuredatabricks.net/serving-endpoints

# Databricks dbrx-instruct call
response = completion(
    model="databricks/databricks-dbrx-instruct",
    messages = [{ "content": "Hello, how are you?","role": "user"}]
)

Proxy:

model_list:
  - model_name: dbrx-instruct
    litellm_params:
      model: databricks/databricks-dbrx-instruct
      api_key: os.environ/DATABRICKS_API_KEY
      api_base: os.environ/DATABRICKS_API_BASE
      user_agent: "mycompany/1.0.0"  # Optional: for partner attribution

Proxy 시작 후 테스트:

$ litellm --config /path/to/config.yaml --debug

OpenAI Python v1.0.0+:

import openai

client = openai.OpenAI(
    api_key="sk-<your-litellm-api-key>",  # pass litellm proxy key, if you're using virtual keys
    base_url="http://0.0.0.0:4000"  # litellm-proxy-base url
)

response = client.chat.completions.create(
    model="dbrx-instruct",
    messages = [
        {
            "role": "system",
            "content": "Be a good human!"
        },
        {
            "role": "user",
            "content": "What do you know about earth?"
        }
    ]
)
print(response)

curl:

curl --location 'http://0.0.0.0:4000/chat/completions' \
  --header "Authorization: Bearer ***" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "dbrx-instruct",
    "messages": [
      {
        "role": "system",
        "content": "Be a good human!"
      },
      {
        "role": "user",
        "content": "What do you know about earth?"
      }
    ],
  }'

추가 파라미터 전달 - max_tokens, temperature

SDK:

# !uv add litellm
from litellm import completion
import os

## set ENV variables
os.environ["DATABRICKS_API_KEY"] = "databricks key"
os.environ["DATABRICKS_API_BASE"] = "databricks api base"

# databricks dbrx call
response = completion(
    model="databricks/databricks-dbrx-instruct",
    messages = [{ "content": "Hello, how are you?","role": "user"}],
    max_tokens=20,
    temperature=0.5,
)

Proxy:

model_list:
  - model_name: llama-3
    litellm_params:
      model: databricks/databricks-meta-llama-3-70b-instruct
      api_key: os.environ/DATABRICKS_API_KEY
      max_tokens: 20
      temperature: 0.5

Thinking / reasoning_content 사용

LiteLLM은 OpenAI의 reasoning_effort를 Anthropic의 thinking 파라미터로 변환해요.

SDK:

from litellm import completion
import os

# set ENV variables (can also be passed in to .completion() - e.g. `api_base`, `api_key`)
os.environ["DATABRICKS_API_KEY"] = "databricks key"
os.environ["DATABRICKS_API_BASE"] = "databricks base url"

resp = completion(
    model="databricks/databricks-claude-sonnet-5",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    reasoning_effort="low",
)

reasoning_effort	thinking
"low"	"budget_tokens": 1024
"medium"	"budget_tokens": 2048
"high"	"budget_tokens": 4096

Proxy:

- model_name: claude-sonnet-5
  litellm_params:
    model: databricks/databricks-claude-sonnet-5
    api_key: os.environ/DATABRICKS_API_KEY
    api_base: os.environ/DATABRICKS_API_BASE

litellm --config /path/to/config.yaml

curl http://0.0.0.0:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <YOUR-...KEY>" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [{"role": "user", "content": "What is the capital of France?"}],
    "reasoning_effort": "low"
  }'

예상 응답:

ModelResponse(
    id='chatcmpl-c542d76d-f675-4e87-8e5f-05855f5d0f5e',
    created=1740470510,
    model='claude-sonnet-5',
    object='chat.completion',
    system_fingerprint=None,
    choices=[
        Choices(
            finish_reason='stop',
            index=0,
            message=Message(
                content="The capital of France is Paris.",
                role='assistant',
                tool_calls=None,
                function_call=None,
                provider_specific_fields={
                    'citations': None,
                    'thinking_blocks': [
                        {
                            'type': 'thinking',
                            'thinking': 'The capital of France is Paris. This is a very straightforward factual question.',
                            'signature': 'EuYBCkQYAiJAy6...'
                        }
                    ]
                }
            ),
            thinking_blocks=[
                {
                    'type': 'thinking',
                    'thinking': 'The capital of France is Paris. This is a very straightforward factual question.',
                    'signature': 'EuYBCkQYAiJAy6AGB...'
                }
            ],
            reasoning_content='The capital of France is Paris. This is a very straightforward factual question.'
        )
    ],
    usage=Usage(
        completion_tokens=68,
        prompt_tokens=42,
        total_tokens=110,
        ...
    )
)

알려진 제한: thinking blocks를 Claude로 다시 전달하는 지원 Issue

Citations

Databricks를 통해 서빙되는 Anthropic 모델은 citation 메타데이터를 반환할 수 있어요. LiteLLM은 이를 response.choices[0].message.provider_specific_fields["citations"]로 노출해요.

Anthropic 모델에 thinking 전달

thinking 파라미터를 Anthropic 모델에도 전달할 수 있어요.

SDK:

from litellm import completion
import os

os.environ["DATABRICKS_API_KEY"] = "databricks key"
os.environ["DATABRICKS_API_BASE"] = "databricks base url"

response = litellm.completion(
    model="databricks/databricks-claude-3-7-sonnet",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    thinking={"type": "enabled", "budget_tokens": 1024},
)

Proxy:

curl http://0.0.0.0:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "databricks/databricks-claude-3-7-sonnet",
    "messages": [{"role": "user", "content": "What is the capital of France?"}],
    "thinking": {"type": "enabled", "budget_tokens": 1024}
  }'

지원 Databricks Chat Completion 모델

모든 Databricks 모델을 지원해요. litellm 요청 시 model=databricks/<any-model-on-databricks> 접두사로 설정하기만 하면 돼요.

모델 이름	명령
databricks/databricks-claude-3-7-sonnet	completion(model='databricks/databricks/databricks-claude-3-7-sonnet', messages=messages)
databricks-meta-llama-3-1-70b-instruct	completion(model='databricks/databricks-meta-llama-3-1-70b-instruct', messages=messages)
databricks-meta-llama-3-1-405b-instruct	completion(model='databricks/databricks-meta-llama-3-1-405b-instruct', messages=messages)
databricks-dbrx-instruct	completion(model='databricks/databricks-dbrx-instruct', messages=messages)
databricks-meta-llama-3-70b-instruct	completion(model='databricks/databricks-meta-llama-3-70b-instruct', messages=messages)
databricks-llama-2-70b-chat	completion(model='databricks/databricks-llama-2-70b-chat', messages=messages)
databricks-mixtral-8x7b-instruct	completion(model='databricks/databricks-mixtral-8x7b-instruct', messages=messages)
databricks-mpt-30b-instruct	completion(model='databricks/databricks-mpt-30b-instruct', messages=messages)
databricks-mpt-7b-instruct	completion(model='databricks/databricks-mpt-7b-instruct', messages=messages)

임베딩 모델 (Embedding Models)

Databricks 임베딩 모델에는 추가 파라미터 instruction을 전달할 수 있어요.

SDK:

# !uv add litellm
from litellm import embedding
import os

## set ENV variables
os.environ["DATABRICKS_API_KEY"] = "databricks key"
os.environ["DATABRICKS_API_BASE"] = "databricks url"

# Databricks bge-large-en call
response = litellm.embedding(
    model="databricks/databricks-bge-large-en",
    input=["good morning from litellm"],
    instruction="Represent this sentence for searching relevant passages:",
)

Proxy:

model_list:
  - model_name: bge-large
    litellm_params:
      model: databricks/databricks-bge-large-en
      api_key: os.environ/DATABRICKS_API_KEY
      api_base: os.environ/DATABRICKS_API_BASE
      instruction: "Represent this sentence for searching relevant passages:"

더 알아보기 (Learn more)

• LiteLLM 개요
• LiteLLM Python SDK 사용법
• LiteLLM Proxy Server 설정
• 지원되는 공급업체 전체 목록