LiteLLM과 함께 Elasticsearch 로깅 사용하기

OpenTelemetry를 사용해 LLM 요청, 응답, 비용, 성능 데이터를 Elasticsearch로 보내 분석과 모니터링을 하는 방법을 알려드릴게요.

출처: 문서

본문

빠른 시작

1. Elasticsearch 시작

# Using Docker (simplest)docker run -d \  --name elasticsearch \  -p 9200:9200 \  -e "discovery.type=single-node" \  -e "xpack.security.enabled=false" \  docker.elastic.co/elasticsearch/elasticsearch:8.18.2

2. OpenTelemetry Collector 설정

OTEL collector 구성 파일 otel_config.yaml을 만들어요.

receivers:  otlp:    protocols:      grpc:        endpoint: 0.0.0.0:4317      http:        endpoint: 0.0.0.0:4318processors:  batch:    timeout: 1s    send_batch_size: 1024exporters:  debug:    verbosity: detailed  otlphttp/elastic:    endpoint: "http://localhost:9200"    headers:       "Content-Type": "application/json"service:  pipelines:    metrics:      receivers: [otlp]      exporters: [debug, otlphttp/elastic]    traces:      receivers: [otlp]      exporters: [debug, otlphttp/elastic]    logs:       receivers: [otlp]      exporters: [debug, otlphttp/elastic]

OpenTelemetry collector를 시작해요.

docker run -p 4317:4317 -p 4318:4318 \    -v $(pwd)/otel_config.yaml:/etc/otel-collector-config.yaml \    otel/opentelemetry-collector:latest \    --config=/etc/otel-collector-config.yaml

3. OpenTelemetry 의존성 설치

uv add opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp

4. LiteLLM 구성

LiteLLM Proxy
Python SDK

config.yaml 파일을 만들어요.

model_list:  - model_name: gpt-5.6-terra    litellm_params:      model: openai/gpt-5.6-terra      api_key: os.environ/OPENAI_API_KEYlitellm_settings:  callbacks: ["otel"]general_settings:  otel: true

환경 변수를 설정하고 프록시를 시작해요.

export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317"litellm --config config.yaml

Python 코드에서 OpenTelemetry를 구성해요.

import litellmimport os# Configure OpenTelemetryos.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "http://localhost:4317"# Enable OTEL logginglitellm.callbacks = ["otel"]# Make your LLM callsresponse = litellm.completion(    model="gpt-5.6-terra",    messages=[{"role": "user", "content": "Hello, world!"}])

5. 통합 테스트

로깅이 동작하는지 확인하기 위해 테스트 요청을 보내 보세요.

Test Proxy
Test Python SDK

curl -X POST "http://localhost:4000/v1/chat/completions" \  -H "Content-Type: application/json" \  -H "Authorization: Bearer ***" \  -d '{    "model": "gpt-5.6-terra",    "messages": [{"role": "user", "content": "Hello from LiteLLM!"}]  }'

import litellmresponse = litellm.completion(    model="gpt-5.6-terra",    messages=[{"role": "user", "content": "Hello from LiteLLM!"}],    user="test-user")print("Response:", response.choices[0].message.content)

6. 동작 확인

# Check if traces are being created in Elasticsearchcurl "localhost:9200/_search?pretty&size=1"

LLM 요청에 대한 구조화된 필드가 있는 OpenTelemetry trace 데이터가 보여야 해요.

7. Kibana에서 시각화

Kibana를 시작해 LLM 텔레메트리 데이터를 시각화해요.

docker run -d --name kibana --link elasticsearch:elasticsearch -p 5601:5601 docker.elastic.co/kibana/kibana:8.18.2

http://localhost:5601에서 Kibana를 열고 LiteLLM 트레이스용 인덱스 패턴을 만들어요.

프로덕션 설정

Elasticsearch Cloud 사용:

otel_config.yaml을 업데이트해요.

exporters:  otlphttp/elastic:    endpoint: "https://your-deployment.es.region.cloud.es.io"    headers:       "Authorization": "Bearer your-api-key"      "Content-Type": "application/json"

Docker Compose (전체 스택):

# docker-compose.ymlversion: '3.8'services:  elasticsearch:    image: docker.elastic.co/elasticsearch/elasticsearch:8.18.2    environment:      - discovery.type=single-node      - xpack.security.enabled=false    ports:      - "9200:9200"        otel-collector:    image: otel/opentelemetry-collector:latest    command: ["--config=/etc/otel-collector-config.yaml"]    volumes:      - ./otel_config.yaml:/etc/otel-collector-config.yaml    ports:      - "4317:4317"      - "4318:4318"    depends_on:      - elasticsearch        litellm:    image: docker.litellm.ai/berriai/litellm:latest    ports:      - "4000:4000"    environment:      - OPENAI_API_KEY=${OPENAI_API_KEY}      - OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317    command: ["--config", "/app/config.yaml"]    volumes:      - ./config.yaml:/app/config.yaml    depends_on:      - otel-collector

config.yaml:

model_list:  - model_name: gpt-5.6-terra    litellm_params:      model: openai/gpt-5.6-terra      api_key: os.environ/OPENAI_API_KEYlitellm_settings:  callbacks: ["otel"]general_settings:  master_key: os.environ/LITELLM_MASTER_KEY  otel: true

LiteLLM과 함께 Elasticsearch 로깅 사용하기