Prometheus & Grafana 모니터링

Prometheus & Grafana 모니터링 (Prometheus Grafana)

vLLM 배포를 Prometheus로 수집하고 Grafana로 시각화하는 오프라인·온라인 구성 예제입니다. docker-compose로 Prometheus·Grafana·vLLM을 한 번에 띄우고 대시보드 JSON을 임포트해 모니터링 환경을 갖춥니다.

출처: 문서

본문

docker-compose.yaml이 인프라를 구성하고, prometheus.yaml이 vLLM 지표 엔드포인트를 스크레이프하며, grafana.json이 성능 대시보드를 정의합니다.

README.md

# Prometheus and Grafana

This is a simple example that shows you how to connect vLLM metric logging to the Prometheus/Grafana stack. For this example, we launch Prometheus and Grafana via Docker. You can checkout other methods through [Prometheus](https://prometheus.io/) and [Grafana](https://grafana.com/) websites.

Install:

- [`docker`](https://docs.docker.com/engine/install/)
- [`docker compose`](https://docs.docker.com/compose/install/linux/#install-using-the-repository)

## Launch

Prometheus metric logging is enabled by default in the OpenAI-compatible server. Launch via the entrypoint:

```bash
vllm serve mistralai/Mistral-7B-v0.1 \
    --max-model-len 2048

Launch Prometheus and Grafana servers with docker compose:

docker compose up

Submit some sample requests to the server:

wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json

vllm bench serve \
    --model mistralai/Mistral-7B-v0.1 \
    --tokenizer mistralai/Mistral-7B-v0.1 \
    --endpoint /v1/completions \
    --dataset-name sharegpt \
    --dataset-path ShareGPT_V3_unfiltered_cleaned_split.json \
    --request-rate 3.0

Navigating to http://localhost:8000/metrics will show the raw Prometheus metrics being exposed by vLLM.

Grafana Dashboard

Navigate to http://localhost:3000. Log in with the default username (admin) and password (admin).

Add Prometheus Data Source

Navigate to http://localhost:3000/connections/datasources/new and select Prometheus.

On Prometheus configuration page, we need to add the Prometheus Server URL in Connection. For this setup, Grafana and Prometheus are running in separate containers, but Docker creates DNS name for each container. You can just use http://prometheus:9090.

Click Save & Test. You should get a green check saying "Successfully queried the Prometheus API.".

Import Dashboard

Navigate to http://localhost:3000/dashboard/import, upload grafana.json, and select the prometheus datasource. You should see a screen that looks like the following:

Grafana Dashboard Image


### docker-compose.yaml

```yaml
# docker-compose.yaml
version: "3"

services:
  prometheus:
    image: prom/prometheus:latest
    extra_hosts:
      - "host.docker.internal:host-gateway"     # allow a direct connection from container to the local machine
    ports:
      - "9090:9090"   # the default port used by Prometheus
    volumes:
      - ${PWD}/prometheus.yaml:/etc/prometheus/prometheus.yml # mount Prometheus config file

  grafana:
    image: grafana/grafana:latest
    depends_on:
      - prometheus
    ports:
      - "3000:3000" # the default port used by Grafana

prometheus.yaml

# prometheus.yaml
global:
  scrape_interval: 5s
  evaluation_interval: 30s

scrape_configs:
  - job_name: vllm
    static_configs:
      - targets:
          - 'host.docker.internal:8000'

더 알아보기 (Learn more)