베타 Observability 저지 엔드포인트

베타 Observability 저지 엔드포인트 (Beta Observability Judges Endpoints)

(beta) Observability API - judges입니다. 평가 기준이 되는 저지(Judge)를 만들고, 조회·수정·삭제하며, 대화에 바로 실행하는 엔드포인트예요.

출처: 문서

본문

LLM 출력을 평가하는 저지를 관리하는 API예요. 저지를 정의하면 그 기준대로 응답의 충실성·정확성 등을 점수/분류로 평가하고, 저장된 저지를 특정 대화에 바로(live) 실행할 수도 있답니다.

GET /v1/observability/judges — Get judges with optional filtering and search (저지 목록 조회)

선택적 필터링과 검색으로 저지 목록을 가져옵니다.

응답 필드 (200 Successful Response):

judges#PaginatedResultJudgePreview (필수) — 페이지네이션된 저지 미리보기.

TypeScript:

import { Mistral } from "@mistralai/mistralai";

const mistral = new Mistral({
  apiKey: proces...EY"] ?? "",
});

async function run() {
  const result = await mistral.beta.observability.judges.list({});

  console.log(result);
}

run();

Python:

from mistralai.client import Mistral
import os

with Mistral(
    api_key=os.getenv("MISTRAL_API_KEY", ""),
) as mistral:

    res = mistral.beta.observability.judges.list(page_size=50, page=1)

    # Handle response
    print(res)

curl:

curl https://api.mistral.ai/v1/observability/judges \
 -X GET \
 -H 'Authorization: Bearer ***'

응답 예시 (200):

{
  "judges": {
    "count": 87
  }
}

POST /v1/observability/judges — Create a new judge (저지 생성)

새 저지를 만듭니다.

요청 본문:

name#string (필수) — 저지 이름.
description#string (필수) — 저지 설명.
instructions#string (필수) — 평가 지침.
model_name#string (필수) — 평가에 사용할 모델.
output#JudgeClassificationOutput|JudgeRegressionOutput (필수) — 출력 형식(분류 또는 회귀).
tools#array<string> (필수) — 사용할 도구 목록.

응답 필드 (201 Successful Response):

id#string (필수), name#string (필수), description#string (필수), instructions#string (필수), model_name#string (필수), output#JudgeClassificationOutput|JudgeRegressionOutput (필수), tools#array<string> (필수), created_at#date-time (필수), updated_at#date-time (필수), deleted_at#date-time|null (필수), owner_id#string (필수), workspace_id#string (필수), base_revision#string|null, up_revision#string|null, down_revision#string|null

TypeScript:

import { Mistral } from "@mistralai/mistralai";

const mistral = new Mistral({
  apiKey: proces...EY"] ?? "",
});

async function run() {
  const result = await mistral.beta.observability.judges.create({
    name: "<value>",
    description: "border freely down whenever broadly whenever restructure catalyze after",
    modelName: "<value>",
    output: {
      type: "REGRESSION",
      min: 0,
      minDescription: "<value>",
      max: 1,
      maxDescription: "<value>",
    },
    instructions: "<value>",
    tools: [
      "<value 1>",
      "<value 2>",
    ],
  });

  console.log(result);
}

run();

Python:

from mistralai.client import Mistral
import os

with Mistral(
    api_key=os.getenv("MISTRAL_API_KEY", ""),
) as mistral:

    res = mistral.beta.observability.judges.create(name="<value>", description="border freely down whenever broadly whenever restructure catalyze after", model_name="<value>", output={
        "type": "REGRESSION",
        "min": 0,
        "min_description": "<value>",
        "max": 1,
        "max_description": "<value>",
    }, instructions="<value>", tools=[
        "<value 1>",
        "<value 2>",
    ])

    # Handle response
    print(res)

curl:

curl https://api.mistral.ai/v1/observability/judges \
 -X POST \
 -H 'Authorization: Bearer ***' \
 -H 'Content-Type: application/json' \
 -d '{
  "description": "My Judge description.",
  "instructions": "Evaluate the response.",
  "model_name": "mistral-small-latest",
  "name": "My Judge",
  "output": {
    "options": [
      {
        "description": "My Judge",
        "value": "approved"
      }
    ]
  },
  "tools": [
    "approved"
  ]
}'

응답 예시 (201):

{
  "created_at": "2025-12-17T10:25:07.818693Z",
  "deleted_at": null,
  "description": "My resource description.",
  "id": "019b2bd7-96e7-7219-8c0b-45a73da50088",
  "instructions": "Evaluate the response.",
  "model_name": "mistral-small-latest",
  "name": "My resource",
  "output": {
    "options": [
      {
        "description": "My Judge",
        "value": "approved"
      }
    ]
  },
  "owner_id": "9c0ab39f-0cd0-46cd-bd30-8bf2d50be5ce",
  "tools": [
    "approved"
  ],
  "updated_at": "2025-12-17T10:41:03.469341Z",
  "workspace_id": "019b2bd7-96e7-7219-8c0b-45a73da50088"
}

GET /v1/observability/judges/{judge_id} — Get judge by id (저지 조회)

ID로 저지를 가져옵니다.

경로 파라미터:

judge_id#string (필수)

응답 필드 (200 Successful Response): 생성 응답과 동일한 필드 목록.

TypeScript:

import { Mistral } from "@mistralai/mistralai";

const mistral = new Mistral({
  apiKey: proces...EY"] ?? "",
});

async function run() {
  const result = await mistral.beta.observability.judges.fetch({
    judgeId: "19ae5cf8-2ade-4a40-b9d2-730aaebe8429",
  });

  console.log(result);
}

run();

Python:

from mistralai.client import Mistral
import os

with Mistral(
    api_key=os.getenv("MISTRAL_API_KEY", ""),
) as mistral:

    res = mistral.beta.observability.judges.fetch(judge_id="19ae5cf8-2ade-4a40-b9d2-730aaebe8429")

    # Handle response
    print(res)

curl:

curl https://api.mistral.ai/v1/observability/judges/{judge_id} \
 -X GET \
 -H 'Authorization: Bearer ***'

응답 예시 (200): 생성 응답과 동일한 Judge JSON 구조입니다.

PUT /v1/observability/judges/{judge_id} — Update a judge (저지 수정)

저지를 수정합니다.

경로 파라미터:

judge_id#string (필수)

요청 본문:

name#string (필수), description#string (필수), instructions#string (필수), model_name#string (필수), output#JudgeClassificationOutput|JudgeRegressionOutput (필수), tools#array<string> (필수)

TypeScript:

import { Mistral } from "@mistralai/mistralai";

const mistral = new Mistral({
  apiKey: proces...EY"] ?? "",
});

async function run() {
  await mistral.beta.observability.judges.update({
    judgeId: "9f28c7db-1fb7-4e1c-b137-d7039561ddb7",
    updateJudgeRequest: {
      name: "<value>",
      description: "noteworthy and unless",
      modelName: "<value>",
      output: {
        type: "REGRESSION",
        min: 0,
        minDescription: "<value>",
        max: 1,
        maxDescription: "<value>",
      },
      instructions: "<value>",
      tools: [],
    },
  });

}

run();

Python:

from mistralai.client import Mistral
import os

with Mistral(
    api_key=os.getenv("MISTRAL_API_KEY", ""),
) as mistral:

    mistral.beta.observability.judges.update(judge_id="9f28c7db-1fb7-4e1c-b137-d7039561ddb7", name="<value>", description="noteworthy and unless", model_name="<value>", output={
        "type": "REGRESSION",
        "min": 0,
        "min_description": "<value>",
        "max": 1,
        "max_description": "<value>",
    }, instructions="<value>", tools=[])

    # Use the SDK ...

curl:

curl https://api.mistral.ai/v1/observability/judges/{judge_id} \
 -X PUT \
 -H 'Authorization: Bearer ***' \
 -H 'Content-Type: application/json' \
 -d '{
  "description": "Checks whether the answer is grounded in the retrieved context.",
  "instructions": "Score the response from 1 to 5 based on factual accuracy and faithfulness to the sources.",
  "model_name": "mistral-large-latest",
  "name": "Faithfulness",
  "output": {
    "options": [
      {
        "description": "My Judge",
        "value": "approved"
      }
    ]
  },
  "tools": [
    "web_search"
  ]
}'

DELETE /v1/observability/judges/{judge_id} — Delete a judge (저지 삭제)

저지를 삭제합니다.

경로 파라미터:

judge_id#string (필수)

TypeScript:

import { Mistral } from "@mistralai/mistralai";

const mistral = new Mistral({
  apiKey: proces...EY"] ?? "",
});

async function run() {
  await mistral.beta.observability.judges.delete({
    judgeId: "80deecde-e10f-409c-a13a-c242d3760f6e",
  });

}

run();

Python:

from mistralai.client import Mistral
import os

with Mistral(
    api_key=os.getenv("MISTRAL_API_KEY", ""),
) as mistral:

    mistral.beta.observability.judges.delete(judge_id="80deecde-e10f-409c-a13a-c242d3760f6e")

    # Use the SDK ...

curl:

curl https://api.mistral.ai/v1/observability/judges/{judge_id} \
 -X DELETE \
 -H 'Authorization: Bearer ***' \
 -H 'Content-Type: application/json'

POST /v1/observability/judges/{judge_id}/live-judging — Run a saved judge on a conversation (저장된 저지 실행)

저장된 저지를 대화에 대해 실행합니다.

경로 파라미터:

judge_id#string (필수)

요청 본문:

messages#array<map<any>> (필수) — 평가할 대화 메시지.
properties#map<any>|null — 추가 속성.

응답 필드 (200 Successful Response):

analysis#string (필수) — 분석 텍스트.
answer#string|number (필수) — 저지 판정 결과.

TypeScript:

import { Mistral } from "@mistralai/mistralai";

const mistral = new Mistral({
  apiKey: proces...EY"] ?? "",
});

async function run() {
  const result = await mistral.beta.observability.judges.judgeConversation({
    judgeId: "7fb9f06a-22fb-45db-ad58-6a5715280755",
    judgeConversationRequest: {
      messages: [],
    },
  });

  console.log(result);
}

run();

Python:

from mistralai.client import Mistral
import os

with Mistral(
    api_key=os.getenv("MISTRAL_API_KEY", ""),
) as mistral:

    res = mistral.beta.observability.judges.judge_conversation(judge_id="7fb9f06a-22fb-45db-ad58-6a5715280755", messages=[])

    # Handle response
    print(res)

curl:

curl https://api.mistral.ai/v1/observability/judges/{judge_id}/live-judging \
 -X POST \
 -H 'Authorization: Bearer ***' \
 -H 'Content-Type: application/json' \
 -d '{
  "messages": [
    [
      null
    ]
  ]
}'

응답 예시 (200):

{
  "analysis": "Example analysis.",
  "answer": "Example answer."
}

더 알아보기 (Learn more)

Beta Observability Judges Endpoints — 공식 API 문서
Observability — 관측성 안내