ElevenLabs

ElevenLabs

ElevenLabs는 음성-텍스트 변환 API를 통한 음성-텍스트 기능을 포함해 고품질 AI 음성 기술을 제공해요.

출처: 문서

본문

개요 (Overview)

속성 설명
설명 여러 언어와 화자 분리(diarization)를 지원하는 음성-텍스트 변환 및 텍스트-음성 변환 기능을 갖춘 고급 AI 음성 기술
LiteLLM 라우트 elevenlabs/
공급자 문서 ElevenLabs API
지원 엔드포인트 /audio/transcriptions, /audio/speech

빠른 시작 (Quick Start)

기본 사용법

import litellm

# Transcribe audio file
with open("audio.mp3", "rb") as audio_file:
    response = litellm.transcription(
        model="elevenlabs/scribe_v1",
        file=audio_file,
        api_key="your-elevenlabs-api-key"  # or set ELEVENLABS_API_KEY env var
    )
print(response.text)

고급 기능 (화자 분리와 언어 지정)

import litellm

# Transcribe with speaker diarization and language specification
with open("audio.wav", "rb") as audio_file:
    response = litellm.transcription(
        model="elevenlabs/scribe_v1",
        file=audio_file,
        language="en",  # Language hint (maps to language_code)
        temperature=0.3,  # Control randomness in transcription
        diarize=True,  # Enable speaker diarization
        api_key="your-elevenlabs-api-key"
    )

print(f"Transcription: {response.text}")
print(f"Language: {response.language}")

# Access word-level timestamps if available
if hasattr(response, 'words') and response.words:
    for word_info in response.words:
        print(f"Word: {word_info['word']}, Start: {word_info['start']}, End: {word_info['end']}")

비동기 사용법

import litellm
import asyncio

async def transcribe_audio():
    with open("audio.mp3", "rb") as audio_file:
        response = await litellm.atranscription(
            model="elevenlabs/scribe_v1",
            file=audio_file,
            api_key="your-elevenlabs-api-key"
        )
    return response.text

# Run async transcription
result = asyncio.run(transcribe_audio())
print(result)

LiteLLM Proxy 사용법

1. Proxy 구성

config.yaml:

model_list:
  - model_name: elevenlabs-transcription
    litellm_params:
      model: elevenlabs/scribe_v1
      api_key: os.environ/ELEVENLABS_API_KEY

general_settings:
  master_key: your-master-key

environment variables:

export ELEVENLABS_API_KEY="your-elevenlabs-api-key"
export LITELLM_MASTER_KEY="your-master-key"

2. Proxy 시작

litellm --config config.yaml
# Proxy will be available at http://localhost:4000

3. 전사 요청

curl:

curl http://localhost:4000/v1/audio/transcriptions \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: multipart/form-data" \
  -F file="@audio.mp3" \
  -F model="elevenlabs-transcription" \
  -F language="en" \
  -F temperature="0.3"

OpenAI Python SDK:

from openai import OpenAI

# Initialize client with your LiteLLM proxy URL
client = OpenAI(
    base_url="http://localhost:4000",
    api_key="your-litellm-api-key"
)

# Transcribe audio file
with open("audio.mp3", "rb") as audio_file:
    response = client.audio.transcriptions.create(
        model="elevenlabs-transcription",
        file=audio_file,
        language="en",
        temperature=0.3,  # ElevenLabs-specific parameters
        diarize=True,
        speaker_boost=True,
        custom_vocabulary="technical,AI,machine learning"
    )
print(response.text)

JavaScript/Node.js:

import OpenAI from 'openai';
import fs from 'fs';

const openai = new OpenAI({
    baseURL: 'http://localhost:4000',
    apiKey: 'your-...'
});

async function transcribeAudio() {
    const response = await openai.audio.transcriptions.create({
        file: fs.createReadStream('audio.mp3'),
        model: 'elevenlabs-transcription',
        language: 'en',
        temperature: 0.3,
        diarize: true,
        speaker_boost: true
    });
    console.log(response.text);
}
transcribeAudio();

응답 형식

ElevenLabs는 OpenAI 호환 형식으로 전사 응답을 반환해요.

{
  "text": "Hello, this is a sample transcription with multiple speakers.",
  "task": "transcribe",
  "language": "en",
  "words": [
    {
      "word": "Hello",
      "start": 0.0,
      "end": 0.5
    },
    {
      "word": "this",
      "start": 0.5,
      "end": 0.8
    }
  ]
}

일반적인 문제

  • 잘못된 API 키: ELEVENLABS_API_KEY가 올바르게 설정되었는지 확인

Text-to-Speech (TTS)

ElevenLabs는 여러 음성, 언어, 오디오 형식을 지원하는 고품질 TTS 기능을 제공해요.

개요

속성 설명
설명 ElevenLabs의 고급 TTS 모델로 텍스트를 자연스러운 음성으로 변환
LiteLLM 라우트 elevenlabs/
지원 작업 /audio/speech
공급자 문서 ElevenLabs TTS API

지원 모델

모델 라우트 설명
Eleven v3 elevenlabs/eleven_v3 가장 표현력이 좋은 모델. 70+ 언어, 음향 효과와 일시정지용 오디오 태그 지원
Eleven Multilingual v2 elevenlabs/eleven_multilingual_v2 기본 TTS 모델. 29개 언어, 안정적이고 프로덕션 준비

빠른 시작

import litellm
import os

os.environ["ELEVENLABS_API_KEY"] = "your-elevenlabs-api-key"

# Basic usage with voice mapping
audio = litellm.speech(
    model="elevenlabs/eleven_multilingual_v2",
    input="Testing ElevenLabs speech from LiteLLM.",
    voice="alloy",  # Maps to ElevenLabs voice ID automatically
)

# Save audio to file
with open("test_output.mp3", "wb") as f:
    f.write(audio.read())

Eleven v3 오디오 태그 사용

Eleven v3는 텍스트에 음향 효과와 일시정지를 추가하는 오디오 태그를 지원해요.

import litellm
import os

os.environ["ELEVENLABS_API_KEY"] = "your-elevenlabs-api-key"

audio = litellm.speech(
    model="elevenlabs/eleven_v3",
    input='Welcome back. <sfx>applause</sfx> Today we have a special guest. <pause duration="1.5s"/> Let me introduce them.',
    voice="alloy",
)

with open("eleven_v3_output.mp3", "wb") as f:
    f.write(audio.read())

고급 사용법: 파라미터 오버라이드와 ElevenLabs 전용 기능

import litellm
import os

os.environ["ELEVENLABS_API_KEY"] = "your-elevenlabs-api-key"

# Example showing parameter overriding and ElevenLabs-specific parameters
audio = litellm.speech(
    model="elevenlabs/eleven_multilingual_v2",
    input="Testing ElevenLabs speech from LiteLLM.",
    voice="alloy",  # Can use mapped voice name or raw ElevenLabs voice_id
    response_format="pcm",  # Maps to ElevenLabs output_format
    speed=1.1,  # Maps to voice_settings.speed
    # ElevenLabs-specific parameters - passed directly to API
    pronunciation_dictionary_locators=[
        {"pronunciation_dictionary_id": "dict_123", "version_id": "v1"}
    ],
    model_id="eleven_multilingual_v2",  # Override model if needed
)

# Save audio to file
with open("test_output.mp3", "wb") as f:
    f.write(audio.read())

음성 매핑 (Voice Mapping)

LiteLLM은 일반 OpenAI 음성 이름을 ElevenLabs 음성 ID로 자동 매핑해요:

OpenAI 음성 ElevenLabs 음성 ID 설명
alloy 21m00Tcm4TlvDq8ikWAM Rachel - 중립적이고 균형 잡힘
amber 5Q0t7uMcjvnagumLfvZi Paul - 따뜻하고 친근함
ash AZnzlk1XvdvUeBnXmlld Domi - 활기참
august D38z5RcWu1voky8WS1ja Fin - 프로페셔널
blue 2EiwWnXFnvU5JabPnv8n Clyde - 깊고 권위 있음
coral 9BWtsMINqrJLrRacOk9x Aria - 표현력 있음
lily EXAVITQu4vr4xnSDxMaLS Sarah - 친근함
onyx 29vD33N1CtxCmqQRPOHJ Drew - 강함
sage CwhRBWXzGAHq8TQ4Fs17 Roger - 차분함
verse CYw3kZ02Hs0563khs1Fj Dave - 대화적

사용자 지정 음성 ID를 직접 전달할 수도 있어요. 매핑에 없는 음성 이름이면 LiteLLM이 그대로 사용해요.

audio = litellm.speech(
    model="elevenlabs/eleven_multilingual_v2",
    input="Testing with a custom voice.",
    voice="21m00Tcm4TlvDq8ikWAM",  # Direct ElevenLabs voice ID
)

응답 형식 매핑

LiteLLM은 OpenAI 응답 형식을 ElevenLabs 출력 형식으로 매핑해요:

OpenAI 형식 ElevenLabs 형식
mp3 mp3_44100_128
pcm pcm_44100
opus opus_48000_128

output_format 파라미터로 ElevenLabs 전용 출력 형식을 직접 전달할 수도 있어요.

지원 파라미터

audio = litellm.speech(
    model="elevenlabs/eleven_multilingual_v2",  # Required
    input="Text to convert to speech",  # Required
    voice="alloy",  # Required: Voice selection (mapped or raw ID)
    response_format="mp3",  # Optional: Audio format (mp3, pcm, opus)
    speed=1.0,  # Optional: Speech speed (maps to voice_settings.speed)
    # ElevenLabs-specific parameters (passed directly):
    model_id="eleven_multilingual_v2",  # Optional: Override model
    voice_settings={  # Optional: Voice customization
        "stability": 0.5,
        "similarity_boost": 0.75,
        "speed": 1.0
    },
    pronunciation_dictionary_locators=[  # Optional: Custom pronunciation
        {"pronunciation_dictionary_id": "dict_123", "version_id": "v1"}
    ],
)

LiteLLM Proxy

config.yaml:

model_list:
  - model_name: elevenlabs-tts
    litellm_params:
      model: elevenlabs/eleven_multilingual_v2
      api_key: os.environ/ELEVENLABS_API_KEY

general_settings:
  master_key: your-master-key

간단 사용법 (OpenAI 파라미터):

curl http://localhost:4000/v1/audio/speech \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "elevenlabs-tts",
    "input": "Testing ElevenLabs speech via the LiteLLM proxy.",
    "voice": "alloy",
    "response_format": "mp3"
  }' \
  --output speech.mp3

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:4000",
    api_key="your-litellm-api-key"
)

response = client.audio.speech.create(
    model="elevenlabs-tts",
    input="Testing ElevenLabs speech via the LiteLLM proxy.",
    voice="alloy",
    response_format="mp3",
)

# Save audio
with open("speech.mp3", "wb") as f:
    f.write(response.content)

ElevenLabs 전용 파라미터 (extra_body 사용):

curl http://localhost:4000/v1/audio/speech \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "elevenlabs-tts",
    "input": "Testing ElevenLabs speech via the LiteLLM proxy.",
    "voice": "alloy",
    "response_format": "pcm",
    "extra_body": {
      "pronunciation_dictionary_locators": [
        {"pronunciation_dictionary_id": "dict_123", "version_id": "v1"}
      ],
      "voice_settings": {
        "speed": 1.1,
        "stability": 0.5,
        "similarity_boost": 0.75
      }
    }
  }' \
  --output speech.mp3

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:4000",
    api_key="your-litellm-api-key"
)

response = client.audio.speech.create(
    model="elevenlabs-tts",
    input="Testing ElevenLabs speech via the LiteLLM proxy.",
    voice="alloy",
    response_format="pcm",
    extra_body={
        "pronunciation_dictionary_locators": [
            {"pronunciation_dictionary_id": "dict_123", "version_id": "v1"}
        ],
        "voice_settings": {
            "speed": 1.1,
            "stability": 0.5,
            "similarity_boost": 0.75
        }
    }
)

# Save audio
with open("speech.mp3", "wb") as f:
    f.write(response.content)

더 알아보기 (Learn more)