LLaDA 2.1
LLaDA 2.1
이 문서는 Ant Group의 InclusionAI 팀이 개발한 대규모 이산 확산 언어 모델(dLLM) 시리즈인 LLaDA 2.1을 SGLang으로 배포하고 호출하는 방법을 설명해요. 확산 기반 접근으로 병렬 토큰 드래프팅과 반복적 디노이징을 통해 생성 중 자기 수정이 가능해요. 원문 페이지에는 하드웨어 플랫폼과 모델 크기를 골라 명령을 자동 생성해 주는 대화형 선택기가 포함되어 있어요.
출처: 문서
본문
1. 모델 소개
LLaDA 2.1은 Ant Group의 InclusionAI 팀이 개발한 일련의 대규모 이산 확산 언어 모델(dLLM)이에요. 텍스트를 왼쪽에서 오른쪽으로 한 토큰씩 생성하는 기존의 자기회귀(autoregressive) 모델과 달리, LLaDA 2.1은 확산 기반 접근 방식을 사용해요 — 토큰을 병렬로 드래프팅하고 반복적인 디노이징을 통해 정제하면서 생성 중 자기 수정이 가능해요.
주요 특징:
- 토큰 편집 (Token Editing, T2T + M2T): Mask-to-Token(M2T)과 Token-to-Token(T2T) 편집을 결합해, 토큰을 언마스킹할 뿐만 아니라 이미 생성된 토큰을 생성 도중에 수정할 수 있어요
- 이중 디코딩 모드 (Dual Decoding Modes): 최대 처리량을 위한 T2T 정제가 적용된 속도 모드(Speed Mode, S)와, 보수적인 임계값과 더 높은 벤치마크 점수를 위한 품질 모드(Quality Mode, Q)
- MoE 아키텍처 (MoE Architecture): 두 변형 모두 효율적인 확장을 위해 Mixture-of-Experts 아키텍처를 사용해요
- dLLM 최초의 대규모 RL (First Large-Scale RL for dLLMs): 확산 언어 모델을 위해 특별히 설계된 최초의 강화학습 프레임워크를 구현해 추론과 지시 따르기 능력을 개선해요
- 번개처럼 빠른 디코딩 (Lightning-Fast Decoding): 100B 모델의 HumanEval+에서 최대 892 tokens/s
사용 가능한 모델:
| 모델 | 파라미터 | 아키텍처 | 컨텍스트 길이 | HuggingFace |
|---|---|---|---|---|
| LLaDA2.1-mini | 16B | MoE (20 layers, 16 attention heads) | 32,768 tokens | inclusionAI/LLaDA2.1-mini |
| LLaDA2.1-flash | 100B | MoE | 32,768 tokens | inclusionAI/LLaDA2.1-flash |
라이선스:
Apache 2.0. 자세한 내용은 공식 LLaDA2.X 저장소를 참고하세요.
2. SGLang 설치
SGLang은 여러 설치 방법을 제공해요. 하드웨어 플랫폼과 요구 사항에 가장 적합한 설치 방법을 선택할 수 있어요.
설치 지침은 공식 SGLang 설치 가이드를 참고하세요.
3. 모델 배포
이 섹션은 서로 다른 하드웨어 플랫폼과 사용 사례에 최적화된 배포 설정을 제공해요.
3.1 기본 설정
대화형 명령 생성기 (Interactive Command Generator): 아래 설정 선택기를 사용해 하드웨어 플랫폼, 모델 크기, 디코딩 모드에 맞는 배포 명령을 자동으로 생성할 수 있어요. SGLang은 NVIDIA H100, H200, B200과 AMD MI300X, MI325X, MI355X GPU에서 LLaDA-2.1 서빙을 지원해요. (대화형 위젯은 원문 페이지에서 동작하므로, 기본 설정 기준 명령은 다음과 같아요.)
python -m sglang.launch_server \
--model-path inclusionAI/LLaDA2.1-mini \
--dllm-algorithm JointThreshold \
--tp 1 \
--trust-remote-code \
--mem-fraction-static 0.8 \
--max-running-requests 1 \
--attention-backend flashinfer
3.2 설정 팁
dLLM 전용 파라미터:
| 파라미터 | 설명 | 권장 값 |
|---|---|---|
--dllm-algorithm |
확산 디코딩 알고리즘 | JointThreshold |
--trust-remote-code |
LLaDA 모델 로딩에 필요 | 항상 활성화 |
--mem-fraction-static |
KV cache용 정적 메모리 비율 | 0.8 |
--max-running-requests |
최대 동시 요청 수 | 1 (최상의 품질) |
--attention-backend |
어텐션 계산 백엔드 | flashinfer |
디코딩 모드 비교:
| 모드 | 임계값 | 속도 | 품질 | 최적 용도 |
|---|---|---|---|---|
| 품질 모드 (Q) | 보수적(Conservative) | 보통(Moderate) | 더 높은 벤치마크 점수 | 정확도가 중요한 작업 |
| 속도 모드 (S) | 공격적(Aggressive) | 매우 빠름, T2T 편집에 의존 | 약간 낮음 | 처리량이 중요한 작업 |
하드웨어 요구 사항:
- LLaDA2.1-mini (16B): 약 47 GB VRAM, 단일 GPU에서 실행 (TP=1)
- LLaDA2.1-flash (100B): 다중 GPU 설정 필요 (H100/H200에서 TP=4, B200에서 TP=2)
4. 모델 호출
4.1 배포
위에서 생성한 명령으로 서버를 시작하세요, 예시:
python -m sglang.launch_server \
--model-path inclusionAI/LLaDA2.1-mini \
--dllm-algorithm JointThreshold \
--tp 1 \
--trust-remote-code \
--mem-fraction-static 0.8 \
--max-running-requests 1 \
--attention-backend flashinfer \
--host 0.0.0.0 \
--port 8000
4.2 기본 사용법
기본 API 사용법과 요청 예시는 다음을 참고하세요:
간단한 완성 예시:
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8000/v1",
api_key="EMPTY"
)
response = client.chat.completions.create(
model="inclusionAI/LLaDA2.1-mini",
messages=[
{"role": "user", "content": "Explain what a diffusion language model is in simple terms."}
],
max_tokens=1024
)
print(response.choices[0].message.content)
출력 예시:
Sure! Let's break it down in simple terms.
A **diffusion language model** is a type of artificial intelligence that learns to generate text—like sentences, stories, or emails—by studying a lot of written text.
Here's how it works, using a simple real-life analogy:
Imagine you have a big book full of stories. A diffusion language model is trying to learn how to write a new story. Instead of being told the rules, it starts by looking at all the words in the book and trying to understand how words usually go together.
Now, think of the process like this:
1. **Start with random noise**: The model begins with a completely random set of words (like a scribble on paper).
2. ** ** "clean up" the noise**: It gradually "denoises" the noise by turning it into meaningful text, word by word, based on what it learned learned from the book.
3. **Learn from patterns**: As it does this, it learns patterns—like how words often follow each other, or how sentences start.
4. **Generate new text**: Once it's learned the patterns, it can create new, coherent sentences or stories by starting from a and and building it up word by word.
So, the "diffusion" part comes from the idea of going from random noise to clear, meaningful text—like turning a scribble into a full story.
In short:
A diffusion language model is an AI that learns to write text by reading lots of books and gradually turning random noise into coherent, meaningful sentences based on what it learned.
4.3 고급 사용법
4.3.1 스트리밍
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8000/v1",
api_key="EMPTY"
)
response = client.chat.completions.create(
model="inclusionAI/LLaDA2.1-mini",
messages=[
{"role": "user", "content": "Write a Python function to compute the Fibonacci sequence."}
],
max_tokens=2048,
stream=True
)
for chunk in response:
if chunk.choices and len(chunk.choices) > 0:
delta = chunk.choices[0].delta
if delta.content:
print(delta.content, end="", flush=True)
print()
출력 예시:
Here are several ways to implement the Fibonacci sequence in Python:
## 1. Recursive Approach (Simple but Inefficient)
```python
def fibonacci_recursive(n):
"""
Compute the nth Fibonacci number using recursion.
Args:
n (int): The position in the Fibonacci sequence (0-indexed)
Returns:
int: The nth Fibonacci number
Raises:
ValueError: If n is negative
"""
if n < 0:
raise ValueError("n must be non-negative")
if n <= 1:
return n
return fibonacci_recursive(n - 1) + fibonacci_recursive(n - 2)
# Example usage
print(fibonacci_recursive(10)) # Output: 55
```
## 2. Iterative Approach (Efficient)
...
4.3.2 코드 생성
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8000/v1",
api_key="EMPTY"
)
response = client.chat.completions.create(
model="inclusionAI/LLaDA2.1-mini",
messages=[
{"role": "user", "content": "Write a Python function that checks if a string is a palindrome. Include docstring and test cases."}
],
max_tokens=2048
)
print(response.choices[0].message.content)
출력 예시:
```python
def is_palindrome(s):
"""
Check if a string is a palindrome.
A palindrome is a word, phrase, or sequence that reads the same backward as forward.
This function ignores case, spaces, punctuation, and non characters characters.
Args:
s (str): The string to check
Returns:
bool: True if the string is a palindrome, False otherwise
Examples:
>>> is_palindrome("racecar")
True
>>> is_palindrome("A man a plan a canal Panama")
True
>>> is_palindrome("race a car")
False
>>> is_palindrome("")
True
>>> is_palindrome("a")
True
"""
# Remove non-alphanumeric characters and convert to lowercase
cleaned = ''.join(char.lower() for char in s if char.isalnum())
# Check if the cleaned string reads the same forwards and backwards
return cleaned == cleaned[::-1]
# Test cases
def test_is_palindrome():
"""Test the is_palindrome function with various inputs."""
# Test basic palindromes
assert is_palindrome("racecar") == True
assert is_palindrome("level") == True
assert is_palindrome("madam") == True
assert is_palindrome("radar") == True
# Test palindromes with spaces and punctuation
assert is_palindrome("A man a plan a canal Panama") == True
assert is_palindrome("race a car") == False
assert is_palindrome("Was it a car or a cat I saw?") == True
assert is_palindrome("Madam, I'm Adam") == True
# Test edge cases
assert is_palindrome("") == True
assert is_palindrome("a") == True
assert is_palindrome("A") == True
assert is_palindrome("Aa") == True
# Test non-palindromes
assert is_palindrome("hello") == False
assert is_palindrome("world") == False
assert is_palindrome("python") == False
# Test single characters
assert is_palindrome("1") == True
assert is_palindrome("1") == True
print("All tests passed!")
# Run the tests
if __name__ == "__main__":
# Example usage
print("Testing isalindrome function:")
print(f"'racecar' {is_palindrome('racecar')}")
print(f"'A man a plan a canal Panama': {is_palindrome('A man a plan a canal Panama')}")
print(f"'race a car': {is_palindrome('race a car')}")
print(f"'hello': {is_palindrome('hello')}")
# Run tests
test_is_palindrome()
```
This implementation includes:
1. **Comprehensive function** `is_palindrome()` that:
- Ignores case by converting to lowercase
- Removes all non-alphanumeric characters (spaces, punctuation, etc.)
- Uses string slicing (`[::-1]`) to reverse the string
2. **Detailed docstring** explaining:
- What the function does
- How it works
- Return value
- Examples of usage
3. **Extensive test cases** covering:
- Basic palindromes
- Palindromes with spaces and punctuation
- Edge cases (empty string, single character)
- Non-palindromes
- Mixed case scenarios
4. **Test function** that uses assertions to verify the function works correctly
The function efficiently handles real-world palindrome checking by ignoring case, spaces, and punctuation, making it suitable for phrases like "A man a plan a canal Panama".
5. 벤치마크
이 섹션은 비교 가능한 벤치마크 결과를 위해 산업 표준 설정을 사용해요.
5.1 속도 벤치마크
테스트 환경:
- 하드웨어: NVIDIA B200 (4x)
- SGLang 버전: 0.5.8+
5.1.1 LLaDA2.1-mini
모델 배포:
python -m sglang.launch_server \
--model-path inclusionAI/LLaDA2.1-mini \
--dllm-algorithm JointThreshold \
--tp 1 \
--trust-remote-code \
--mem-fraction-static 0.8 \
--max-running-requests 1 \
--attention-backend flashinfer
- 지연 시간 벤치마크
python -m sglang.bench_serving \
--backend sglang \
--model inclusionAI/LLaDA2.1-mini \
--dataset-name random \
--random-input-len 1000 \
--random-output-len 1000 \
--num-prompts 10 \
--max-concurrency 1 \
--request-rate inf
- 지연 시간 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 1
Successful requests: 10
Benchmark duration (s): 9.90
Total input tokens: 6101
Total input text tokens: 6101
Total generated tokens: 4220
Total generated tokens (retokenized): 3433
Request throughput (req/s): 1.01
Input token throughput (tok/s): 616.26
Output token throughput (tok/s): 426.26
Peak output token throughput (tok/s): 1010.00
Peak concurrent requests: 3
Total token throughput (tok/s): 1042.53
Concurrency: 1.00
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 988.87
Median E2E Latency (ms): 655.27
P90 E2E Latency (ms): 1952.50
P99 E2E Latency (ms): 2932.19
---------------Time to First Token----------------
Mean TTFT (ms): 152.74
Median TTFT (ms): 150.37
P99 TTFT (ms): 229.78
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 2.16
Median TPOT (ms): 2.08
P99 TPOT (ms): 3.72
---------------Inter-Token Latency----------------
Mean ITL (ms): 2.10
Median ITL (ms): 1.99
P95 ITL (ms): 4.03
P99 ITL (ms): 6.34
Max ITL (ms): 26.59
==================================================
- 처리량 벤치마크
python -m sglang.bench_serving \
--backend sglang \
--model inclusionAI/LLaDA2.1-mini \
--dataset-name random \
--random-input-len 1000 \
--random-output-len 1000 \
--num-prompts 500 \
--max-concurrency 100 \
--request-rate inf
- 처리량 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 100
Successful requests: 500
Benchmark duration (s): 467.74
Total input tokens: 249831
Total input text tokens: 249831
Total generated tokens: 252662
Total generated tokens (retokenized): 189717
Request throughput (req/s): 1.07
Input token throughput (tok/s): 534.12
Output token throughput (tok/s): 540.17
Peak output token throughput (tok/s): 1753.00
Peak concurrent requests: 105
Total token throughput (tok/s): 1074.30
Concurrency: 90.77
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 84912.27
Median E2E Latency (ms): 86564.26
P90 E2E Latency (ms): 110567.26
P99 E2E Latency (ms): 114303.38
---------------Time to First Token----------------
Mean TTFT (ms): 83920.39
Median TTFT (ms): 85669.54
P99 TTFT (ms): 112969.91
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 2.67
Median TPOT (ms): 1.65
P99 TPOT (ms): 4.43
---------------Inter-Token Latency----------------
Mean ITL (ms): 1.69
Median ITL (ms): 1.46
P95 ITL (ms): 3.96
P99 ITL (ms): 4.84
Max ITL (ms): 92.08
==================================================
5.1.2 LLaDA2.1-flash
모델 배포:
python -m sglang.launch_server \
--model-path inclusionAI/LLaDA2.1-flash \
--dllm-algorithm JointThreshold \
--tp 4 \
--trust-remote-code \
--mem-fraction-static 0.8 \
--max-running-requests 1 \
--attention-backend flashinfer
- 지연 시간 벤치마크
python -m sglang.bench_serving \
--backend sglang \
--model inclusionAI/LLaDA2.1-flash \
--dataset-name random \
--random-input-len 1000 \
--random-output-len 1000 \
--num-prompts 10 \
--max-concurrency 1 \
--request-rate inf
- 지연 시간 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 1
Successful requests: 10
Benchmark duration (s): 14.46
Total input tokens: 6101
Total input text tokens: 6101
Total generated tokens: 4220
Total generated tokens (retokenized): 3276
Request throughput (req/s): 0.69
Input token throughput (tok/s): 421.79
Output token throughput (tok/s): 291.75
Peak output token throughput (tok/s): 676.00
Peak concurrent requests: 3
Total token throughput (tok/s): 713.53
Concurrency: 1.00
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 1445.16
Median E2E Latency (ms): 968.06
P90 E2E Latency (ms): 3101.86
P99 E2E Latency (ms): 4208.49
---------------Time to First Token----------------
Mean TTFT (ms): 231.63
Median TTFT (ms): 242.67
P99 TTFT (ms): 341.33
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 3.04
Median TPOT (ms): 2.79
P99 TPOT (ms): 5.33
---------------Inter-Token Latency----------------
Mean ITL (ms): 3.05
Median ITL (ms): 2.41
P95 ITL (ms): 7.25
P99 ITL (ms): 8.27
Max ITL (ms): 29.27
==================================================
- 처리량 벤치마크
python -m sglang.bench_serving \
--backend sglang \
--model inclusionAI/LLaDA2.1-flash \
--dataset-name random \
--random-input-len 1000 \
--random-output-len 1000 \
--num-prompts 500 \
--max-concurrency 100 \
--request-rate inf
- 처리량 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 100
Successful requests: 500
Benchmark duration (s): 671.85
Total input tokens: 249831
Total input text tokens: 249831
Total generated tokens: 252662
Total generated tokens (retokenized): 177961
Request throughput (req/s): 0.74
Input token throughput (tok/s): 371.85
Output token throughput (tok/s): 376.07
Peak output token throughput (tok/s): 1521.00
Peak concurrent requests: 103
Total token throughput (tok/s): 747.92
Concurrency: 91.28
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 122658.36
Median E2E Latency (ms): 125265.55
P90 E2E Latency (ms): 159554.07
P99 E2E Latency (ms): 165174.88
---------------Time to First Token----------------
Mean TTFT (ms): 121009.17
Median TTFT (ms): 124437.80
P99 TTFT (ms): 163579.29
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 4.73
Median TPOT (ms): 2.16
P99 TPOT (ms): 7.13
---------------Inter-Token Latency----------------
Mean ITL (ms): 2.38
Median ITL (ms): 1.40
P95 ITL (ms): 6.89
P99 ITL (ms): 8.60
Max ITL (ms): 176.78
==================================================
5.2 정확도 벤치마크
5.2.1 GSM8K 벤치마크
python -m sglang.test.few_shot_gsm8k \
--num-questions 200 \
--port 8000
결과:
Accuracy: 0.895
Invalid: 0.000
Latency: 100.552 s
Output throughput: 262.094 token/s