Glyph
Glyph
Glyph은 Zhipu AI가 개발한 강력한 언어 모델로, 추론·함수 호출·멀티모달 이해에서 뛰어난 능력을 갖추고 있어요. 복잡한 문제 해결을 위한 내장 추론 능력, BF16·FP8 양자화 변형, 처리량·지연 시간 양쪽에 최적화된 고성능을 제공하며 NVIDIA B200/H100/H200과 AMD MI300X/MI325X/MI355X GPU를 지원해요.
이 문서는 SGLang으로 Glyph를 배포하고, thinking 모드와 함수 호출 같은 고급 기능, 벤치마크 실행 방법을 설명해요.
원문 페이지에는 하드웨어 플랫폼·양자화 방법·기타 옵션을 골라 배포 명령을 자동 생성하는 대화형 선택기가 포함되어 있어요. 위키에서는 렌더링되지 않으니 아래 섹션의 명령어를 직접 참고하면 돼요.
출처: 문서
본문
1. 모델 소개
Glyph은 Zhipu AI가 개발한 강력한 언어 모델로, 추론·함수 호출·멀티모달 이해에서 뛰어난 능력을 갖추고 있어요.
하드웨어 지원: NVIDIA B200/H100/H200, AMD MI300X/MI325X/MI355X
주요 특징:
- 고급 추론: 복잡한 문제 해결을 위한 내장 추론 능력
- 여러 양자화 형식: 서로 다른 성능/메모리 트레이드오프를 위한 BF16, FP8 변형 제공
- 고성능: 처리량과 지연 시간 시나리오 모두에 최적화
제공 모델:
- BF16 (풀 프리시전): zai-org/Glyph
- FP8 (8-bit 양자화): zai-org/Glyph-FP8
라이선스:
라이선스 세부사항은 공식 Glyph 모델 카드를 참고하세요.
2. SGLang 설치
SGLang은 여러 설치 방법을 제공해요. 하드웨어 플랫폼과 요구사항에 맞는 설치 방법을 선택하면 돼요.
설치 안내는 공식 SGLang 설치 가이드를 참고하세요.
3. 모델 배포
이 섹션에서는 하드웨어 플랫폼과 사용 사례에 맞게 최적화된 배포 구성을 제공해요.
3.1 기본 구성
대화형 명령 생성기: 아래 구성 선택기를 사용하면 하드웨어 플랫폼·양자화 방법·기타 옵션에 맞는 배포 명령을 자동으로 생성할 수 있어요.
3.2 구성 팁
- Thinking Budget:
--enable-custom-logit-processor플래그를 사용하고 요청에서Glm4MoeThinkingBudgetLogitProcessor를 전달해 모델의 thinking 토큰 수를 제한할 수 있어요. 전체 Thinking Budget 사용 예시는 GLM-4.5 cookbook 페이지를 참고하세요.
4. 모델 호출
4.1 기본 사용법
기본 API 사용법과 요청 예시는 아래를 참고하세요:
4.2 고급 사용법
4.2.1 Thinking 모드
Glyph는 향상된 추론을 위한 thinking 모드를 지원해요. 배포 시 추론 파서를 활성화하면 thinking 영역과 content 영역을 분리할 수 있어요:
python -m sglang.launch_server \
--model-path zai-org/Glyph \
--reasoning-parser glm45 \
--tp 4
Thinking 과정과 함께 스트리밍하기:
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:30000/v1",
api_key="EMPTY"
)
# Enable streaming to see the thinking process in real-time
response = client.chat.completions.create(
model="zai-org/Glyph",
messages=[
{"role": "user", "content": "Solve this problem step by step: What is 15% of 240?"}
],
temperature=0.7,
max_tokens=2048,
stream=True
)
# Process the stream
has_thinking = False
has_answer = False
thinking_started = False
for chunk in response:
if chunk.choices and len(chunk.choices) > 0:
delta = chunk.choices[0].delta
# Print thinking process
if hasattr(delta, 'reasoning_content') and delta.reasoning_content:
if not thinking_started:
print("=============== Thinking ==================", flush=True)
thinking_started = True
has_thinking = True
print(delta.reasoning_content, end="", flush=True)
# Print answer content
if delta.content:
# Close thinking section and add content header
if has_thinking and not has_answer:
print("\n=============== Content ==================", flush=True)
has_answer = True
print(delta.content, end="", flush=True)
print()
참고: 추론 파서는 모델의 단계별 사고 과정을 캡처해서, 모델이 결론에 도달하는 방식을 볼 수 있게 해줘요.
Thinking 모드 끄기:
특정 요청에 대해 thinking 모드를 끄려면:
response = client.chat.completions.create(
model="zai-org/Glyph",
messages=[{"role": "user", "content": "What is the capital of France?"}],
extra_body={"chat_template_kwargs": {"enable_thinking": False}}
)
4.2.2 함수 호출(Tool Calling)
Glyph는 함수 호출 기능을 지원해요. 함수 호출 파서를 활성화하세요:
python -m sglang.launch_server \
--model-path zai-org/Glyph \
--reasoning-parser glm45 \
--tool-call-parser glm45 \
--tp 4
Python 예시 (Thinking 과정 포함):
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:30000/v1",
api_key="EMPTY"
)
# Define available tools
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city name"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "Temperature unit"
}
},
"required": ["location"]
}
}
}
]
# Make request with streaming to see thinking process
response = client.chat.completions.create(
model="zai-org/Glyph",
messages=[
{"role": "user", "content": "What's the weather in Beijing?"}
],
tools=tools,
temperature=0.7,
stream=True
)
# Process streaming response
thinking_started = False
has_thinking = False
tool_calls_accumulator = {}
for chunk in response:
if chunk.choices and len(chunk.choices) > 0:
delta = chunk.choices[0].delta
# Print thinking process
if hasattr(delta, 'reasoning_content') and delta.reasoning_content:
if not thinking_started:
print("=============== Thinking =================", flush=True)
thinking_started = True
has_thinking = True
print(delta.reasoning_content, end="", flush=True)
# Accumulate tool calls
if hasattr(delta, 'tool_calls') and delta.tool_calls:
# Close thinking section if needed
if has_thinking and thinking_started:
print("\n=============== Content =================\n", flush=True)
thinking_started = False
for tool_call in delta.tool_calls:
index = tool_call.index
if index not in tool_calls_accumulator:
tool_calls_accumulator[index] = {
'name': None,
'arguments': ''
}
if tool_call.function:
if tool_call.function.name:
tool_calls_accumulator[index]['name'] = tool_call.function.name
if tool_call.function.arguments:
tool_calls_accumulator[index]['arguments'] += tool_call.function.arguments
# Print content
if delta.content:
print(delta.content, end="", flush=True)
# Print accumulated tool calls
for index, tool_call in sorted(tool_calls_accumulator.items()):
print(f"Tool Call: {tool_call['name']}")
print(f" Arguments: {tool_call['arguments']}")
print()
출력 예시:
=============== Thinking =================
The user is asking about the weather in Beijing. I need to use the get_weather function to retrieve this information.
I should call the function with location="Beijing".
=============== Content =================
Tool Call: get_weather
Arguments: {"location": "Beijing", "unit": "celsius"}
참고:
- 추론 파서는 모델이 어떻게 도구를 사용할지 결정하는지를 보여줘요
- 함수 호출은 함수 이름과 인자로 명확하게 표시돼요
- 이후 함수를 실행하고 결과를 다시 보내 대화를 이어갈 수 있어요
함수 호출 결과 처리하기:
# After getting the tool call, execute the function
def get_weather(location, unit="celsius"):
# Your actual weather API call here
return f"The weather in {location} is 22°{unit[0].upper()} and sunny."
# Send tool result back to the model
messages = [
{"role": "user", "content": "What's the weather in Beijing?"},
{
"role": "assistant",
"content": None,
"tool_calls": [{
"id": "call_123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": '{"location": "Beijing", "unit": "celsius"}'
}
}]
},
{
"role": "tool",
"tool_call_id": "call_123",
"content": get_weather("Beijing", "celsius")
}
]
final_response = client.chat.completions.create(
model="zai-org/Glyph",
messages=messages,
temperature=0.7
)
print(final_response.choices[0].message.content)
# Output: "The weather in Beijing is currently 22°C and sunny."
5. 벤치마크
이 섹션은 비교 가능한 벤치마크 결과를 위해 업계 표준 구성을 사용해요.
5.1 속도 벤치마크
테스트 환경:
- 하드웨어: NVIDIA B200 (1x)
- 모델: Glyph
- SGLang 버전: 0.5.6.post1
벤치마크 방법론:
프레임워크와 하드웨어 플랫폼 간 결과를 비교할 수 있도록 업계 표준 벤치마크 구성을 사용해요.
5.1.1 표준 시나리오 벤치마크
- 모델 배포
python -m sglang.launch_server \
--model zai-org/Glyph \
--tp 2
5.1.1.1 낮은 동시성
- 벤치마크 명령:
python -m sglang.bench_serving \
--backend sglang \
--model zai-org/Glyph \
--dataset-name random \
--random-input-len 1000 \
--random-output-len 1000 \
--num-prompts 10 \
--max-concurrency 1 \
--request-rate inf
- 테스트 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 1
Successful requests: 10
Benchmark duration (s): 17.03
Total input tokens: 6101
Total input text tokens: 6101
Total input vision tokens: 0
Total generated tokens: 4220
Total generated tokens (retokenized): 4220
Request throughput (req/s): 0.59
Input token throughput (tok/s): 358.17
Output token throughput (tok/s): 247.74
Peak output token throughput (tok/s): 251.00
Peak concurrent requests: 3
Total token throughput (tok/s): 605.91
Concurrency: 1.00
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 1702.14
Median E2E Latency (ms): 1361.72
---------------Time to First Token----------------
Mean TTFT (ms): 22.35
Median TTFT (ms): 22.61
P99 TTFT (ms): 23.76
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 3.99
Median TPOT (ms): 3.99
P99 TPOT (ms): 4.01
---------------Inter-Token Latency----------------
Mean ITL (ms): 3.99
Median ITL (ms): 3.99
P95 ITL (ms): 4.03
P99 ITL (ms): 4.12
Max ITL (ms): 7.46
==================================================
5.1.1.2 중간 동시성
- 벤치마크 명령:
python -m sglang.bench_serving \
--backend sglang \
--model zai-org/Glyph \
--dataset-name random \
--random-input-len 1000 \
--random-output-len 1000 \
--num-prompts 80 \
--max-concurrency 16 \
--request-rate inf
- 테스트 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 16
Successful requests: 80
Benchmark duration (s): 16.27
Total input tokens: 39668
Total input text tokens: 39668
Total input vision tokens: 0
Total generated tokens: 40805
Total generated tokens (retokenized): 40804
Request throughput (req/s): 4.92
Input token throughput (tok/s): 2438.06
Output token throughput (tok/s): 2507.94
Peak output token throughput (tok/s): 3069.00
Peak concurrent requests: 26
Total token throughput (tok/s): 4946.00
Concurrency: 13.44
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 2733.43
Median E2E Latency (ms): 2892.98
---------------Time to First Token----------------
Mean TTFT (ms): 33.10
Median TTFT (ms): 27.73
P99 TTFT (ms): 49.34
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 5.33
Median TPOT (ms): 5.39
P99 TPOT (ms): 5.86
---------------Inter-Token Latency----------------
Mean ITL (ms): 5.30
Median ITL (ms): 4.89
P95 ITL (ms): 5.54
P99 ITL (ms): 21.17
Max ITL (ms): 25.14
==================================================
5.1.1.3 높은 동시성
- 벤치마크 명령:
python -m sglang.bench_serving \
--backend sglang \
--model zai-org/Glyph \
--dataset-name random \
--random-input-len 1000 \
--random-output-len 1000 \
--num-prompts 500 \
--max-concurrency 100 \
--request-rate inf
- 테스트 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 100
Successful requests: 500
Benchmark duration (s): 25.67
Total input tokens: 249831
Total input text tokens: 249831
Total input vision tokens: 0
Total generated tokens: 252662
Total generated tokens (retokenized): 252657
Request throughput (req/s): 19.48
Input token throughput (tok/s): 9733.69
Output token throughput (tok/s): 9843.99
Peak output token throughput (tok/s): 13398.00
Peak concurrent requests: 127
Total token throughput (tok/s): 19577.68
Concurrency: 89.49
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 4593.75
Median E2E Latency (ms): 4431.03
---------------Time to First Token----------------
Mean TTFT (ms): 48.66
Median TTFT (ms): 35.88
P99 TTFT (ms): 120.61
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 9.10
Median TPOT (ms): 9.55
P99 TPOT (ms): 11.00
---------------Inter-Token Latency----------------
Mean ITL (ms): 9.01
Median ITL (ms): 6.51
P95 ITL (ms): 23.19
P99 ITL (ms): 25.54
Max ITL (ms): 52.93
==================================================
5.1.2 Reasoning 시나리오 벤치마크
5.1.2.1 낮은 동시성
- 벤치마크 명령:
python -m sglang.bench_serving \
--backend sglang \
--model zai-org/Glyph \
--dataset-name random \
--random-input-len 1000 \
--random-output-len 8000 \
--num-prompts 10 \
--max-concurrency 1 \
--request-rate inf
- 테스트 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 1
Successful requests: 10
Benchmark duration (s): 201.53
Total input tokens: 6101
Total input text tokens: 6101
Total input vision tokens: 0
Total generated tokens: 44462
Total generated tokens (retokenized): 44455
Request throughput (req/s): 0.05
Input token throughput (tok/s): 30.27
Output token throughput (tok/s): 220.63
Peak output token throughput (tok/s): 251.00
Peak concurrent requests: 2
Total token throughput (tok/s): 250.90
Concurrency: 1.00
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 20151.45
Median E2E Latency (ms): 21576.31
---------------Time to First Token----------------
Mean TTFT (ms): 2362.23
Median TTFT (ms): 23.03
P99 TTFT (ms): 21310.14
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 4.00
Median TPOT (ms): 4.00
P99 TPOT (ms): 4.01
---------------Inter-Token Latency----------------
Mean ITL (ms): 4.00
Median ITL (ms): 4.00
P95 ITL (ms): 4.05
P99 ITL (ms): 4.08
Max ITL (ms): 5.67
==================================================
5.1.2.2 중간 동시성
- 벤치마크 명령:
python -m sglang.bench_serving \
--backend sglang \
--model zai-org/Glyph \
--dataset-name random \
--random-input-len 1000 \
--random-output-len 8000 \
--num-prompts 80 \
--max-concurrency 16 \
--request-rate inf
- 테스트 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 16
Successful requests: 80
Benchmark duration (s): 118.67
Total input tokens: 39668
Total input text tokens: 39668
Total input vision tokens: 0
Total generated tokens: 318306
Total generated tokens (retokenized): 318270
Request throughput (req/s): 0.67
Input token throughput (tok/s): 334.27
Output token throughput (tok/s): 2682.26
Peak output token throughput (tok/s): 3264.00
Peak concurrent requests: 19
Total token throughput (tok/s): 3016.53
Concurrency: 13.74
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 20387.23
Median E2E Latency (ms): 20466.09
---------------Time to First Token----------------
Mean TTFT (ms): 132.47
Median TTFT (ms): 27.19
P99 TTFT (ms): 583.15
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 5.09
Median TPOT (ms): 5.13
P99 TPOT (ms): 5.19
---------------Inter-Token Latency----------------
Mean ITL (ms): 5.09
Median ITL (ms): 5.08
P95 ITL (ms): 5.18
P99 ITL (ms): 5.57
Max ITL (ms): 522.26
==================================================
5.1.2.3 높은 동시성
- 벤치마크 명령:
python -m sglang.bench_serving \
--backend sglang \
--model zai-org/Glyph \
--dataset-name random \
--random-input-len 1000 \
--random-output-len 8000 \
--num-prompts 320 \
--max-concurrency 64 \
--request-rate inf
- 테스트 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 64
Successful requests: 320
Benchmark duration (s): 150.00
Total input tokens: 158939
Total input text tokens: 158939
Total input vision tokens: 0
Total generated tokens: 1301025
Total generated tokens (retokenized): 1300901
Request throughput (req/s): 2.13
Input token throughput (tok/s): 1059.59
Output token throughput (tok/s): 8673.49
Peak output token throughput (tok/s): 11899.00
Peak concurrent requests: 71
Total token throughput (tok/s): 9733.09
Concurrency: 54.71
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 25645.42
Median E2E Latency (ms): 26913.26
---------------Time to First Token----------------
Mean TTFT (ms): 163.75
Median TTFT (ms): 93.67
P99 TTFT (ms): 426.19
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 6.27
Median TPOT (ms): 6.39
P99 TPOT (ms): 6.59
---------------Inter-Token Latency----------------
Mean ITL (ms): 6.27
Median ITL (ms): 0.17
P95 ITL (ms): 32.94
P99 ITL (ms): 67.89
Max ITL (ms): 136.00
==================================================
5.1.3 Summarization 시나리오 벤치마크
5.1.3.1 낮은 동시성
- 벤치마크 명령:
python -m sglang.bench_serving \
--backend sglang \
--model zai-org/Glyph \
--dataset-name random \
--random-input-len 8000 \
--random-output-len 1000 \
--num-prompts 10 \
--max-concurrency 1 \
--request-rate inf
- 테스트 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 1
Successful requests: 10
Benchmark duration (s): 17.44
Total input tokens: 41941
Total input text tokens: 41941
Total input vision tokens: 0
Total generated tokens: 4220
Total generated tokens (retokenized): 4220
Request throughput (req/s): 0.57
Input token throughput (tok/s): 2405.19
Output token throughput (tok/s): 242.00
Peak output token throughput (tok/s): 250.00
Peak concurrent requests: 2
Total token throughput (tok/s): 2647.19
Concurrency: 1.00
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 1742.54
Median E2E Latency (ms): 1412.47
---------------Time to First Token----------------
Mean TTFT (ms): 53.48
Median TTFT (ms): 45.05
P99 TTFT (ms): 98.57
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 4.01
Median TPOT (ms): 4.01
P99 TPOT (ms): 4.03
---------------Inter-Token Latency----------------
Mean ITL (ms): 4.01
Median ITL (ms): 4.01
P95 ITL (ms): 4.06
P99 ITL (ms): 4.09
Max ITL (ms): 4.95
==================================================
5.1.3.2 중간 동시성
- 벤치마크 명령:
python -m sglang.bench_serving \
--backend sglang \
--model zai-org/Glyph \
--dataset-name random \
--random-input-len 8000 \
--random-output-len 1000 \
--num-prompts 80 \
--max-concurrency 16 \
--request-rate inf
- 테스트 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 16
Successful requests: 80
Benchmark duration (s): 16.90
Total input tokens: 300020
Total input text tokens: 300020
Total input vision tokens: 0
Total generated tokens: 41669
Total generated tokens (retokenized): 41668
Request throughput (req/s): 4.73
Input token throughput (tok/s): 17753.58
Output token throughput (tok/s): 2465.75
Peak output token throughput (tok/s): 3005.00
Peak concurrent requests: 25
Total token throughput (tok/s): 20219.33
Concurrency: 13.68
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 2890.33
Median E2E Latency (ms): 3069.55
---------------Time to First Token----------------
Mean TTFT (ms): 41.46
Median TTFT (ms): 31.75
P99 TTFT (ms): 93.18
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 5.52
Median TPOT (ms): 5.58
P99 TPOT (ms): 6.14
---------------Inter-Token Latency----------------
Mean ITL (ms): 5.48
Median ITL (ms): 5.13
P95 ITL (ms): 5.93
P99 ITL (ms): 20.76
Max ITL (ms): 36.01
==================================================
5.1.3.3 높은 동시성
- 벤치마크 명령:
python -m sglang.bench_serving \
--backend sglang \
--model zai-org/Glyph \
--dataset-name random \
--random-input-len 8000 \
--random-output-len 1000 \
--num-prompts 320 \
--max-concurrency 64 \
--request-rate inf
- 테스트 결과:
============ Serving Benchmark Result ============
Backend: sglang
Traffic request rate: inf
Max request concurrency: 64
Successful requests: 320
Benchmark duration (s): 35.54
Total input tokens: 1273893
Total input text tokens: 1273893
Total input vision tokens: 0
Total generated tokens: 170000
Total generated tokens (retokenized): 169994
Request throughput (req/s): 9.01
Input token throughput (tok/s): 35848.57
Output token throughput (tok/s): 4783.96
Peak output token throughput (tok/s): 8396.00
Peak concurrent requests: 80
Total token throughput (tok/s): 40632.53
Concurrency: 59.26
----------------End-to-End Latency----------------
Mean E2E Latency (ms): 6580.96
Median E2E Latency (ms): 6248.74
---------------Time to First Token----------------
Mean TTFT (ms): 345.27
Median TTFT (ms): 96.06
P99 TTFT (ms): 2823.92
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms): 12.26
Median TPOT (ms): 12.53
P99 TPOT (ms): 23.58
---------------Inter-Token Latency----------------
Mean ITL (ms): 11.76
Median ITL (ms): 6.57
P95 ITL (ms): 27.66
P99 ITL (ms): 91.24
Max ITL (ms): 2609.64
==================================================
5.2 정확도 벤치마크
표준 벤치마크에서 모델 정확도를 기록해요:
5.2.1 GSM8K 벤치마크
- 벤치마크 명령
python -m sglang.test.few_shot_gsm8k \
--num-questions 200
- 테스트 결과
Accuracy: 0.890
Invalid: 0.000
Latency: 3.718 s
Output throughput: 5245.606 token/s
더 알아보기 (Learn more)
- Glyph 모델 카드
- GLM-4.5 cookbook 페이지 (Thinking Budget 사용 예시)
- SGLang 설치 가이드
- SGLang 기본 사용 가이드