Qwen3-Coder-Next

3.2 Configuration Tips

• 컨텍스트 길이: 모델은 네이티브로 최대 256K 토큰을 지원해요. OOM 문제가 발생하면 --context-length 32768을 시도해 보세요.
• 도구 사용: 도구 호출 기능을 활성화하려면 --tool-call-parser qwen3_coder 플래그를 사용하세요.
• 샘플링 파라미터: SGLang이 모델의 generation_config.json에서 권장 샘플링 파라미터를 자동 적용해요. 수동 구성은 필요 없어요.
• Mamba Radix Cache: Qwen3-Coder-Next의 하이브리드 Gated Delta Networks 아키텍처는 --mamba-radix-cache-strategy를 통해 두 가지 mamba 스케줄링 전략을 지원해요:
  • V1 (no_buffer): 기본값. overlap 스케줄러 없음, 더 낮은 메모리 사용.
  • V2 (extra_buffer): --mamba-radix-cache-strategy extra_buffer --page-size 64로 overlap 스케줄링과 branching point 캐싱을 활성화. FLA 커널 백엔드 필요. 더 높은 mamba 상태 메모리를 처리량 향상과 맞바꿔요. 비-KV-cache-바운드 시나리오에서는 엄격히 우월하고, KV-cache-바운드 경우에는 overlap 스케줄링 이점과 줄어든 최대 동시성을 저울질해야 해요. --page-size는 FLA_CHUNK_SIZE % page_size == 0 또는 page_size % FLA_CHUNK_SIZE == 0을 만족해야 해요(FLA_CHUNK_SIZE는 현재 64).
• Xeon CPU 서비스 구성: SGLang CPU 서버 문서의 serving engine 실행 섹션에서 Notes 부분을 참조해 TP(텐서 병렬) 및 NUMA 바인딩 설정을 올바르게 구성하는 방법을 이해하세요.

4. Model Invocation

배포 명령:

python -m sglang.launch_server \
  --model Qwen/Qwen3-Coder-Next \
  --tp 2 \
  --tool-call-parser qwen3_coder \
  --host 0.0.0.0 \
  --port 30000

4.1 Basic Usage

기본 API 사용법과 요청 예시는 다음을 참조하세요:

• SGLang Basic Usage Guide

4.2 Advanced Usage

4.2.1 Code Generation Example

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY"
)

response = client.chat.completions.create(
    model="Qwen/Qwen3-Coder-Next",
    messages=[
        {"role": "user", "content": "Write a Python function that implements binary search on a sorted list. Include type hints."}
    ],
    max_tokens=2048
)

print(response.choices[0].message.content)

출력 예시:

Here's a Python function implementing binary search on a sorted list, with comprehensive type hints:

```python
from typing import Sequence, TypeVar, Optional

T = TypeVar('T')

def binary_search(sorted_list: Sequence[T], target: T) -> Optional[int]:
    """
    Perform binary search on a sorted list to find the index of a target element.

    Args:
        sorted_list: A sequence (e.g., list, tuple) sorted in ascending order.
        target: The element to search for in the list.

    Returns:
        The index of the target element if found, or None if not found.

    Time Complexity: O(log n)
    Space Complexity: O(1)

    Note:
        The function assumes the list is sorted in ascending order.
        If the list contains duplicate elements, it returns the index of one of them.
    """
    left = 0
    right = len(sorted_list) - 1

    while left <= right:
        mid = (left + right) // 2
        mid_val = sorted_list[mid]

        if mid_val == target:
            return mid
        elif mid_val < target:
            left = mid + 1
        else:
            right = mid - 1

    return None

Example usage:

# Example 1: Finding an existing element
numbers = [1, 3, 5, 7, 9, 11]
print(binary_search(numbers, 7))  # Output: 3

# Example 2: Element not in the list
print(binary_search(numbers, 4))  # Output: None

# Example 3: Empty list
print(binary_search([], 5))  # Output: None

# Example 4: Single element
print(binary_search([1], 1))  # Output: 0
print(binary_search([1], 2))  # Output: None

Key features:

• Uses TypeVar to support generic types (as long as comparison operations are defined)
• Returns Optional[int] to indicate either the index or no match found
• Uses Sequence[T] to accept any sequence type (list, tuple, etc.)
• Includes comprehensive docstring with time/space complexity
• Implements standard iterative binary search for O(1) space complexity

#### 4.2.2 Streaming Example

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY"
)

response = client.chat.completions.create(
    model="Qwen/Qwen3-Coder-Next",
    messages=[
        {"role": "user", "content": "Explain the difference between a stack and a queue in 3 sentences."}
    ],
    max_tokens=512,
    stream=True
)

for chunk in response:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
print()

출력 예시:

A **stack** follows the **Last In, First Out (LIFO)** principle, meaning the last element added is the first one removed—operations like `push` (add) and `pop` (remove) occur at the same end, called the *top*. In contrast, a **queue** follows the **First In, First Out (FIFO)** principle, where elements are added at the *back* (enqueue) and removed from the *front* (dequeue), preserving the order of insertion. This structural difference makes stacks ideal for tasks like function call management and expression evaluation, while queues suit scheduling, buffering, and breadth-first traversal.

4.2.3 Tool Calling Example

Qwen3-Coder-Next는 도구 호출 기능을 지원해요. 위 배포 명령에 --tool-call-parser qwen3_coder가 포함되어 있는지 확인하세요.

Python 예시:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY"
)

# Define available tools
tools = [
    {
        "type": "function",
        "function": {
            "name": "execute_code",
            "description": "Execute Python code and return the result",
            "parameters": {
                "type": "object",
                "properties": {
                    "code": {
                        "type": "string",
                        "description": "The Python code to execute"
                    }
                },
                "required": ["code"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="Qwen/Qwen3-Coder-Next",
    messages=[
        {"role": "user", "content": "Calculate the factorial of 10 using Python"}
    ],
    tools=tools
)

# Check if the model wants to call a tool
if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    print(f"Tool: {tool_call.function.name}")
    print(f"Arguments: {tool_call.function.arguments}")
else:
    print(response.choices[0].message.content)

출력 예시:

Tool: execute_code
Arguments: {"code": "import math\nmath.factorial(10)"}

5. Benchmark

5.1 Speed Benchmark

테스트 환경:

• 하드웨어: NVIDIA B200 GPU (2x)
• 모델: Qwen/Qwen3-Coder-Next
• 텐서 병렬 처리(Tensor Parallelism): 2
• sglang 버전: 0.5.8+

5.1.1 Standard Scenario Benchmark

• 모델 배포 명령:

python -m sglang.launch_server \
  --model Qwen/Qwen3-Coder-Next \
  --tp 2 \
  --host 0.0.0.0 \
  --port 30000

5.1.1.1 Low Concurrency

• 벤치마크 명령:

python3 -m sglang.bench_serving \
  --backend sglang \
  --host 127.0.0.1 \
  --port 30000 \
  --model Qwen/Qwen3-Coder-Next \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 1000 \
  --num-prompts 10 \
  --max-concurrency 1

• 결과:

============ Serving Benchmark Result ============
Backend:                                 sglang
Traffic request rate:                    inf
Max request concurrency:                 1
Successful requests:                     10
Benchmark duration (s):                  27.86
Total input tokens:                      6101
Total input text tokens:                 6101
Total generated tokens:                  4220
Total generated tokens (retokenized):    4218
Request throughput (req/s):              0.36
Input token throughput (tok/s):          219.00
Output token throughput (tok/s):         151.48
Peak output token throughput (tok/s):    166.00
Peak concurrent requests:                2
Total token throughput (tok/s):          370.48
Concurrency:                             1.00
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   2784.14
Median E2E Latency (ms):                 2258.08
P90 E2E Latency (ms):                    5044.43
P99 E2E Latency (ms):                    6130.52
---------------Time to First Token----------------
Mean TTFT (ms):                          161.68
Median TTFT (ms):                        168.09
P99 TTFT (ms):                           183.26
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          6.19
Median TPOT (ms):                        6.23
P99 TPOT (ms):                           6.32
---------------Inter-Token Latency----------------
Mean ITL (ms):                           6.23
Median ITL (ms):                         6.23
P95 ITL (ms):                            6.51
P99 ITL (ms):                            6.64
Max ITL (ms):                            13.45
==================================================

5.1.1.2 Medium Concurrency

• 벤치마크 명령:

python3 -m sglang.bench_serving \
  --backend sglang \
  --host 127.0.0.1 \
  --port 30000 \
  --model Qwen/Qwen3-Coder-Next \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 1000 \
  --num-prompts 80 \
  --max-concurrency 16

• 결과:

============ Serving Benchmark Result ============
Backend:                                 sglang
Traffic request rate:                    inf
Max request concurrency:                 16
Successful requests:                     80
Benchmark duration (s):                  39.06
Total input tokens:                      39668
Total input text tokens:                 39668
Total generated tokens:                  40805
Total generated tokens (retokenized):    40789
Request throughput (req/s):              2.05
Input token throughput (tok/s):          1015.62
Output token throughput (tok/s):         1044.73
Peak output token throughput (tok/s):    1664.00
Peak concurrent requests:                21
Total token throughput (tok/s):          2060.34
Concurrency:                             14.16
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   6910.97
Median E2E Latency (ms):                 7248.27
P90 E2E Latency (ms):                    11612.63
P99 E2E Latency (ms):                    13933.91
---------------Time to First Token----------------
Mean TTFT (ms):                          183.48
Median TTFT (ms):                        156.50
P99 TTFT (ms):                           311.46
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          13.61
Median TPOT (ms):                        13.59
P99 TPOT (ms):                           21.11
---------------Inter-Token Latency----------------
Mean ITL (ms):                           13.22
Median ITL (ms):                         9.76
P95 ITL (ms):                            10.43
P99 ITL (ms):                            158.04
Max ITL (ms):                            394.39
==================================================

5.1.1.3 High Concurrency

• 벤치마크 명령:

python3 -m sglang.bench_serving \
  --backend sglang \
  --host 127.0.0.1 \
  --port 30000 \
  --model Qwen/Qwen3-Coder-Next \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 1000 \
  --num-prompts 500 \
  --max-concurrency 100

• 결과:

============ Serving Benchmark Result ============
Backend:                                 sglang
Traffic request rate:                    inf
Max request concurrency:                 100
Successful requests:                     500
Benchmark duration (s):                  102.81
Total input tokens:                      249831
Total input text tokens:                 249831
Total generated tokens:                  252662
Total generated tokens (retokenized):    252536
Request throughput (req/s):              4.86
Input token throughput (tok/s):          2429.99
Output token throughput (tok/s):         2457.53
Peak output token throughput (tok/s):    5299.00
Peak concurrent requests:                109
Total token throughput (tok/s):          4887.52
Concurrency:                             94.28
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   19385.20
Median E2E Latency (ms):                 17584.09
P90 E2E Latency (ms):                    36762.15
P99 E2E Latency (ms):                    42518.35
---------------Time to First Token----------------
Mean TTFT (ms):                          270.62
Median TTFT (ms):                        159.65
P99 TTFT (ms):                           938.90
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          38.57
Median TPOT (ms):                        41.78
P99 TPOT (ms):                           53.28
---------------Inter-Token Latency----------------
Mean ITL (ms):                           37.90
Median ITL (ms):                         18.26
P95 ITL (ms):                            167.82
P99 ITL (ms):                            311.45
Max ITL (ms):                            993.20
==================================================

5.1.2 Reasoning Scenario Benchmark

• 모델 배포 명령:

python -m sglang.launch_server \
  --model Qwen/Qwen3-Coder-Next \
  --tp 2 \
  --host 0.0.0.0 \
  --port 30000

5.1.2.1 Low Concurrency

• 벤치마크 명령:

python3 -m sglang.bench_serving \
  --backend sglang \
  --host 127.0.0.1 \
  --port 30000 \
  --model Qwen/Qwen3-Coder-Next \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 8000 \
  --num-prompts 10 \
  --max-concurrency 1

• 결과:

============ Serving Benchmark Result ============
Backend:                                 sglang
Traffic request rate:                    inf
Max request concurrency:                 1
Successful requests:                     10
Benchmark duration (s):                  285.02
Total input tokens:                      6101
Total input text tokens:                 6101
Total generated tokens:                  44462
Total generated tokens (retokenized):    44432
Request throughput (req/s):              0.04
Input token throughput (tok/s):          21.41
Output token throughput (tok/s):         156.00
Peak output token throughput (tok/s):    173.00
Peak concurrent requests:                2
Total token throughput (tok/s):          177.40
Concurrency:                             1.00
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   28499.54
Median E2E Latency (ms):                 30424.65
P90 E2E Latency (ms):                    49132.26
P99 E2E Latency (ms):                    51075.28
---------------Time to First Token----------------
Mean TTFT (ms):                          95.51
Median TTFT (ms):                        93.86
P99 TTFT (ms):                           112.56
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          6.24
Median TPOT (ms):                        6.30
P99 TPOT (ms):                           6.60
---------------Inter-Token Latency----------------
Mean ITL (ms):                           6.39
Median ITL (ms):                         6.34
P95 ITL (ms):                            7.16
P99 ITL (ms):                            7.42
Max ITL (ms):                            12.48
==================================================

5.1.2.2 Medium Concurrency

• 벤치마크 명령:

python3 -m sglang.bench_serving \
  --backend sglang \
  --host 127.0.0.1 \
  --port 30000 \
  --model Qwen/Qwen3-Coder-Next \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 8000 \
  --num-prompts 80 \
  --max-concurrency 16

• 결과:

============ Serving Benchmark Result ============
Backend:                                 sglang
Traffic request rate:                    inf
Max request concurrency:                 16
Successful requests:                     80
Benchmark duration (s):                  237.77
Total input tokens:                      39668
Total input text tokens:                 39668
Total generated tokens:                  318306
Total generated tokens (retokenized):    315646
Request throughput (req/s):              0.34
Input token throughput (tok/s):          166.83
Output token throughput (tok/s):         1338.72
Peak output token throughput (tok/s):    1727.00
Peak concurrent requests:                19
Total token throughput (tok/s):          1505.55
Concurrency:                             13.88
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   41266.21
Median E2E Latency (ms):                 41010.10
P90 E2E Latency (ms):                    77574.22
P99 E2E Latency (ms):                    82688.04
---------------Time to First Token----------------
Mean TTFT (ms):                          140.73
Median TTFT (ms):                        84.52
P99 TTFT (ms):                           365.86
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          10.32
Median TPOT (ms):                        10.38
P99 TPOT (ms):                           10.87
---------------Inter-Token Latency----------------
Mean ITL (ms):                           10.34
Median ITL (ms):                         10.19
P95 ITL (ms):                            10.75
P99 ITL (ms):                            11.18
Max ITL (ms):                            206.79
==================================================

5.1.2.3 High Concurrency

• 벤치마크 명령:

python3 -m sglang.bench_serving \
  --backend sglang \
  --host 127.0.0.1 \
  --port 30000 \
  --model Qwen/Qwen3-Coder-Next \
  --dataset-name random \
  --random-input-len 1000 \
  --random-output-len 8000 \
  --num-prompts 320 \
  --max-concurrency 64

• 결과:

============ Serving Benchmark Result ============
Backend:                                 sglang
Traffic request rate:                    inf
Max request concurrency:                 64
Successful requests:                     320
Benchmark duration (s):                  384.82
Total input tokens:                      158939
Total input text tokens:                 158939
Total generated tokens:                  1301025
Total generated tokens (retokenized):    1299908
Request throughput (req/s):              0.83
Input token throughput (tok/s):          413.02
Output token throughput (tok/s):         3380.83
Peak output token throughput (tok/s):    4317.00
Peak concurrent requests:                69
Total token throughput (tok/s):          3793.85
Concurrency:                             56.42
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   67847.54
Median E2E Latency (ms):                 70724.38
P90 E2E Latency (ms):                    120888.83
P99 E2E Latency (ms):                    133234.48
---------------Time to First Token----------------
Mean TTFT (ms):                          212.24
Median TTFT (ms):                        115.96
P99 TTFT (ms):                           652.93
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          16.76
Median TPOT (ms):                        16.99
P99 TPOT (ms):                           18.18
---------------Inter-Token Latency----------------
Mean ITL (ms):                           16.64
Median ITL (ms):                         15.83
P95 ITL (ms):                            31.64
P99 ITL (ms):                            90.85
Max ITL (ms):                            576.60
==================================================

5.1.3 Summarization Scenario Benchmark

5.1.3.1 Low Concurrency

• 벤치마크 명령:

python3 -m sglang.bench_serving \
  --backend sglang \
  --host 127.0.0.1 \
  --port 30000 \
  --model Qwen/Qwen3-Coder-Next \
  --dataset-name random \
  --random-input-len 8000 \
  --random-output-len 1000 \
  --num-prompts 10 \
  --max-concurrency 1

• 결과:

============ Serving Benchmark Result ============
Backend:                                 sglang
Traffic request rate:                    inf
Max request concurrency:                 1
Successful requests:                     10
Benchmark duration (s):                  29.42
Total input tokens:                      41941
Total input text tokens:                 41941
Total generated tokens:                  4220
Total generated tokens (retokenized):    4220
Request throughput (req/s):              0.34
Input token throughput (tok/s):          1425.35
Output token throughput (tok/s):         143.42
Peak output token throughput (tok/s):    169.00
Peak concurrent requests:                3
Total token throughput (tok/s):          1568.77
Concurrency:                             1.00
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   2941.19
Median E2E Latency (ms):                 2411.84
P90 E2E Latency (ms):                    5661.26
P99 E2E Latency (ms):                    6497.45
---------------Time to First Token----------------
Mean TTFT (ms):                          139.46
Median TTFT (ms):                        160.33
P99 TTFT (ms):                           184.30
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          6.56
Median TPOT (ms):                        6.65
P99 TPOT (ms):                           7.29
---------------Inter-Token Latency----------------
Mean ITL (ms):                           6.65
Median ITL (ms):                         6.68
P95 ITL (ms):                            7.39
P99 ITL (ms):                            7.51
Max ITL (ms):                            16.34
==================================================

5.1.3.2 Medium Concurrency

• 벤치마크 명령:

python3 -m sglang.bench_serving \
  --backend sglang \
  --host 127.0.0.1 \
  --port 30000 \
  --model Qwen/Qwen3-Coder-Next \
  --dataset-name random \
  --random-input-len 8000 \
  --random-output-len 1000 \
  --num-prompts 80 \
  --max-concurrency 16

• 결과:

============ Serving Benchmark Result ============
Backend:                                 sglang
Traffic request rate:                    inf
Max request concurrency:                 16
Successful requests:                     80
Benchmark duration (s):                  41.62
Total input tokens:                      300020
Total input text tokens:                 300020
Total generated tokens:                  41669
Total generated tokens (retokenized):    41664
Request throughput (req/s):              1.92
Input token throughput (tok/s):          7208.67
Output token throughput (tok/s):         1001.19
Peak output token throughput (tok/s):    1536.00
Peak concurrent requests:                21
Total token throughput (tok/s):          8209.86
Concurrency:                             14.27
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   7421.29
Median E2E Latency (ms):                 7985.77
P90 E2E Latency (ms):                    12122.09
P99 E2E Latency (ms):                    14595.05
---------------Time to First Token----------------
Mean TTFT (ms):                          248.49
Median TTFT (ms):                        179.25
P99 TTFT (ms):                           915.90
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          14.13
Median TPOT (ms):                        14.28
P99 TPOT (ms):                           24.02
---------------Inter-Token Latency----------------
Mean ITL (ms):                           13.80
Median ITL (ms):                         10.46
P95 ITL (ms):                            11.00
P99 ITL (ms):                            173.14
Max ITL (ms):                            823.32
==================================================

5.1.3.3 High Concurrency

• 벤치마크 명령:

python3 -m sglang.bench_serving \
  --backend sglang \
  --host 127.0.0.1 \
  --port 30000 \
  --model Qwen/Qwen3-Coder-Next \
  --dataset-name random \
  --random-input-len 8000 \
  --random-output-len 1000 \
  --num-prompts 320 \
  --max-concurrency 64

• 결과:

============ Serving Benchmark Result ============
Backend:                                 sglang
Traffic request rate:                    inf
Max request concurrency:                 64
Successful requests:                     320
Benchmark duration (s):                  85.74
Total input tokens:                      1273893
Total input text tokens:                 1273893
Total generated tokens:                  170000
Total generated tokens (retokenized):    169983
Request throughput (req/s):              3.73
Input token throughput (tok/s):          14858.12
Output token throughput (tok/s):         1982.80
Peak output token throughput (tok/s):    3734.00
Peak concurrent requests:                70
Total token throughput (tok/s):          16840.92
Concurrency:                             59.75
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   16008.12
Median E2E Latency (ms):                 15460.65
P90 E2E Latency (ms):                    27705.81
P99 E2E Latency (ms):                    32874.74
---------------Time to First Token----------------
Mean TTFT (ms):                          476.99
Median TTFT (ms):                        177.50
P99 TTFT (ms):                           3014.39
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          29.81
Median TPOT (ms):                        31.19
P99 TPOT (ms):                           45.53
---------------Inter-Token Latency----------------
Mean ITL (ms):                           29.29
Median ITL (ms):                         15.75
P95 ITL (ms):                            173.94
P99 ITL (ms):                            202.00
Max ITL (ms):                            2783.23
==================================================

5.2 Accuracy Benchmark

5.2.1 GSM8K Benchmark

• 벤치마크 명령:

python benchmark/gsm8k/bench_sglang.py --port 30000

• 테스트 결과:

Accuracy: 0.965
Invalid: 0.000
Latency: 26.407 s
Output throughput: 929.132 token/s

5.2.2 MMLU Benchmark

• 벤치마크 명령:

cd benchmark/mmlu
bash download_data.sh
python3 bench_sglang.py --port 30000

• 테스트 결과:

subject: abstract_algebra, #q:100, acc: 0.780
subject: anatomy, #q:135, acc: 0.807
subject: astronomy, #q:152, acc: 0.921
subject: business_ethics, #q:100, acc: 0.820
subject: clinical_knowledge, #q:265, acc: 0.860
subject: college_biology, #q:144, acc: 0.944
subject: college_chemistry, #q:100, acc: 0.590
subject: college_computer_science, #q:100, acc: 0.820
subject: college_mathematics, #q:100, acc: 0.800
subject: college_medicine, #q:173, acc: 0.803
subject: college_physics, #q:102, acc: 0.775
subject: computer_security, #q:100, acc: 0.880
subject: conceptual_physics, #q:235, acc: 0.936
subject: econometrics, #q:114, acc: 0.807
subject: electrical_engineering, #q:145, acc: 0.834
subject: elementary_mathematics, #q:378, acc: 0.854
subject: formal_logic, #q:126, acc: 0.802
subject: global_facts, #q:100, acc: 0.610
subject: high_school_biology, #q:310, acc: 0.971
subject: high_school_chemistry, #q:203, acc: 0.803
subject: high_school_computer_science, #q:100, acc: 0.920
subject: high_school_european_history, #q:165, acc: 0.891
subject: high_school_geography, #q:198, acc: 0.929
subject: high_school_government_and_politics, #q:193, acc: 0.969
subject: high_school_macroeconomics, #q:390, acc: 0.903
subject: high_school_mathematics, #q:270, acc: 0.689
subject: high_school_microeconomics, #q:238, acc: 0.962
subject: high_school_physics, #q:151, acc: 0.854
subject: high_school_psychology, #q:545, acc: 0.947
subject: high_school_statistics, #q:216, acc: 0.815
subject: high_school_us_history, #q:204, acc: 0.907
subject: high_school_world_history, #q:237, acc: 0.937
subject: human_aging, #q:223, acc: 0.821
subject: human_sexuality, #q:131, acc: 0.840
subject: international_law, #q:121, acc: 0.934
subject: jurisprudence, #q:108, acc: 0.870
subject: logical_fallacies, #q:163, acc: 0.847
subject: machine_learning, #q:112, acc: 0.812
subject: management, #q:103, acc: 0.922
subject: marketing, #q:234, acc: 0.923
subject: medical_genetics, #q:100, acc: 0.970
subject: miscellaneous, #q:783, acc: 0.941
subject: moral_disputes, #q:346, acc: 0.850
subject: moral_scenarios, #q:895, acc: 0.726
subject: nutrition, #q:306, acc: 0.915
subject: philosophy, #q:311, acc: 0.859
subject: prehistory, #q:324, acc: 0.889
subject: professional_accounting, #q:282, acc: 0.723
subject: professional_law, #q:1534, acc: 0.648
subject: professional_medicine, #q:272, acc: 0.923
subject: professional_psychology, #q:612, acc: 0.845
subject: public_relations, #q:110, acc: 0.782
subject: security_studies, #q:245, acc: 0.796
subject: sociology, #q:201, acc: 0.925
subject: us_foreign_policy, #q:100, acc: 0.950
subject: virology, #q:166, acc: 0.572
subject: world_religions, #q:171, acc: 0.883
Total latency: 208.985
Average accuracy: 0.834

더 알아보기

• Qwen3-Coder-Next (Hugging Face)
• Qwen3-Coder