Laguna-XS.2

3.2 Configuration Tips

• Trust remote code (--trust-remote-code): Laguna-XS.2은 Hugging Face Hub에 커스텀 모델링/config 코드를 실어 보내므로 서버가 모델을 로드하려면 이 플래그가 필요해요.
• 양자화: NVFP4는 Blackwell(B200 / B300)이 필요해요. BF16과 FP8은 H200 또는 B200 둘 다에서 실행돼요. FP8의 첫 실행은 멀티세션 DeepGEMM JIT 사전 컴파일(약 10-20분)을 트리거해요. python3 -m sglang.compile_deep_gemm --model poolside/Laguna-XS.2-FP8로 미리 웜업해서 재시작마다 그 비용을 피하세요.
• Reasoning parser (--reasoning-parser poolside_v1): thinking... response 세그먼트를 reasoning_content로 분리해 content가 최종 답만 담도록 해요. content에 원시 thinking 태그를 원할 때만 비활성화하세요.
• Tool call parser (--tool-call-parser poolside_v1): OpenAI 호환 도구 호출 스트리밍에 필요해요. 채팅 전용 배포에서만 비활성화하세요.
• DP attention: 더 높은 처리량 배포를 위해 DP-Attention 토글을 활성화하세요 — --dp <N> --enable-dp-attention을 생성하며 --dp가 --tp와 일치해요(필요하면 별도 튜닝).
• Thinking 기본값: Thinking은 모델 수준에서 기본적으로 꺼져 있어요. 요청마다 extra_body={"chat_template_kwargs": {"enable_thinking": True}}로 옵트인하세요.

4. Model Invocation

아래 샘플은 서버가 http://localhost:30000/v1에서 접근 가능하다고 가정해요.

4.1 Basic Chat

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY",
)

resp = client.chat.completions.create(
    model="poolside/Laguna-XS.2",
    messages=[
        {"role": "user", "content": "What is the difference between TCP and UDP?"}
    ],
    max_tokens=1024,
)
print(resp.choices[0].message.content)

출력 예시:

TCP (Transmission Control Protocol) and UDP (User Datagram Protocol) are two core protocols of the Internet Protocol (IP) suite, both used for network communication but with key differences:

## Connection Handling
- **TCP**: Connection-oriented protocol that establishes a connection before data transfer (like a phone call)
- **UDP**: Connectionless protocol that sends data without establishing a connection (like sending a letter)

## Reliability
- **TCP**: Guaranteed delivery with error checking, retransmission of lost packets, and flow control
- **UDP**: No guarantee of delivery; packets may be lost, duplicated, or arrive out of order

## Speed & Overhead
- **TCP**: Slower due to connection setup, acknowledgment overhead, and error correction mechanisms
- **UDP**: Faster with minimal overhead since it doesn't wait for acknowledgments or retransmit lost data

## Use Cases
- **TCP**: Web browsing (HTTP/HTTPS), email (SMTP), file transfers (FTP), database connections
- **UDP**: Video streaming, online gaming, VoIP calls, DNS queries, live broadcasts

In essence, TCP prioritizes reliability over speed, while UDP prioritizes speed over reliability.

4.2 Reasoning (Thinking Mode)

Laguna-XS.2는 thinking... response 태그 사이에서 추론을 생성해요. --reasoning-parser poolside_v1 플래그가 thinking 텍스트를 reasoning_content로 분리해 content가 최종 답만 담도록 해요. Thinking은 요청마다 옵트인돼요:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY",
)

resp = client.chat.completions.create(
    model="poolside/Laguna-XS.2",
    messages=[
        {"role": "user", "content": "If a train travels at 60 km/h for 2.5 hours, how far does it go?"}
    ],
    max_tokens=4096,
    extra_body={"chat_template_kwargs": {"enable_thinking": True}},
)

print("====== Reasoning Content ======")
print(resp.choices[0].message.reasoning_content)
print("====== Answer ======")
print(resp.choices[0].message.content)

출력 예시:

====== Reasoning Content ======
The user is asking a straightforward math problem about distance, speed, and time. I need to calculate the distance using the formula:

Distance = Speed × Time

Given:
- Speed = 60 km/h
- Time = 2.5 hours

So the calculation would be:
Distance = 60 × 2.5 = 150 km

This is a simple multiplication problem. I should provide a clear, direct answer and maybe explain the calculation briefly.

====== Answer ======
To find the distance, use the formula:

Distance = Speed × Time
Distance = 60 km/h × 2.5 h = 150 km

The train travels **150 kilometers**.

thinking을 비활성화하려면 extra_body를 생략(기본적으로 꺼짐)하거나 chat_template_kwargs={"enable_thinking": False}를 명시적으로 전달하세요.

4.3 Tool Calling

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:30000/v1",
    api_key="EMPTY",
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "The city name"},
                },
                "required": ["location"],
            },
        },
    }
]

resp = client.chat.completions.create(
    model="poolside/Laguna-XS.2",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
)

msg = resp.choices[0].message
print("====== Reasoning Content ======")
print(msg.reasoning_content)
print("====== Content ======")
print(msg.content)
print("====== Tool Calls ======")
for tc in msg.tool_calls or []:
    print(f"  Function: {tc.function.name}")
    print(f"  Arguments: {tc.function.arguments}")

출력 예시:

====== Reasoning Content ======
None
====== Content ======

I'll check the current weather in Tokyo for you.

====== Tool Calls ======
  Function: get_weather
  Arguments: {"location": "Tokyo"}

reasoning_content가 None인 이유는 thinking이 기본적으로 꺼져 있기 때문이고, content는 도구 호출 앞에 오는 짧은 assistant 메시지를 담아요. 도구 호출 앞에 interleaved reasoning을 원한다면 extra_body={"chat_template_kwargs": {"enable_thinking": True}}를 추가하세요.

5. Benchmark

5.1 Accuracy Benchmark

테스트 환경:

• 하드웨어: NVIDIA H200 (4×H200)
• 모델: poolside/Laguna-XS.2 (BF16)
• 텐서 병렬 처리(Tensor Parallelism): 4
• SGLang 버전: 0.5.12.dev20260509+g096ad02b0 (#24204 병합 커밋을 담은 nightly wheel; 원래 PR과 동일한 코드 경로)
• Reasoning Parser: poolside_v1
• Tool Call Parser: poolside_v1
• 샘플링: temperature=0.6, max_tokens=16384, chat_template_kwargs={"enable_thinking": true}, n_repeats=1
• 그레이더: NeMo-Skills math_verify (math) 및 eval_mcq (multichoice)

결과 (PR #24204에서):

5.2 Speed Benchmark

테스트 환경:

• 하드웨어: NVIDIA H200 (TP=1은 1×H200, TP=4는 4×H200)
• 모델: poolside/Laguna-XS.2 (BF16)
• SGLang 버전: 0.5.12.dev20260509+g096ad02b0 (#24204 병합 커밋을 담은 nightly wheel; 원래 PR과 동일한 코드 경로)
• 워크로드: sglang.bench_serving --backend sglang --dataset-name random (기본값: --random-input-len 1024 --random-output-len 1024 --random-range-ratio 0.0)
• 서버 플래그는 위 정확도 실행과 동일.

5.2.1 Latency Benchmark (10 prompts, concurrency = 1)

python3 -m sglang.bench_serving --backend sglang \
  --host 0.0.0.0 --port 30000 \
  --dataset-name random --num-prompts 10 --max-concurrency 1

5.2.2 Throughput Benchmark (1000 prompts, concurrency = 100)

python3 -m sglang.bench_serving --backend sglang \
  --host 0.0.0.0 --port 30000 \
  --dataset-name random --num-prompts 1000 --max-concurrency 100

TP=4는 cc=100 임의 워크로드에서 TP=1 대비 약 2.0배 총 토큰 처리량과 약 1.7배 낮은 평균 TTFT를 제공해요.

더 알아보기

• Laguna-XS.2 (Hugging Face)
• Laguna-XS.2-FP8 (Hugging Face)
• Laguna-XS.2-NVFP4 (Hugging Face)