프롬프트 강화가 있는 확산 모델

프롬프트 강화가 있는 확산 모델 (Diffusion Models with Prompt Enhancement, PE)

이 페이지는 내장 프롬프트 강화(Prompt Enhancement, PE) 기능 또는 별도의 SGLang 제공 PE 모델로 ERNIE-Image를 실행하는 방법을 설명해요. 기본적으로 확산 서버는 SGLang의 네이티브 Ministral3 구현으로 PE 모델을 프로세스 내에서 로드해요. PE 모델이 독립적인 리소스나 스케일링이 필요할 때는 별도의 SGLang 서버로 배포할 수 있어요.

출처: 문서

본문

빠른 시작 (Quick Start)

기본적으로 확산 서버는 SGLang의 네이티브 Ministral3 구현으로 PE 모델을 프로세스 내(in-process)에서 로드해요. PE 모델이 독립적인 리소스나 스케일링(scaling)이 필요할 때는 별도의 SGLang 서버로 배포하세요. 이 문서는 baidu/ERNIE-Image을 예시로 사용해요.

내장 네이티브 PE 구현(기본값)으로 모델 실행:

# Terminal 1: launch server
sglang serve --model-path baidu/ERNIE-Image --port ${PORT}

# Terminal 2: launch client
curl -X POST http://${HOST}:${PORT}/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "This is a photograph depicting an urban street scene. Shot at eye level, it shows a covered pedestrian or commercial street. Slightly below the center of the frame, a cyclist rides away from the camera toward the background, appearing as a dark silhouette against backlighting with indistinct details. The ground is paved with regular square tiles, bisected by a prominent tactile paving strip running through the scene, whose raised textures are clearly visible under the light. Light streams in diagonally from the right side of the frame, creating a strong backlight effect with a distinct Tyndall effect—visible light beams illuminating dust or vapor in the air and casting long shadows across the street. Several pedestrians appear on the left side and in the distance, some with their backs to the camera and others walking sideways, all rendered as silhouettes or semi-silhouettes. The overall color palette is warm, dominated by golden yellows and dark browns, evoking the atmosphere of dusk or early morning.",
    "height": 1264,
    "width": 848,
    "num_inference_steps":   50,
    "guidance_scale": 4.0,
    "use_pe": true
  }'

SGLang 제공 PE 모델로 실행(고성능):

# Terminal 1: launch SGLang PE model server
sglang serve --model-path /path/to/baidu/ERNIE-Image/pe/ --port ${PE_PORT}

# Terminal 2: launch diffusion model server with PE server
sglang serve --model-path /path/to/baidu/ERNIE-Image/ \
  --pe-server-url "http://${HOST}:${PE_PORT}" \
  --port ${PORT}

# Terminal 3: launch client
curl -X POST http://${HOST}:${PORT}/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "This is a photograph depicting an urban street scene. Shot at eye level, it shows a covered pedestrian or commercial street. Slightly below the center of the frame, a cyclist rides away from the camera toward the background, appearing as a dark silhouette against backlighting with indistinct details. The ground is paved with regular square tiles, bisected by a prominent tactile paving strip running through the scene, whose raised textures are clearly visible under the light. Light streams in diagonally from the right side of the frame, creating a strong backlight effect with a distinct Tyndall effect—visible light beams illuminating dust or vapor in the air and casting long shadows across the street. Several pedestrians appear on the left side and in the distance, some with their backs to the camera and others walking sideways, all rendered as silhouettes or semi-silhouettes. The overall color palette is warm, dominated by golden yellows and dark browns, evoking the atmosphere of dusk or early morning.",
    "height": 1264,
    "width": 848,
    "num_inference_steps": 50,
    "guidance_scale": 4.0,
    "use_pe": true
  }'

메모리 제약이 있는 프로세스 내 배포의 경우, 네이티브 PE 디코더는 레이어 단위 offload(layerwise offload)를 지원해요:

sglang serve --model-path baidu/ERNIE-Image \
  --layerwise-offload-components pe \
  --port ${PORT}

이 옵션은 PE 디코더 레이어를 CPU에서 스트리밍하며 프롬프트 강화 지연 시간을 늘릴 수 있어요. --pe-server-url로 외부 PE 서버를 선택한 경우에는 적용되지 않아요.

지원 매트릭스 (Support matrix)

Model Built-in PE SGLang PE Server
ERNIE-Image ✅ ✅

Ascend NPU 환경 (Ascend NPU Environment)

GLM-Image 같은 AR 단계가 있는 확산 모델 문서를 참고해 주세요.

더 알아보기