DMR 예시

DMR 예시

Docker Model Runner를 사용한 완전한 워크플로 예시를 살펴봐요.

출처: 문서

본문

샘플 프로젝트

이제 Docker Model Runner로 구동되는 생성형 AI 애플리케이션을 만들기 시작할 수 있어요.

기존 GenAI 애플리케이션을 시험해 보려면 다음 단계를 따르세요:

1. 샘플 앱을 설정합니다. 다음 저장소를 클론하고 실행해요:

   $ git clone https://github.com/docker/hello-genai.git
   

2. 터미널에서 hello-genai 디렉터리로 이동합니다.
3. run.sh를 실행해 선택한 모델을 풀하고 앱을 실행합니다.
4. 저장소 README에 지정된 주소에서 브라우저로 앱을 엽니다.

GenAI 앱의 인터페이스가 보이면 프롬프트 입력을 시작할 수 있어요.

이제 로컬 모델로 구동되는 여러분만의 GenAI 앱과 상호작용할 수 있어요. 몇 가지 프롬프트를 시도해 보면 응답이 얼마나 빠른지 알 수 있어요. 이 모든 것이 Docker로 여러분의 머신에서 실행되죠.

GitHub Actions에서 Model Runner 사용

다음은 GitHub 워크플로의 일부로 Model Runner를 사용하는 방법의 예시예요. 이 예시는 Model Runner를 설치하고, 설치를 테스트하고, 모델을 풀하고 실행하고, API로 모델과 상호작용하고, 모델을 삭제해요.

dmr-run.yml:

name: Docker Model Runner Example Workflow
permissions:
  contents: read
on:
  workflow_dispatch:
    inputs:
      test_model:
        description: 'Model to test with (default: ai/smollm2:360M-Q4_K_M)'
        required: false
        type: string
        default: 'ai/smollm2:360M-Q4_K_M'
jobs:
  dmr-test:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    steps:
      - name: Set up Docker
        uses: docker/setup-docker-action@v5

      - name: Install docker-model-plugin
        run: |
          echo "Installing docker-model-plugin..."
          # Add Docker's official GPG key:
          sudo apt-get update
          sudo apt-get install ca-certificates curl
          sudo install -m 0755 -d /etc/apt/keyrings
          sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
          sudo chmod a+r /etc/apt/keyrings/docker.asc
          # Add the repository to Apt sources:
          echo \
            "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
            $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
            sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
          sudo apt-get update
          sudo apt-get install -y docker-model-plugin
          echo "Installation completed successfully"

      - name: Test docker model version
        run: |
          echo "Testing docker model version command..."
          sudo docker model version
          # Verify the command returns successfully
          if [ $? -eq 0 ]; then
            echo "✅ docker model version command works correctly"
          else
            echo "❌ docker model version command failed"
            exit 1
          fi

      - name: Pull the provided model and run it
        run: |
          MODEL="${{ github.event.inputs.test_model || 'ai/smollm2:360M-Q4_K_M' }}"
          echo "Testing with model: $MODEL"
          # Test model pull
          echo "Pulling model..."
          sudo docker model pull "$MODEL"
          if [ $? -eq 0 ]; then
            echo "✅ Model pull successful"
          else
            echo "❌ Model pull failed"
            exit 1
          fi
          # Test basic model run (with timeout to avoid hanging)
          echo "Testing docker model run..."
          timeout 60s sudo docker model run "$MODEL" "Give me a fact about whales." || {
            exit_code=$?
            if [ $exit_code -eq 124 ]; then
              echo "✅ Model run test completed (timed out as expected for non-interactive test)"
            else
              echo "❌ Model run failed with exit code: $exit_code"
              exit 1
            fi
          }

      - name: Test API endpoint
        run: |
          MODEL="${{ github.event.inputs.test_model || 'ai/smollm2:360M-Q4_K_M' }}"
          echo "Testing API endpoint with model: $MODEL"
          # Test API call with curl
          echo "Testing API call..."
          RESPONSE=$(curl -s http://localhost:12434/engines/llama.cpp/v1/chat/completions \
            -H "Content-Type: application/json" \
            -d "{
              \"model\": \"$MODEL\",
              \"messages\": [
                {
                  \"role\": \"user\",
                  \"content\": \"Say hello\"
                }
              ],
              \"top_k\": 1,
              \"temperature\": 0
            }")
          if [ $? -eq 0 ]; then
            echo "✅ API call successful"
            echo "Response received: $RESPONSE"
            # Check if response contains "hello" (case-insensitive)
            if echo "$RESPONSE" | grep -qi "hello"; then
              echo "✅ Response contains 'hello' (case-insensitive)"
            else
              echo "❌ Response does not contain 'hello'"
              echo "Full response: $RESPONSE"
              exit 1
            fi
          else
            echo "❌ API call failed"
            exit 1
          fi

      - name: Test model cleanup
        run: |
          MODEL="${{ github.event.inputs.test_model || 'ai/smollm2:360M-Q4_K_M' }}"
          echo "Cleaning up test model..."
          sudo docker model rm "$MODEL" || echo "Model removal failed or model not found"
          # Verify model was removed
          echo "Verifying model cleanup..."
          sudo docker model ls
          echo "✅ Model cleanup completed"

      - name: Report success
        if: success()
        run: |
          echo "🎉 Docker Model Runner daily health check completed successfully!"
          echo "All tests passed:"
          echo " ✅ docker-model-plugin installation successful"
          echo " ✅ docker model version command working"
          echo " ✅ Model pull and run operations successful"
          echo " ✅ API endpoint operations successful"
          echo " ✅ Cleanup operations successful"

관련 페이지

• Models and Compose

더 알아보기 (Learn more)