본문 바로가기
WIKI 기술 지식 베이스

RAG using DeepSeek-R1 in Amazon SageMaker

원문 보기 위키 갱신

출처: 문서

본문

Amazon SageMaker에서 DeepSeek-R1을 사용한 RAG

이 튜토리얼에서는 Amazon OpenSearch Service와 DeepSeek-R1 모델을 사용해 검색 증강 생성(RAG)을 구현하는 방법을 보여 줘요.

Amazon OpenSearch Service 대신 자체 관리형 OpenSearch를 사용하는 경우에는 블루프린트를 사용해 DeepSeek-R1 모델에 대한 커넥터를 생성하세요. 커넥터 생성에 대해 자세히 알아보려면 커넥터를 참고하세요. 그런 다음 바로 4단계로 이동하세요.

your_ 접두사로 시작하는 플레이스홀더는 여러분의 값으로 바꾸세요. {: .note}

사전 준비 사항

시작하기 전에 다음 사전 준비 사항을 충족하세요.

Amazon 설정을 구성할 때는 이 튜토리얼에서 언급한 값만 변경하세요. 그 외의 모든 설정은 기본값 그대로 두세요. {: .important}

Amazon SageMaker에 DeepSeek-R1 배포

이 블로그 포스트의 안내에 따라 DeepSeek-R1 모델을 Amazon SageMaker에 배포하세요.

Amazon SageMaker DeepSeek-R1 모델의 Amazon Resource Name(ARN)과 URL을 기록해 두세요. 이후 단계에서 사용하게 돼요.

OpenSearch 클러스터 생성

Amazon OpenSearch Service 콘솔로 이동해 OpenSearch 도메인을 생성하세요.

도메인 ARN과 URL을 기록해 두세요. 이후 단계에서 사용하게 돼요.

1단계: Amazon SageMaker 액세스용 IAM 역할 생성

Amazon SageMaker에서 DeepSeek-R1 모델을 호출하려면 적절한 권한을 가진 AWS Identity and Access Management(IAM) 역할을 만들어야 해요. 커넥터는 이 역할을 사용해 모델을 호출해요.

IAM 콘솔로 이동해 my_invoke_sagemaker_deepseek_model_role이라는 이름의 새 IAM 역할을 만들고, 다음 신뢰 정책과 권한을 추가하세요:

  • 사용자 지정 신뢰 정책:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": "es.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

{% include copy.html %}

  • 권한:
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "sagemaker:InvokeEndpoint"
            ],
            "Resource": [
                "your_sagemaker_model_inference_endpoint_arn"
            ]
        }
    ]
}

{% include copy.html %}

역할 ARN을 기록해 두세요. 이후 단계에서 사용하게 돼요.

2단계: Amazon OpenSearch Service에서 IAM 역할 구성

다음 단계에 따라 Amazon OpenSearch Service에서 IAM 역할을 구성하세요.

2.1단계: 커넥터 요청 서명용 IAM 역할 생성

Create Connector API 요청에 서명하기 위한 새 IAM 역할을 생성하세요.

my_create_sagemaker_deepseek_connector_role이라는 이름의 IAM 역할을 다음 신뢰 정책과 권한으로 생성하세요:

  • 사용자 지정 신뢰 정책:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "your_iam_user_arn"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

{% include copy.html %}

3단계에서 your_iam_user_arn IAM 사용자를 사용해 이 역할을 수임하게 돼요.

  • 권한:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "iam:PassRole",
      "Resource": "your_iam_role_arn_created_in_step1"
    },
    {
      "Effect": "Allow",
      "Action": "es:ESHttpPost",
      "Resource": "your_opensearch_domain_arn"
    }
  ]
}

{% include copy.html %}

이 역할 ARN을 기록해 두세요. 이후 단계에서 사용하게 돼요.

2.2단계: 백엔드 역할 매핑

다음 단계에 따라 백엔드 역할을 매핑하세요:

  1. OpenSearch Dashboards에 로그인하고 상단 메뉴에서 Security를 선택하세요.
  2. Roles를 선택한 다음 ml_full_access 역할을 선택하세요.
  3. ml_full_access 역할 세부 정보 페이지에서 Mapped users를 선택한 다음 Manage mapping을 선택하세요.
  4. 다음 이미지와 같이 2.1단계에서 생성한 IAM 역할 ARN을 Backend roles 필드에 입력하세요. 백엔드 역할 매핑
  5. Map을 선택하세요.

이제 OpenSearch 클러스터에 IAM 역할이 성공적으로 구성됐어요.

3단계: 커넥터 생성

다음 단계에 따라 DeepSeek-R1 모델용 커넥터를 생성하세요. 커넥터 생성에 대해 자세히 알아보려면 커넥터를 참고하세요.

다음 Python 코드를 일회성 임시 자격 증명과 함께 실행하세요.

import boto3
import requests 
from requests_aws4auth import AWS4Auth

host = 'your_amazon_opensearch_domain_endpoint'
region = 'your_amazon_opensearch_domain_region'
service = 'es'

assume_role_response = boto3.Session().client('sts').assume_role(
  RoleArn="your_iam_role_arn_created_in_step2.1",
  RoleSessionName="your_session_name"
)
credentials = assume_role_response["Credentials"]
awsauth = AWS4Auth(credentials["AccessKeyId"], credentials["SecretAccessKey"], region, service, session_token=credentials["SessionToken"])


path = '/_plugins/_ml/connectors/_create'
url = host + path

payload = {
  "name": "DeepSeek R1 model connector",
  "description": "Connector for my Sagemaker DeepSeek model",
  "version": "1.0",
  "protocol": "aws_sigv4",
  "credential": {
    "roleArn": "your_iam_role_arn_created_in_step1"
  },
  "parameters": {
    "service_name": "sagemaker",
    "region": "your_sagemaker_model_region",
    "do_sample": true,
    "top_p": 0.9,
    "temperature": 0.7,
    "max_new_tokens": 512
  },
  "actions": [
    {
      "action_type": "PREDICT",
      "method": "POST",
      "url": "your_sagemaker_model_inference_endpoint",
      "headers": {
        "content-type": "application/json"
      },
      "request_body": "{ \"inputs\": \"${parameters.inputs}\", \"parameters\": {\"do_sample\": ${parameters.do_sample}, \"top_p\": ${parameters.top_p}, \"temperature\": ${parameters.temperature}, \"max_new_tokens\": ${parameters.max_new_tokens}} }",
      "post_process_function": "\n      if (params.result == null || params.result.length == 0) {\n        throw new Exception('No response available');\n      }\n      \n      def completion = params.result[0].generated_text;\n      return '{' +\n               '\"name\": \"response\",'+\n               '\"dataAsMap\": {' +\n                  '\"completion\":\"' + escape(completion) + '\"}' +\n             '}';\n    "
    }
  ]
}

headers = {"Content-Type": "application/json"}

r = requests.post(url, auth=awsauth, json=payload, headers=headers)
print(r.status_code)
print(r.text)

{% include copy.html %}

스크립트는 커넥터 ID를 출력해요:

{"connector_id":"HnS5sJQBVQUimUskjpFl"}

커넥터 ID를 기록해 두세요. 다음 단계에서 사용하게 돼요.

4단계: 모델 생성 및 테스트

OpenSearch Dashboards에 로그인하고 DevTools 콘솔을 연 다음, 다음 요청을 실행해 DeepSeek-R1 모델을 생성하고 테스트하세요.

  1. 모델 그룹을 생성하세요:

    POST /_plugins/_ml/model_groups/_register
{
    "name": "Sagemaker DeepSeek model",
    "description": "Test model group for Sagemaker DeepSeek model"
}

    {% include copy-curl.html %}

    응답에 모델 그룹 ID가 포함돼요:

    {
  "model_group_id": "H3S8sJQBVQUimUskW5Fm",
  "status": "CREATED"
}

  2. 모델을 등록하세요:

    POST /_plugins/_ml/models/_register
{
  "name": "Sagemaker DeepSeek R1 model",
  "function_name": "remote",
  "description": "DeepSeek R1 model on Sagemaker",
  "model_group_id": "H3S8sJQBVQUimUskW5Fm",
  "connector_id": "HnS5sJQBVQUimUskjpFl"
}

    {% include copy-curl.html %}

    응답에 모델 ID가 포함돼요:

    {
  "task_id": "Sim9sJQBts7fa6byEh1S",
  "status": "CREATED",
  "model_id": "Sym9sJQBts7fa6byEh1-"
}

  3. 모델을 배포하세요:

    POST /_plugins/_ml/models/Sym9sJQBts7fa6byEh1-/_deploy

    {% include copy-curl.html %}

    응답에 배포 작업의 task ID가 포함돼요:

    {
  "task_id": "TCm9sJQBts7fa6byex2j",
  "task_type": "DEPLOY_MODEL",
  "status": "COMPLETED"
}

  4. 모델을 테스트하세요:

    POST /_plugins/_ml/models/Sym9sJQBts7fa6byEh1-/_predict
{
  "parameters": {
    "inputs": "hello"
  }
}

    {% include copy-curl.html %}

    응답에 모델이 생성한 텍스트가 포함돼요:

    {
  "inference_results": [
    {
      "output": [
        {
          "name": "response",
          "dataAsMap": {
            "response": [
              {
                "generated_text": """hello<think>

</think>

Hello! How can I assist you today? 😊"""
              }
            ]
          }
        }
      ],
      "status_code": 200
    }
  ]
}

5단계: RAG 구성

다음 단계에 따라 RAG를 구성하세요.

5.1단계: 검색 파이프라인 생성

RAG 프로세서를 포함하는 검색 파이프라인을 생성하세요:

PUT /_search/pipeline/my-conversation-search-pipeline-deepseek
{
  "response_processors": [
    {
      "retrieval_augmented_generation": {
        "tag": "Demo pipeline",
        "description": "Demo pipeline Using DeepSeek R1",
        "model_id": "Sym9sJQBts7fa6byEh1-",
        "context_field_list": [
          "text"
        ],
        "system_prompt": "You are a helpful assistant.",
        "user_instructions": "Generate a concise and informative answer in less than 100 words for the given question"
      }
    }
  ]
}

{% include copy-curl.html %}

5.2단계: 벡터 데이터베이스 생성

이 튜토리얼의 1, 2단계를 따라 임베딩 모델과 벡터 인덱스를 생성하세요. 그런 다음 인덱스에 샘플 데이터를 적재하세요:

POST _bulk
{"index": {"_index": "my-nlp-index", "_id": "1"}}
{"text": "Chart and table of population level and growth rate for the Ogden-Layton metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\nThe current metro area population of Ogden-Layton in 2023 is 750,000, a 1.63% increase from 2022.\nThe metro area population of Ogden-Layton in 2022 was 738,000, a 1.79% increase from 2021.\nThe metro area population of Ogden-Layton in 2021 was 725,000, a 1.97% increase from 2020.\nThe metro area population of Ogden-Layton in 2020 was 711,000, a 2.16% increase from 2019."}
{"index": {"_index": "my-nlp-index", "_id": "2"}}
{"text": "Chart and table of population level and growth rate for the New York City metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of New York City in 2023 is 18,937,000, a 0.37% increase from 2022.\\nThe metro area population of New York City in 2022 was 18,867,000, a 0.23% increase from 2021.\\nThe metro area population of New York City in 2021 was 18,823,000, a 0.1% increase from 2020.\\nThe metro area population of New York City in 2020 was 18,804,000, a 0.01% decline from 2019."}
{"index": {"_index": "my-nlp-index", "_id": "3"}}
{"text": "Chart and table of population level and growth rate for the Chicago metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Chicago in 2023 is 8,937,000, a 0.4% increase from 2022.\\nThe metro area population of Chicago in 2022 was 8,901,000, a 0.27% increase from 2021.\\nThe metro area population of Chicago in 2021 was 8,877,000, a 0.14% increase from 2020.\\nThe metro area population of Chicago in 2020 was 8,865,000, a 0.03% increase from 2019."}
{"index": {"_index": "my-nlp-index", "_id": "4"}}
{"text": "Chart and table of population level and growth rate for the Miami metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Miami in 2023 is 6,265,000, a 0.8% increase from 2022.\\nThe metro area population of Miami in 2022 was 6,215,000, a 0.78% increase from 2021.\\nThe metro area population of Miami in 2021 was 6,167,000, a 0.74% increase from 2020.\\nThe metro area population of Miami in 2020 was 6,122,000, a 0.71% increase from 2019."}
{"index": {"_index": "my-nlp-index", "_id": "5"}}
{"text": "Chart and table of population level and growth rate for the Austin metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Austin in 2023 is 2,228,000, a 2.39% increase from 2022.\\nThe metro area population of Austin in 2022 was 2,176,000, a 2.79% increase from 2021.\\nThe metro area population of Austin in 2021 was 2,117,000, a 3.12% increase from 2020.\\nThe metro area population of Austin in 2020 was 2,053,000, a 3.43% increase from 2019."}
{"index": {"_index": "my-nlp-index", "_id": "6"}}
{"text": "Chart and table of population level and growth rate for the Seattle metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Seattle in 2023 is 3,519,000, a 0.86% increase from 2022.\\nThe metro area population of Seattle in 2022 was 3,489,000, a 0.81% increase from 2021.\\nThe metro area population of Seattle in 2021 was 3,461,000, a 0.82% increase from 2020.\\nThe metro area population of Seattle in 2020 was 3,433,000, a 0.79% increase from 2019."}

{% include copy-curl.html %}

5.3단계: 인덱스 검색

벡터 검색을 실행해 벡터 데이터베이스에서 문서를 가져오고, RAG에 DeepSeek 모델을 사용하세요:

GET /my-nlp-index/_search?search_pipeline=my-conversation-search-pipeline-deepseek
{
  "query": {
    "neural": {
      "passage_embedding": {
        "query_text": "What's the population increase of New York City from 2021 to 2023? How is the trending comparing with Miami?",
        "model_id": "USkHsZQBts7fa6bybx3G",
        "k": 5
      }
    }
  },
  "size": 4,
  "_source": [
    "text"
  ],
  "ext": {
    "generative_qa_parameters": {
      "llm_model": "bedrock/claude",
      "llm_question": "What's the population increase of New York City from 2021 to 2023? How is the trending comparing with Miami?",
      "context_size": 5,
      "timeout": 15
    }
  }
}

{% include copy-curl.html %}

응답에는 벡터 검색에서 가져온 관련 문서(hits 배열)와 DeepSeek 모델이 생성한 답변(ext.retrieval_augmented_generation 객체)이 모두 포함돼요:

{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 6,
      "relation": "eq"
    },
    "max_score": 0.05248103,
    "hits": [
      {
        "_index": "my-nlp-index",
        "_id": "2",
        "_score": 0.05248103,
        "_source": {
          "text": """Chart and table of population level and growth rate for the New York City metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\nThe current metro area population of New York City in 2023 is 18,937,000, a 0.37% increase from 2022.\nThe metro area population of New York City in 2022 was 18,867,000, a 0.23% increase from 2021.\nThe metro area population of New York City in 2021 was 18,823,000, a 0.1% increase from 2020.\nThe metro area population of New York City in 2020 was 18,804,000, a 0.01% decline from 2019."""
        }
      },
      {
        "_index": "my-nlp-index",
        "_id": "4",
        "_score": 0.029023321,
        "_source": {
          "text": """Chart and table of population level and growth rate for the Miami metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\nThe current metro area population of Miami in 2023 is 6,265,000, a 0.8% increase from 2022.\nThe metro area population of Miami in 2022 was 6,215,000, a 0.78% increase from 2021.\nThe metro area population of Miami in 2021 was 6,167,000, a 0.74% increase from 2020.\nThe metro area population of Miami in 2020 was 6,122,000, a 0.71% increase from 2019."""
        }
      },
      {
        "_index": "my-nlp-index",
        "_id": "3",
        "_score": 0.028097045,
        "_source": {
          "text": """Chart and table of population level and growth rate for the Chicago metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\nThe current metro area population of Chicago in 2023 is 8,937,000, a 0.4% increase from 2022.\nThe metro area population of Chicago in 2022 was 8,901,000, a 0.27% increase from 2021.\nThe metro area population of Chicago in 2021 was 8,877,000, a 0.14% increase from 2020.\nThe metro area population of Chicago in 2020 was 8,865,000, a 0.03% increase from 2019."""
        }
      },
      {
        "_index": "my-nlp-index",
        "_id": "6",
        "_score": 0.026973149,
        "_source": {
          "text": """Chart and table of population level and growth rate for the Seattle metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\nThe current metro area population of Seattle in 2023 is 3,519,000, a 0.86% increase from 2022.\nThe metro area population of Seattle in 2022 was 3,489,000, a 0.81% increase from 2021.\nThe metro area population of Seattle in 2021 was 3,461,000, a 0.82% increase from 2020.\nThe metro area population of Seattle in 2020 was 3,433,000, a 0.79% increase from 2019."""
        }
      }
    ]
  },
  "ext": {
    "retrieval_augmented_generation": {
      "answer": """You are a helpful assistant.\nGenerate a concise and informative answer in less than 100 words for the given question\nSEARCH RESULT 1: Chart and table of population level and growth rate for the New York City metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\nThe current metro area population of New York City in 2023 is 18,937,000, a 0.37% increase from 2022.\nThe metro area population of New York City in 2022 was 18,867,000, a 0.23% increase from 2021.\nThe metro area population of New York City in 2021 was 18,823,000, a 0.1% increase from 2020.\nThe metro area population of New York City in 2020 was 18,804,000, a 0.01% decline from 2019.\nSEARCH RESULT 2: Chart and table of population level and growth rate for the Miami metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\nThe current metro area population of Miami in 2023 is 6,265,000, a 0.8% increase from 2022.\nThe metro area population of Miami in 2022 was 6,215,000, a 0.78% increase from 2021.\nThe metro area population of Miami in 2021 was 6,167,000, a 0.74% increase from 2020.\nThe metro area population of Miami in 2020 was 6,122,000, a 0.71% increase from 2019.\nSEARCH RESULT 3: Chart and table of population level and growth rate for the Chicago metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\nThe current metro area population of Chicago in 2023 is 8,937,000, a 0.4% increase from 2022.\nThe metro area population of Chicago in 2022 was 8,901,000, a 0.27% increase from 2021.\nThe metro area population of Chicago in 2021 was 8,877,000, a 0.14% increase from 2020.\nThe metro area population of Chicago in 2020 was 8,865,000, a 0.03% increase from 2019.\nSEARCH RESULT 4: Chart and table of population level and growth rate for the Seattle metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\nThe current metro area population of Seattle in 2023 is 3,519,000, a 0.86% increase from 2022.\nThe metro area population of Seattle in 2022 was 3,489,000, a 0.81% increase from 2021.\nThe metro area population of Seattle in 2021 was 3,461,000, a 0.82% increase from 2020.\nThe metro area population of Seattle in 2020 was 3,433,000, a 0.79% increase from 2019.\nQUESTION: What's the population increase of New York City from 2021 to 2023? How is the trending comparing with Miami\nAlright, let's tackle this question step by step. The user is asking for the population increase of New York City from 2021 to 2023 and how this trend compares to Miami's. 

First, I'll look through the search results to find the relevant data. From SEARCH RESULT 1, I see the populations for NYC in 2021, 2022, and 2023. In 2021, it was 18,823,000, and by 2023, it's 18,937,000. That's an increase of 114,000 over two years.

Next, I'll calculate the annual growth rates. From 2021 to 2022, the growth rate was 0.23%, and from 2022 to 2023, it's 0.37%. So, the trend shows an increase in the growth rate each year.

Now, looking at Miami in SEARCH RESULT 2, the population in 2021 was 6,167,000, and in 2023, it's 6,265,000. That's an increase of 98,000 over the same period. The growth rates were 0.74% in 2021-2022 and 0.8% in 2022-2023, also showing an increasing trend but at a higher rate than NYC.

Putting it all together, NYC's population increased by 114,000 with growth rates rising from 0.23% to 0.37%. Miami saw a slightly smaller increase of 98,000 but with higher growth rates, from 0.74% to 0.8%. So, Miami's growth is both higher in absolute terms and has a faster increasing rate compared to NYC.
</think>

The population of New York City increased by 114,000 from 2021 to 2023. The growth rate rose from 0.1% in 2021 to 0.37% in 2023. Comparatively, Miami's population increased by 98,000 during the same period"""
    }
  }
}

더 알아보기 (Learn more)

함께 읽는 실무 이야기

우리 조직에도 적용하고 싶다면, AI 시스템 구축 서비스를 살펴보세요.