RAG chatbot
출처: 문서
본문
RAG 챗봇
대규모 언어 모델(LLM)의 잘 알려진 한계 중 하나는 지식 기반이 모델이 학습된 시기의 정보만 담고 있다는 거예요. LLM은 최근 사건이나 여러분의 내부 데이터에 대한 지식이 없죠. 검색 증강 생성(RAG)을 사용하면 LLM 지식 기반을 보강할 수 있어요.
이 튜토리얼에서는 에이전트와 도구와 RAG를 사용해 여러분만의 챗봇을 만드는 방법을 알아봐요. RAG는 OpenSearch 인덱스에 담긴 정보로 LLM 지식 기반을 보완해요.
your_ 접두사로 시작하는 플레이스홀더는 여러분의 값으로 바꾸세요.
{: .note}
사전 준비
사전 요건을 충족하고, conversational flow agent를 사용한 RAG 튜토리얼의 1단계를 따라 미국 도시 인구 데이터가 담긴 test_population_data 지식 기반 인덱스를 설정하세요.
임베딩 모델 ID를 기억해 두세요. 다음 단계에서 사용하게 돼요.
1단계: 지식 기반 설정
먼저 인제스트 파이프라인을 생성하세요:
PUT /_ingest/pipeline/test_tech_news_pipeline
{
"description": "text embedding pipeline for tech news",
"processors": [
{
"text_embedding": {
"model_id": "your_text_embedding_model_id",
"field_map": {
"passage": "passage_embedding"
}
}
}
]
}
{% include copy-curl.html %}
다음으로, 최신 기술 뉴스가 담긴 test_tech_news라는 이름의 인덱스를 생성하세요:
PUT test_tech_news
{
"mappings": {
"properties": {
"passage": {
"type": "text"
},
"passage_embedding": {
"type": "knn_vector",
"dimension": 384
}
}
},
"settings": {
"index": {
"knn.space_type": "cosinesimil",
"default_pipeline": "test_tech_news_pipeline",
"knn": "true"
}
}
}
{% include copy-curl.html %}
인덱스에 데이터를 색인하세요:
POST _bulk
{"index":{"_index":"test_tech_news"}}
{"c":"Apple Vision Pro is a mixed-reality headset developed by Apple Inc. It was announced on June 5, 2023, at Apple's Worldwide Developers Conference, and pre-orders began on January 19, 2024. It became available for purchase on February 2, 2024, in the United States.[10] A worldwide launch has yet to be scheduled. The Vision Pro is Apple's first new major product category since the release of the Apple Watch in 2015.[11]\n\nApple markets the Vision Pro as a \"spatial computer\" where digital media is integrated with the real world. Physical inputs—such as motion gestures, eye tracking, and speech recognition—can be used to interact with the system.[10] Apple has avoided marketing the device as a virtual reality headset, along with the use of the terms \"virtual reality\" and \"augmented reality\" when discussing the product in presentations and marketing.[12]\n\nThe device runs visionOS,[13] a mixed-reality operating system derived from iOS frameworks using a 3D user interface; it supports multitasking via windows that appear to float within the user's surroundings,[14] as seen by cameras built into the headset. A dial on the top of the headset can be used to mask the camera feed with a virtual environment to increase immersion. The OS supports avatars (officially called \"Personas\"), which are generated by scanning the user's face; a screen on the front of the headset displays a rendering of the avatar's eyes (\"EyeSight\"), which are used to indicate the user's level of immersion to bystanders, and assist in communication.[15]"}
{"index":{"_index":"test_tech_news"}}
{"passage":"LLaMA (Large Language Model Meta AI) is a family of autoregressive large language models (LLMs), released by Meta AI starting in February 2023.\n\nFor the first version of LLaMA, four model sizes were trained: 7, 13, 33, and 65 billion parameters. LLaMA's developers reported that the 13B parameter model's performance on most NLP benchmarks exceeded that of the much larger GPT-3 (with 175B parameters) and that the largest model was competitive with state of the art models such as PaLM and Chinchilla.[1] Whereas the most powerful LLMs have generally been accessible only through limited APIs (if at all), Meta released LLaMA's model weights to the research community under a noncommercial license.[2] Within a week of LLaMA's release, its weights were leaked to the public on 4chan via BitTorrent.[3]\n\nIn July 2023, Meta released several models as Llama 2, using 7, 13 and 70 billion parameters.\n\nLLaMA-2\n\nOn July 18, 2023, in partnership with Microsoft, Meta announced LLaMA-2, the next generation of LLaMA. Meta trained and released LLaMA-2 in three model sizes: 7, 13, and 70 billion parameters.[4] The model architecture remains largely unchanged from that of LLaMA-1 models, but 40% more data was used to train the foundational models.[5] The accompanying preprint[5] also mentions a model with 34B parameters that might be released in the future upon satisfying safety targets.\n\nLLaMA-2 includes both foundational models and models fine-tuned for dialog, called LLaMA-2 Chat. In further departure from LLaMA-1, all models are released with weights, and are free for many commercial use cases. However, due to some remaining restrictions, the description of LLaMA as open source has been disputed by the Open Source Initiative (known for maintaining the Open Source Definition).[6]\n\nIn November 2023, research conducted by Patronus AI, an artificial intelligence startup company, compared performance of LLaMA-2, OpenAI's GPT-4 and GPT-4-Turbo, and Anthropic's Claude2 on two versions of a 150-question test about information in SEC filings (e.g. Form 10-K, Form 10-Q, Form 8-K, earnings reports, earnings call transcripts) submitted by public companies to the agency where one version of the test required the generative AI models to use a retrieval system to locate the specific SEC filing to answer the questions while the other version provided the specific SEC filing to the models to answer the question (i.e. in a long context window). On the retrieval system version, GPT-4-Turbo and LLaMA-2 both failed to produce correct answers to 81% of the questions, while on the long context window version, GPT-4-Turbo and Claude-2 failed to produce correct answers to 21% and 24% of the questions respectively.[7][8]"}
{"index":{"_index":"test_tech_news"}}
{"passage":"Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon via a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. Using Amazon Bedrock, you can easily experiment with and evaluate top FMs for your use case, privately customize them with your data using techniques such as fine-tuning and Retrieval Augmented Generation (RAG), and build agents that execute tasks using your enterprise systems and data sources. Since Amazon Bedrock is serverless, you don't have to manage any infrastructure, and you can securely integrate and deploy generative AI capabilities into your applications using the AWS services you are already familiar with."}
{% include copy-curl.html %}
2단계: LLM 준비
conversational flow agent를 사용한 RAG 튜토리얼의 2단계를 따라 Amazon Bedrock Claude 모델을 구성하세요.
모델 ID를 기억해 두세요. 다음 단계에서 사용하게 돼요.
3단계: 에이전트 생성
이 튜토리얼에서는 conversational 타입의 에이전트를 만들 거예요.
conversational_flow 에이전트와 conversational 에이전트 모두 대화 기록을 지원해요.
conversational_flow 에이전트와 conversational 에이전트는 다음과 같은 점에서 달라요:
conversational_flow에이전트는 미리 정해진 순서대로 도구를 순차적으로 실행해요.conversational에이전트는 다음에 실행할 도구를 동적으로 선택해요.
이 튜토리얼의 에이전트는 두 가지 도구를 포함해요. 하나는 최신 인구 데이터를 제공하고, 다른 하나는 기술 뉴스를 담고 있죠.
에이전트에는 다음 파라미터가 있어요:
"max_iteration": 5: 에이전트가 LLM을 최대 다섯 번 실행해요."response_filter": "$.completion": Amazon Bedrock Claude 모델 응답에서 LLM 답변을 가져오는 데 필요해요."doc_size": 3(population_data_knowledge_base에 설정): 상위 세 개 문서를 반환하도록 지정해요.
앞서 설명한 사양으로 에이전트를 생성하세요:
POST _plugins/_ml/agents/_register
{
"name": "Chat Agent with RAG",
"type": "conversational",
"description": "this is a test agent",
"llm": {
"model_id": "your_llm_model_id",
"parameters": {
"max_iteration": 5,
"response_filter": "$.completion"
}
},
"memory": {
"type": "conversation_index"
},
"tools": [
{
"type": "VectorDBTool",
"name": "population_data_knowledge_base",
"description": "This tool provides population data of US cities.",
"parameters": {
"input": "${parameters.question}",
"index": "test_population_data",
"source_field": [
"population_description"
],
"model_id": "your_text_embedding_model_id",
"embedding_field": "population_description_embedding",
"doc_size": 3
}
},
{
"type": "VectorDBTool",
"name": "tech_news_knowledge_base",
"description": "This tool provides recent tech news.",
"parameters": {
"input": "${parameters.question}",
"index": "test_tech_news",
"source_field": [
"passage"
],
"model_id": "your_text_embedding_model_id",
"embedding_field": "passage_embedding",
"doc_size": 2
}
}
],
"app_type": "chat_with_rag"
}
{% include copy-curl.html %}
에이전트 ID를 기억해 두세요. 다음 단계에서 사용하게 돼요.
4단계: 에이전트 테스트
conversational 에이전트는 verbose 옵션을 지원해요. verbose를 true로 설정하면 자세한 단계를 확인할 수 있죠.
또는 Get Message Traces API를 호출할 수도 있어요:
GET _plugins/_ml/memory/message/message_id/traces
{% include copy-curl.html %}
대화 시작하기
기술 뉴스와 관련된 질문을 해 보세요:
POST _plugins/_ml/agents/your_agent_id/_execute
{
"parameters": {
"question": "What's vision pro",
"verbose": true
}
}
{% include copy-curl.html %}
응답을 보면 에이전트가 tech_news_knowledge_base 도구를 실행해 상위 두 개 문서를 가져오는 걸 알 수 있어요. 그런 다음 에이전트는 이 문서들을 컨텍스트로 LLM에 전달하고, LLM은 그 컨텍스트를 사용해 답변을 만들어요:
{
"inference_results": [
{
"output": [
{
"name": "memory_id",
"result": "eLVSxI0B8vrNLhb9nxto"
},
{
"name": "parent_interaction_id",
"result": "ebVSxI0B8vrNLhb9nxty"
},
{
"name": "response",
"result": """{
"thought": "I don't have enough context to answer the question directly. Let me check the tech_news_knowledge_base tool to see if it can provide more information.",
"action": "tech_news_knowledge_base",
"action_input": "{\"query\":\"What's vision pro\"}"
}"""
},
{
"name": "response",
"result": """{"_index":"test_tech_news","_source":{"passage":"Apple Vision Pro is a mixed-reality headset developed by Apple Inc. It was announced on June 5, 2023, at Apple\u0027s Worldwide Developers Conference, and pre-orders began on January 19, 2024. It became available for purchase on February 2, 2024, in the United States.[10] A worldwide launch has yet to be scheduled. The Vision Pro is Apple\u0027s first new major product category since the release of the Apple Watch in 2015.[11]\n\nApple markets the Vision Pro as a \"spatial computer\" where digital media is integrated with the real world. Physical inputs—such as motion gestures, eye tracking, and speech recognition—can be used to interact with the system.[10] Apple has avoided marketing the device as a virtual reality headset, along with the use of the terms \"virtual reality\" and \"augmented reality\" when discussing the product in presentations and marketing.[12]\n\nThe device runs visionOS,[13] a mixed-reality operating system derived from iOS frameworks using a 3D user interface; it supports multitasking via windows that appear to float within the user\u0027s surroundings,[14] as seen by cameras built into the headset. A dial on the top of the headset can be used to mask the camera feed with a virtual environment to increase immersion. The OS supports avatars (officially called \"Personas\"), which are generated by scanning the user\u0027s face; a screen on the front of the headset displays a rendering of the avatar\u0027s eyes (\"EyeSight\"), which are used to indicate the user\u0027s level of immersion to bystanders, and assist in communication.[15]"},"_id":"lrU8xI0B8vrNLhb9yBpV","_score":0.6700683}
{"_index":"test_tech_news","_source":{"passage":"Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon via a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. Using Amazon Bedrock, you can easily experiment with and evaluate top FMs for your use case, privately customize them with your data using techniques such as fine-tuning and Retrieval Augmented Generation (RAG), and build agents that execute tasks using your enterprise systems and data sources. Since Amazon Bedrock is serverless, you don\u0027t have to manage any infrastructure, and you can securely integrate and deploy generative AI capabilities into your applications using the AWS services you are already familiar with."},"_id":"mLU8xI0B8vrNLhb9yBpV","_score":0.5604863}
"""
},
{
"name": "response",
"result": "Vision Pro is a mixed-reality headset developed by Apple that was announced in 2023. It uses cameras and sensors to overlay digital objects and information on the real world. The device runs an operating system called visionOS that allows users to interact with windows and apps in a 3D environment using gestures, eye tracking, and voice commands."
}
]
}
]
}
Get Traces API를 사용하면 자세한 단계를 추적할 수 있어요:
GET _plugins/_ml/memory/message/ebVSxI0B8vrNLhb9nxty/traces
{% include copy-curl.html %}
인구 데이터와 관련된 질문을 해 보세요:
POST _plugins/_ml/agents/your_agent_id/_execute
{
"parameters": {
"question": "What's the population of Seattle 2023",
"verbose": true
}
}
{% include copy-curl.html %}
응답을 보면 에이전트가 population_data_knowledge_base 도구를 실행해 상위 세 개 문서를 가져오는 걸 알 수 있어요. 그런 다음 에이전트는 이 문서들을 컨텍스트로 LLM에 전달하고, LLM은 그 컨텍스트를 사용해 답변을 만들어요:
{
"inference_results": [
{
"output": [
{
"name": "memory_id",
"result": "l7VUxI0B8vrNLhb9sRuQ"
},
{
"name": "parent_interaction_id",
"result": "mLVUxI0B8vrNLhb9sRub"
},
{
"name": "response",
"result": """{
"thought": "Let me check the population data tool to find the most recent population estimate for Seattle",
"action": "population_data_knowledge_base",
"action_input": "{\"city\":\"Seattle\"}"
}"""
},
{
"name": "response",
"result": """{"_index":"test_population_data","_source":{"population_description":"Chart and table of population level and growth rate for the Seattle metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Seattle in 2023 is 3,519,000, a 0.86% increase from 2022.\\nThe metro area population of Seattle in 2022 was 3,489,000, a 0.81% increase from 2021.\\nThe metro area population of Seattle in 2021 was 3,461,000, a 0.82% increase from 2020.\\nThe metro area population of Seattle in 2020 was 3,433,000, a 0.79% increase from 2019."},"_id":"BxF5vo0BubpYKX5ER0fT","_score":0.65775126}
{"_index":"test_population_data","_source":{"population_description":"Chart and table of population level and growth rate for the Seattle metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Seattle in 2023 is 3,519,000, a 0.86% increase from 2022.\\nThe metro area population of Seattle in 2022 was 3,489,000, a 0.81% increase from 2021.\\nThe metro area population of Seattle in 2021 was 3,461,000, a 0.82% increase from 2020.\\nThe metro area population of Seattle in 2020 was 3,433,000, a 0.79% increase from 2019."},"_id":"7DrZvo0BVR2NrurbRIAE","_score":0.65775126}
{"_index":"test_population_data","_source":{"population_description":"Chart and table of population level and growth rate for the New York City metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of New York City in 2023 is 18,937,000, a 0.37% increase from 2022.\\nThe metro area population of New York City in 2022 was 18,867,000, a 0.23% increase from 2021.\\nThe metro area population of New York City in 2021 was 18,823,000, a 0.1% increase from 2020.\\nThe metro area population of New York City in 2020 was 18,804,000, a 0.01% decline from 2019."},"_id":"AxF5vo0BubpYKX5ER0fT","_score":0.56461215}
"""
},
{
"name": "response",
"result": "According to the population data tool, the population of Seattle in 2023 is approximately 3,519,000 people, a 0.86% increase from 2022."
}
]
}
]
}
대화 이어서 진행하기
이전 대화를 이어서 진행하려면 memory_id 파라미터에 해당 대화의 conversation ID를 제공하세요:
POST _plugins/_ml/agents/your_agent_id/_execute
{
"parameters": {
"question": "What's the population of Austin 2023, compared with Seattle",
"memory_id": "l7VUxI0B8vrNLhb9sRuQ",
"verbose": true
}
}
{% include copy-curl.html %}
응답을 보면 population_data_knowledge_base가 시애틀의 인구를 반환하지 않는다는 걸 알 수 있어요. 대신 에이전트가 과거 메시지를 참조해 시애틀 인구를 알아내죠:
{
"inference_results": [
{
"output": [
{
"name": "memory_id",
"result": "l7VUxI0B8vrNLhb9sRuQ"
},
{
"name": "parent_interaction_id",
"result": "B7VkxI0B8vrNLhb9mxy0"
},
{
"name": "response",
"result": """{
"thought": "Let me check the population data tool first",
"action": "population_data_knowledge_base",
"action_input": "{\"city\":\"Austin\",\"year\":2023}"
}"""
},
{
"name": "response",
"result": """{"_index":"test_population_data","_source":{"population_description":"Chart and table of population level and growth rate for the Austin metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Austin in 2023 is 2,228,000, a 2.39% increase from 2022.\\nThe metro area population of Austin in 2022 was 2,176,000, a 2.79% increase from 2021.\\nThe metro area population of Austin in 2021 was 2,117,000, a 3.12% increase from 2020.\\nThe metro area population of Austin in 2020 was 2,053,000, a 3.43% increase from 2019."},"_id":"BhF5vo0BubpYKX5ER0fT","_score":0.69129956}
{"_index":"test_population_data","_source":{"population_description":"Chart and table of population level and growth rate for the Austin metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of Austin in 2023 is 2,228,000, a 2.39% increase from 2022.\\nThe metro area population of Austin in 2022 was 2,176,000, a 2.79% increase from 2021.\\nThe metro area population of Austin in 2021 was 2,117,000, a 3.12% increase from 2020.\\nThe metro area population of Austin in 2020 was 2,053,000, a 3.43% increase from 2019."},"_id":"6zrZvo0BVR2NrurbRIAE","_score":0.69129956}
{"_index":"test_population_data","_source":{"population_description":"Chart and table of population level and growth rate for the New York City metro area from 1950 to 2023. United Nations population projections are also included through the year 2035.\\nThe current metro area population of New York City in 2023 is 18,937,000, a 0.37% increase from 2022.\\nThe metro area population of New York City in 2022 was 18,867,000, a 0.23% increase from 2021.\\nThe metro area population of New York City in 2021 was 18,823,000, a 0.1% increase from 2020.\\nThe metro area population of New York City in 2020 was 18,804,000, a 0.01% decline from 2019."},"_id":"AxF5vo0BubpYKX5ER0fT","_score":0.61015373}
"""
},
{
"name": "response",
"result": "According to the population data tool, the population of Austin in 2023 is approximately 2,228,000 people, a 2.39% increase from 2022. This is lower than the population of Seattle in 2023 which is approximately 3,519,000 people, a 0.86% increase from 2022."
}
]
}
]
}