Reranking search results using Amazon Bedrock models
출처: 문서
본문
Amazon Bedrock 모델을 사용한 검색 결과 리랭킹
리랭킹 파이프라인을 사용하면 검색 결과를 리랭킹하면서, 검색 쿼리를 기준으로 검색 결과의 각 문서에 관련성 점수를 매길 수 있어요. 관련성 점수는 cross-encoder 모델이 계산해요.
이 튜토리얼에서는 Amazon Bedrock Rerank API를 사용해서 Amazon Bedrock에서 호스팅되는 모델로 검색 결과를 리랭킹하는 방법을 알아봐요.
your_ 접두사로 시작하는 플레이스홀더는 여러분의 값으로 바꾸세요.
{: .note}
사전 준비: Amazon Bedrock에서 모델 테스트하기
모델을 사용하기 전에 Amazon Bedrock에서 먼저 테스트해 보세요. 리랭킹에 지원되는 모델은 Amazon Bedrock에서 리랭킹을 지원하는 리전 및 모델을 참고하고, 모델 ID는 Amazon Bedrock에서 지원되는 파운데이션 모델을 참고하세요. 리랭킹 테스트를 수행하려면 다음 코드를 사용해요:
import json
import boto3
bedrock_region = "your_bedrock_model_region_like_us-west-2"
bedrock_agent_runtime_client = boto3.client("bedrock-agent-runtime", region_name=bedrock_region)
model_id = "amazon.rerank-v1:0"
response = bedrock_agent_runtime_client.rerank(
queries=[
{
"textQuery": {
"text": "What is the capital city of America?",
},
"type": "TEXT"
}
],
rerankingConfiguration={
"bedrockRerankingConfiguration": {
"modelConfiguration": {
"modelArn": f"arn:aws:bedrock:{bedrock_region}::foundation-model/{model_id}"
},
},
"type": "BEDROCK_RERANKING_MODEL"
},
sources=[
{
"inlineDocumentSource": {
"textDocument": {
"text": "Carson City is the capital city of the American state of Nevada.",
},
"type": "TEXT"
},
"type": "INLINE"
},
{
"inlineDocumentSource": {
"textDocument": {
"text": "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
},
"type": "TEXT"
},
"type": "INLINE"
},
{
"inlineDocumentSource": {
"textDocument": {
"text": "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.",
},
"type": "TEXT"
},
"type": "INLINE"
},
{
"inlineDocumentSource": {
"textDocument": {
"text": "Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."
},
"type": "TEXT"
},
"type": "INLINE"
},
]
)
results = response["results"]
print(json.dumps(results, indent=2))
{% include copy.html %}
리랭킹된 결과는 가장 높은 점수 순으로 정렬돼요:
[
{
"index": 2,
"relevanceScore": 0.7711548805236816
},
{
"index": 0,
"relevanceScore": 0.0025114635936915874
},
{
"index": 1,
"relevanceScore": 2.4876489987946115e-05
},
{
"index": 3,
"relevanceScore": 6.339210358419223e-06
}
]
결과를 인덱스 순으로 정렬하려면 다음 코드를 사용하세요:
print(json.dumps(sorted(results, key=lambda x: x['index']),indent=2))
다음은 인덱스 순으로 정렬한 결과예요:
[
{
"index": 0,
"relevanceScore": 0.0025114635936915874
},
{
"index": 1,
"relevanceScore": 2.4876489987946115e-05
},
{
"index": 2,
"relevanceScore": 0.7711548805236816
},
{
"index": 3,
"relevanceScore": 6.339210358419223e-06
}
]
1단계: 커넥터 만들기 및 모델 등록
커넥터를 만들고 모델을 등록하려면 다음 단계를 따르세요.
1.1단계: 모델용 커넥터 만들기
먼저 모델용 커넥터를 만들어요.
셀프 관리형 OpenSearch를 사용하는 경우에는 AWS 자격 증명을 제공하세요:
POST /_plugins/_ml/connectors/_create
{
"name": "Amazon Bedrock Rerank API",
"description": "Test connector for Amazon Bedrock Rerank API",
"version": 1,
"protocol": "aws_sigv4",
"credential": {
"access_key": "your_access_key",
"secret_key": "your_secret_key",
"session_token": "your_session_token"
},
"parameters": {
"service_name": "bedrock",
"endpoint": "bedrock-agent-runtime",
"region": "your_bedrock_model_region_like_us-west-2",
"api_name": "rerank",
"model_id": "amazon.rerank-v1:0"
},
"actions": [
{
"action_type": "PREDICT",
"method": "POST",
"url": "https://${parameters.endpoint}.${parameters.region}.amazonaws.com/${parameters.api_name}",
"headers": {
"x-amz-content-sha256": "required",
"content-type": "application/json"
},
"pre_process_function": "connector.pre_process.bedrock.rerank",
"request_body": """
{
"queries": ${parameters.queries},
"rerankingConfiguration": {
"bedrockRerankingConfiguration": {
"modelConfiguration": {
"modelArn": "arn:aws:bedrock:${parameters.region}::foundation-model/${parameters.model_id}"
}
},
"type": "BEDROCK_RERANKING_MODEL"
},
"sources": ${parameters.sources}
}
""",
"post_process_function": "connector.post_process.bedrock.rerank"
}
]
}
{% include copy-curl.html %}
Amazon OpenSearch Service를 사용하는 경우에는 Amazon Bedrock에 대한 액세스를 허용하는 AWS Identity and Access Management(IAM) 역할의 Amazon Resource Name(ARN)을 제공할 수 있어요. 자세한 내용은 AWS 문서를 참고하세요. 다음 요청으로 커넥터를 만들어요:
POST /_plugins/_ml/connectors/_create
{
"name": "Amazon Bedrock Rerank API",
"description": "Test connector for Amazon Bedrock Rerank API",
"version": 1,
"protocol": "aws_sigv4",
"credential": {
"roleArn": "your_role_arn_which_allows_access_to_bedrock_agent_runtime_rerank_api"
},
"parameters": {
"service_name": "bedrock",
"endpoint": "bedrock-agent-runtime",
"region": "your_bedrock_model_region_like_us-west-2",
"api_name": "rerank",
"model_id": "amazon.rerank-v1:0"
},
"actions": [
{
"action_type": "PREDICT",
"method": "POST",
"url": "https://${parameters.endpoint}.${parameters.region}.amazonaws.com/${parameters.api_name}",
"headers": {
"x-amz-content-sha256": "required",
"content-type": "application/json"
},
"pre_process_function": "connector.pre_process.bedrock.rerank",
"request_body": """
{
"queries": ${parameters.queries},
"rerankingConfiguration": {
"bedrockRerankingConfiguration": {
"modelConfiguration": {
"modelArn": "arn:aws:bedrock:${parameters.region}::foundation-model/${parameters.model_id}"
}
},
"type": "BEDROCK_RERANKING_MODEL"
},
"sources": ${parameters.sources}
}
""",
"post_process_function": "connector.post_process.bedrock.rerank"
}
]
}
{% include copy-curl.html %}
1.2단계: 모델 등록 및 배포
응답에서 받은 커넥터 ID를 사용해서 모델을 등록하고 배포하세요:
POST /_plugins/_ml/models/_register?deploy=true
{
"name": "Amazon Bedrock Rerank API",
"function_name": "remote",
"description": "test Amazon Bedrock Rerank API",
"connector_id": "your_connector_id"
}
{% include copy-curl.html %}
응답에 포함된 모델 ID를 잘 적어 두세요. 이후 단계에서 사용하게 돼요.
1.3단계: 모델 테스트
Predict API를 사용해서 모델을 테스트해 보세요:
POST _plugins/_ml/_predict/text_similarity/your_model_id
{
"query_text": "What is the capital city of America?",
"text_docs": [
"Carson City is the capital city of the American state of Nevada.",
"The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
"Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.",
"Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."
]
}
{% include copy-curl.html %}
또는 다음 쿼리로 모델을 테스트할 수도 있어요. 이 쿼리는 pre_process_function을 거치지 않고 Rerank API를 직접 호출해요:
POST _plugins/_ml/models/your_model_id/_predict
{
"parameters": {
"queries": [
{
"textQuery": {
"text": "What is the capital city of America?"
},
"type": "TEXT"
}
],
"sources": [
{
"inlineDocumentSource": {
"textDocument": {
"text": "Carson City is the capital city of the American state of Nevada."
},
"type": "TEXT"
},
"type": "INLINE"
},
{
"inlineDocumentSource": {
"textDocument": {
"text": "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan."
},
"type": "TEXT"
},
"type": "INLINE"
},
{
"inlineDocumentSource": {
"textDocument": {
"text": "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district."
},
"type": "TEXT"
},
"type": "INLINE"
},
{
"inlineDocumentSource": {
"textDocument": {
"text": "Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."
},
"type": "TEXT"
},
"type": "INLINE"
}
]
}
}
{% include copy-curl.html %}
커넥터의 pre_process_function은 입력을 Predict API parameters에 필요한 형식으로 변환해 줘요.
기본적으로 Amazon Bedrock Rerank API의 출력은 다음과 같은 형식이에요:
[
{
"index": 2,
"relevanceScore": 0.7711548724998493
},
{
"index": 0,
"relevanceScore": 0.0025114635138098534
},
{
"index": 1,
"relevanceScore": 2.4876490010363496e-05
},
{
"index": 3,
"relevanceScore": 6.339210403977635e-06
}
]
커넥터의 post_process_function은 모델 출력을 Reranker processor가 해석할 수 있는 형식으로 변환하고, 결과를 인덱스 순으로 정렬해 줘요.
응답에는 similarity 출력이 네 개 포함되어 있어요. 각 similarity 출력의 data 배열에는 쿼리에 대한 각 문서의 관련성 점수가 담겨 있어요. similarity 출력은 입력 문서 순서대로 제공되며, 첫 번째 similarity 결과가 첫 번째 문서에 해당해요:
{
"inference_results": [
{
"output": [
{
"name": "similarity",
"data_type": "FLOAT32",
"shape": [
1
],
"data": [
0.0025114636
]
},
{
"name": "similarity",
"data_type": "FLOAT32",
"shape": [
1
],
"data": [
2.487649e-05
]
},
{
"name": "similarity",
"data_type": "FLOAT32",
"shape": [
1
],
"data": [
0.7711549
]
},
{
"name": "similarity",
"data_type": "FLOAT32",
"shape": [
1
],
"data": [
6.3392104e-06
]
}
],
"status_code": 200
}
]
}
2단계: 리랭킹 파이프라인 만들기
리랭킹 파이프라인을 만들려면 다음 단계를 따르세요.
2.1단계: 테스트 데이터 색인하기
다음 요청으로 인덱스에 데이터를 색인하세요:
POST _bulk
{ "index": { "_index": "my-test-data" } }
{ "passage_text" : "Carson City is the capital city of the American state of Nevada." }
{ "index": { "_index": "my-test-data" } }
{ "passage_text" : "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan." }
{ "index": { "_index": "my-test-data" } }
{ "passage_text" : "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district." }
{ "index": { "_index": "my-test-data" } }
{ "passage_text" : "Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states." }
{% include copy-curl.html %}
2.2단계: 리랭킹 파이프라인 만들기
Amazon Bedrock 리랭킹 모델을 사용해서 리랭킹 파이프라인을 만들어요:
PUT /_search/pipeline/rerank_pipeline_bedrock
{
"description": "Pipeline for reranking with Bedrock rerank model",
"response_processors": [
{
"rerank": {
"ml_opensearch": {
"model_id": "your_model_id_created_in_step1"
},
"context": {
"document_fields": ["passage_text"]
}
}
}
]
}
{% include copy-curl.html %}
document_fields에 여러 개의 필드 이름을 지정하면 모든 필드의 값을 먼저 이어 붙인 다음에 리랭킹을 수행해요.
{: .note}
2.3단계: 리랭킹 테스트
먼저 리랭킹 파이프라인을 사용하지 않고 쿼리를 테스트해 보세요:
POST my-test-data/_search
{
"query": {
"match": {
"passage_text": "What is the capital city of America?"
}
},
"highlight": {
"pre_tags": ["<strong>"],
"post_tags": ["</strong>"],
"fields": {"passage_text": {}}
},
"_source": false,
"fields": ["passage_text"]
}
{% include copy-curl.html %}
응답의 첫 번째 문서가 Carson City is the capital city of the American state of Nevada로 나오는데, 잘못된 결과예요:
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 4,
"relation": "eq"
},
"max_score": 2.5045562,
"hits": [
{
"_index": "my-test-data",
"_id": "1",
"_score": 2.5045562,
"fields": {
"passage_text": [
"Carson City is the capital city of the American state of Nevada."
]
},
"highlight": {
"passage_text": [
"Carson <strong>City</strong> <strong>is</strong> <strong>the</strong> <strong>capital</strong> <strong>city</strong> <strong>of</strong> <strong>the</strong> American state <strong>of</strong> Nevada."
]
}
},
{
"_index": "my-test-data",
"_id": "2",
"_score": 0.5807494,
"fields": {
"passage_text": [
"The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan."
]
},
"highlight": {
"passage_text": [
"<strong>The</strong> Commonwealth <strong>of</strong> <strong>the</strong> Northern Mariana Islands <strong>is</strong> a group <strong>of</strong> islands in <strong>the</strong> Pacific Ocean.",
"Its <strong>capital</strong> <strong>is</strong> Saipan."
]
}
},
{
"_index": "my-test-data",
"_id": "3",
"_score": 0.5261191,
"fields": {
"passage_text": [
"Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district."
]
},
"highlight": {
"passage_text": [
"(also known as simply Washington or D.C., and officially as <strong>the</strong> District <strong>of</strong> Columbia) <strong>is</strong> <strong>the</strong> <strong>capital</strong>",
"<strong>of</strong> <strong>the</strong> United States.",
"It <strong>is</strong> a federal district."
]
}
},
{
"_index": "my-test-data",
"_id": "4",
"_score": 0.5083029,
"fields": {
"passage_text": [
"Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."
]
},
"highlight": {
"passage_text": [
"<strong>Capital</strong> punishment (<strong>the</strong> death penalty) has existed in <strong>the</strong> United States since beforethe United States",
"As <strong>of</strong> 2017, <strong>capital</strong> punishment <strong>is</strong> legal in 30 <strong>of</strong> <strong>the</strong> 50 states."
]
}
}
]
}
}
이번에는 리랭킹 파이프라인을 사용해서 쿼리를 테스트해 보세요:
POST my-test-data/_search?search_pipeline=rerank_pipeline_bedrock
{
"query": {
"match": {
"passage_text": "What is the capital city of America?"
}
},
"ext": {
"rerank": {
"query_context": {
"query_text": "What is the capital city of America?"
}
}
},
"highlight": {
"pre_tags": ["<strong>"],
"post_tags": ["</strong>"],
"fields": {"passage_text": {}}
},
"_source": false,
"fields": ["passage_text"]
}
{% include copy-curl.html %}
응답의 첫 번째 문서가 "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district."로 나오는데, 올바른 결과예요:
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 4,
"relation": "eq"
},
"max_score": 0.7711549,
"hits": [
{
"_index": "my-test-data",
"_id": "3",
"_score": 0.7711549,
"fields": {
"passage_text": [
"Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district."
]
},
"highlight": {
"passage_text": [
"(also known as simply Washington or D.C., and officially as <strong>the</strong> District <strong>of</strong> Columbia) <strong>is</strong> <strong>the</strong> <strong>capital</strong>",
"<strong>of</strong> <strong>the</strong> United States.",
"It <strong>is</strong> a federal district."
]
}
},
{
"_index": "my-test-data",
"_id": "1",
"_score": 0.0025114636,
"fields": {
"passage_text": [
"Carson City is the capital city of the American state of Nevada."
]
},
"highlight": {
"passage_text": [
"Carson <strong>City</strong> <strong>is</strong> <strong>the</strong> <strong>capital</strong> <strong>city</strong> <strong>of</strong> <strong>the</strong> American state <strong>of</strong> Nevada."
]
}
},
{
"_index": "my-test-data",
"_id": "2",
"_score": 02.487649e-05,
"fields": {
"passage_text": [
"The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan."
]
},
"highlight": {
"passage_text": [
"<strong>The</strong> Commonwealth <strong>of</strong> <strong>the</strong> Northern Mariana Islands <strong>is</strong> a group <strong>of</strong> islands in <strong>the</strong> Pacific Ocean.",
"Its <strong>capital</strong> <strong>is</strong> Saipan."
]
}
},
{
"_index": "my-test-data",
"_id": "4",
"_score": 6.3392104e-06,
"fields": {
"passage_text": [
"Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."
]
},
"highlight": {
"passage_text": [
"<strong>Capital</strong> punishment (<strong>the</strong> death penalty) has existed in <strong>the</strong> United States since beforethe United States",
"As <strong>of</strong> 2017, <strong>capital</strong> punishment <strong>is</strong> legal in 30 <strong>of</strong> <strong>the</strong> 50 states."
]
}
}
]
},
"profile": {
"shards": []
}
}
query_text 대신 query_text_path를 지정하면 같은 쿼리를 재사용할 수 있어요:
POST my-test-data/_search?search_pipeline=rerank_pipeline_bedrock
{
"query": {
"match": {
"passage_text": "What is the capital city of America?"
}
},
"ext": {
"rerank": {
"query_context": {
"query_text_path": "query.match.passage_text.query"
}
}
},
"highlight": {
"pre_tags": ["<strong>"],
"post_tags": ["</strong>"],
"fields": {"passage_text": {}}
},
"_source": false,
"fields": ["passage_text"]
}
{% include copy-curl.html %}