Bedrock Converse
Bedrock Converse
출처: 문서
본문
기본 사용법 (Basic Usage)
프롬프트로 complete 호출
Colab에서 이 노트북을 여는 경우라면 LlamaIndex 🦙 설치가 필요할 거예요.
%pip install llama-index-llms-bedrock-converse
!pip install llama-index
from llama_index.llms.bedrock_converse import BedrockConverse
profile_name = "Your aws profile name"
resp = BedrockConverse(
model="anthropic.claude-3-haiku-20240307-v1:0",
profile_name=profile_name,
).complete("Paul Graham is ")
print(resp)
메시지 목록으로 chat 호출
from llama_index.core.llms import ChatMessage
from llama_index.llms.bedrock_converse import BedrockConverse
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality"
),
ChatMessage(role="user", content="Tell me a story"),
]
resp = BedrockConverse(
model="anthropic.claude-3-haiku-20240307-v1:0",
profile_name=profile_name,
).chat(messages)
print(resp)
스트리밍 (Streaming)
stream_complete 엔드포인트 사용
from llama_index.llms.bedrock_converse import BedrockConverse
llm = BedrockConverse(
model="anthropic.claude-3-haiku-20240307-v1:0",
profile_name=profile_name,
)
resp = llm.stream_complete("Paul Graham is ")
for r in resp:
print(r.delta, end="")
stream_chat 엔드포인트 사용
from llama_index.llms.bedrock_converse import BedrockConverse
llm = BedrockConverse(
model="anthropic.claude-3-haiku-20240307-v1:0",
profile_name=profile_name,
)
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality"
),
ChatMessage(role="user", content="Tell me a story"),
]
resp = llm.stream_chat(messages)
for r in resp:
print(r.delta, end="")
모델 구성 (Configure Model)
from llama_index.llms.bedrock_converse import BedrockConverse
llm = BedrockConverse(
model="anthropic.claude-3-haiku-20240307-v1:0",
profile_name=profile_name,
)
resp = llm.complete("Paul Graham is ")
print(resp)
Access Keys로 Bedrock에 연결하기
from llama_index.llms.bedrock_converse import BedrockConverse
llm = BedrockConverse(
model="us.amazon.nova-lite-v1:0",
aws_access_key_id="AWS Access Key ID to use",
aws_secret_access_key="AWS Secret Access Key to use",
aws_session_token="AWS Session Token to use",
region_name="AWS Region to use, eg. us-east-1",
)
resp = llm.complete("Paul Graham is ")
print(resp)
함수 호출 (Function Calling)
Claude, Command, Mistral Large 모델은 AWS Bedrock Converse를 통한 네이티브 함수 호출을 지원해요. llm의 predict_and_call 함수를 통해 LlamaIndex 도구와 매끄럽게 통합돼요.
이를 통해 사용자는 도구를 붙이고 LLM이 어떤 도구를 호출할지 (있으면) 스스로 결정하게 할 수 있어요.
에이전트 루프의 일부로 도구 호출을 수행하고 싶다면 agent 안내서를 대신 확인하세요.
참고: AWS Bedrock의 모든 모델이 함수 호출과 Converse API를 지원하는 건 아니에요. 각 LLM의 사용 가능한 기능을 여기서 확인하세요.
from llama_index.llms.bedrock_converse import BedrockConverse
from llama_index.core.tools import FunctionTool
def multiply(a: int, b: int) -> int:
"""Multiple two integers and returns the result integer"""
return a * b
def mystery(a: int, b: int) -> int:
"""Mystery function on two integers."""
return a * b + a + b
mystery_tool = FunctionTool.from_defaults(fn=mystery)
multiply_tool = FunctionTool.from_defaults(fn=multiply)
llm = BedrockConverse(
model="anthropic.claude-3-haiku-20240307-v1:0",
profile_name=profile_name,
)
response = llm.predict_and_call(
[mystery_tool, multiply_tool],
user_msg="What happens if I run the mystery function on 5 and 7",
)
print(str(response))
response = llm.predict_and_call(
[mystery_tool, multiply_tool],
user_msg=(
"""What happens if I run the mystery function on the following pairs of numbers? Generate a separate result for each row:
- 1 and 2
- 8 and 4
- 100 and 20
NOTE: you need to run the mystery function for all of the pairs above at the same time \
"""
),
allow_parallel_tool_calls=True,
)
print(str(response))
for s in response.sources:
print(f"Name: {s.tool_name}, Input: {s.raw_input}, Output: {str(s)}")
비동기 (Async)
from llama_index.llms.bedrock_converse import BedrockConverse
llm = BedrockConverse(
model="anthropic.claude-3-haiku-20240307-v1:0",
aws_access_key_id="AWS Access Key ID to use",
aws_secret_access_key="AWS Secret Access Key to use",
aws_session_token="AWS Session Token to use",
region_name="AWS Region to use, eg. us-east-1",
)
resp = await llm.acomplete("Paul Graham is ")
print(resp)