Bedrock Agents

Bedrock Agents

OpenAI 요청/응답 형식으로 Bedrock Agents를 호출해요.

출처: 문서

본문

개요 (Overview)

속성 설명
설명 Amazon Bedrock Agents는 기반 모델(FM), API, 데이터의 추론을 사용해 사용자 요청을 분해하고 관련 정보를 수집하며 작업을 효율적으로 완료해요
LiteLLM 라우트 bedrock/agent/{AGENT_ID}/{ALIAS_ID}
공급자 문서 AWS Bedrock Agents

빠른 시작 (Quick Start)

LiteLLM용 모델 형식

LiteLLM으로 bedrock agent를 호출하려면 다음 모델 형식을 사용해야 해요. model=bedrock/agent/는 LiteLLM이 bedrock InvokeAgent API를 호출하도록 지시해요.

bedrock/agent/{AGENT_ID}/{ALIAS_ID}

예시:

  • bedrock/agent/L1RT58GYRW/MFPSBCXYTW
  • bedrock/agent/ABCD1234/LIVE

이 ID들은 AWS Bedrock 콘솔의 Agents 아래에서 찾을 수 있어요.

LiteLLM Python SDK

기본 Agent Completion:

import litellm

# Make a completion request to your Bedrock Agent
response = litellm.completion(
    model="bedrock/agent/L1RT58GYRW/MFPSBCXYTW",  # agent/{AGENT_ID}/{ALIAS_ID}
    messages=[
        {
            "role": "user",
            "content": "Hi, I need help with analyzing our Q3 sales data and generating a summary report"
        }
    ],
)

print(response.choices[0].message.content)
print(f"Response cost: ${response._hidden_params['response_cost']}")

Agent 응답 스트리밍:

import litellm

# Stream responses from your Bedrock Agent
response = litellm.completion(
    model="bedrock/agent/L1RT58GYRW/MFPSBCXYTW",
    messages=[
        {
            "role": "user",
            "content": "Can you help me plan a marketing campaign and provide step-by-step execution details?"
        }
    ],
    stream=True,
)

for chunk in response:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

LiteLLM Proxy

1. config.yaml에서 모델 설정:

model_list:
  - model_name: bedrock-agent-1
    litellm_params:
      model: bedrock/agent/L1RT58GYRW/MFPSBCXYTW
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: us-west-2
  - model_name: bedrock-agent-2
    litellm_params:
      model: bedrock/agent/AGENT456/ALIAS789
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: us-east-1

2. LiteLLM Proxy 시작:

litellm --config config.yaml

3. Bedrock Agents에 요청:

curl (기본 요청):

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "bedrock-agent-1",
    "messages": [
      {
        "role": "user",
        "content": "Analyze our customer data and suggest retention strategies"
      }
    ]
  }'

curl (스트리밍 요청):

curl http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ***" \
  -d '{
    "model": "bedrock-agent-2",
    "messages": [
      {
        "role": "user",
        "content": "Create a comprehensive social media strategy for our new product"
      }
    ],
    "stream": true
  }'

OpenAI Python SDK:

from openai import OpenAI

# Initialize client with your LiteLLM proxy URL
client = OpenAI(
    base_url="http://localhost:4000",
    api_key="your-litellm-api-key",
)

# Make a completion request to your agent
response = client.chat.completions.create(
    model="bedrock-agent-1",
    messages=[
        {
            "role": "user",
            "content": "Help me prepare for the quarterly business review meeting"
        }
    ],
)
print(response.choices[0].message.content)

OpenAI SDK로 스트리밍:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:4000",
    api_key="your-litellm-api-key",
)

# Stream agent responses
stream = client.chat.completions.create(
    model="bedrock-agent-2",
    messages=[
        {
            "role": "user",
            "content": "Walk me through launching a new feature beta program"
        }
    ],
    stream=True,
)

for chunk in stream:
    if chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="")

공급자별 파라미터 (Provider-specific Parameters)

OpenAI가 아닌 파라미터는 agent에 custom 파라미터로 전달돼요.

SDK:

from litellm import completion

response = litellm.completion(
    model="bedrock/agent/L1RT58GYRW/MFPSBCXYTW",
    messages=[
        {
            "role": "user",
            "content": "Hi who is ishaan cto of litellm, tell me 10 things about him",
        }
    ],
    invocationId="my-test-invocation-id",  # PROVIDER-SPECIFIC VALUE
)

Proxy 설정:

model_list:
  - model_name: bedrock-agent-1
    litellm_params:
      model: bedrock/agent/L1RT58GYRW/MFPSBCXYTW
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY
      aws_region_name: us-west-2
      invocationId: my-test-invocation-id

더 알아보기 (Learn more)