Bedrock (boto3) SDK

Bedrock 透传端点 - 以原生格式调用提供商特定端点（无翻译）。

功能	支持	注意事项
成本跟踪	❌	如果您需要此功能，请告知我们
日志记录	✅	适用于所有集成
终端用户跟踪	❌	如果您需要此功能，请告知我们
流式传输	✅

只需将 https://bedrock-runtime.{aws_region_name}.amazonaws.com 替换为 LITELLM_PROXY_BASE_URL/bedrock 🚀

示例用法

curl -X POST 'http://0.0.0.0:4000/bedrock/model/cohere.command-r-v1:0/converse' \
-H 'Authorization: Bearer anything' \
-H 'Content-Type: application/json' \
-d '{
    "messages": [
         {"role": "user",
        "content": [{"text": "Hello"}]
    }
    ]
}'

支持所有 Bedrock 端点（包括流式传输）。

查看所有 Bedrock 端点

快速开始

让我们调用 Bedrock /converse 端点

1. 将 AWS Keys 添加到您的环境中

export AWS_ACCESS_KEY_ID=""  # Access key
export AWS_SECRET_ACCESS_KEY="" # Secret access key
export AWS_REGION_NAME="" # us-east-1, us-east-2, us-west-1, us-west-2

2. 启动 LiteLLM 代理

litellm

# RUNNING on http://0.0.0.0:4000

3. 测试一下！

让我们调用 Bedrock converse 端点

curl -X POST 'http://0.0.0.0:4000/bedrock/model/cohere.command-r-v1:0/converse' \
-H 'Authorization: Bearer anything' \
-H 'Content-Type: application/json' \
-d '{
    "messages": [
         {"role": "user",
        "content": [{"text": "Hello"}]
    }
    ]
}'

示例

http://0.0.0.0:4000/bedrock 后面的任何内容都被视为提供商特定路由，并相应处理。

关键更改

原始端点	替换为
https://bedrock-runtime.{aws_region_name}.amazonaws.com	http://0.0.0.0:4000 (LITELLM_PROXY_BASE_URL="http://0.0.0.0:4000")
AWS4-HMAC-SHA256..	Bearer anything（如果在代理上设置了虚拟密钥，请使用 Bearer LITELLM_VIRTUAL_KEY）

示例 1：Converse API

LiteLLM 代理调用

curl -X POST 'http://0.0.0.0:4000/bedrock/model/cohere.command-r-v1:0/converse' \
-H 'Authorization: Bearer sk-anything' \
-H 'Content-Type: application/json' \
-d '{
    "messages": [
         {"role": "user",
        "content": [{"text": "Hello"}]
    }
    ]
}'

直接 Bedrock API 调用

curl -X POST 'https://bedrock-runtime.us-west-2.amazonaws.com/model/cohere.command-r-v1:0/converse' \
-H 'Authorization: AWS4-HMAC-SHA256..' \
-H 'Content-Type: application/json' \
-d '{
    "messages": [
         {"role": "user",
        "content": [{"text": "Hello"}]
    }
    ]
}'

示例 2：应用 Guardrail

LiteLLM 代理调用

curl "http://0.0.0.0:4000/bedrock/guardrail/guardrailIdentifier/version/guardrailVersion/apply" \
    -H 'Authorization: Bearer sk-anything' \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [{"text": {"text": "Hello world"}}],
      "source": "INPUT"
       }'

直接 Bedrock API 调用

curl "https://bedrock-runtime.us-west-2.amazonaws.com/guardrail/guardrailIdentifier/version/guardrailVersion/apply" \
    -H 'Authorization: AWS4-HMAC-SHA256..' \
    -H 'Content-Type: application/json' \
    -X POST \
    -d '{
      "contents": [{"text": {"text": "Hello world"}}],
      "source": "INPUT"
       }'

示例 3：查询知识库

curl -X POST "http://0.0.0.0:4000/bedrock/knowledgebases/{knowledgeBaseId}/retrieve" \
-H 'Authorization: Bearer sk-anything' \
-H 'Content-Type: application/json' \
-d '{
    "nextToken": "string",
    "retrievalConfiguration": { 
        "vectorSearchConfiguration": { 
          "filter": { ... },
          "numberOfResults": number,
          "overrideSearchType": "string"
        }
    },
    "retrievalQuery": { 
        "text": "string"
    }
}'

直接 Bedrock API 调用

curl -X POST "https://bedrock-agent-runtime.us-west-2.amazonaws.com/knowledgebases/{knowledgeBaseId}/retrieve" \
-H 'Authorization: AWS4-HMAC-SHA256..' \
-H 'Content-Type: application/json' \
-d '{
    "nextToken": "string",
    "retrievalConfiguration": { 
        "vectorSearchConfiguration": { 
          "filter": { ... },
          "numberOfResults": number,
          "overrideSearchType": "string"
        }
    },
    "retrievalQuery": { 
        "text": "string"
    }
}'

高级 - 与虚拟密钥一起使用

前提条件

• 设置带数据库的代理

使用此功能可以避免向开发人员提供原始 AWS Keys，同时仍允许他们使用 AWS Bedrock 端点。

用法

1. 设置环境

export DATABASE_URL=""
export LITELLM_MASTER_KEY=""
export AWS_ACCESS_KEY_ID=""  # Access key
export AWS_SECRET_ACCESS_KEY="" # Secret access key
export AWS_REGION_NAME="" # us-east-1, us-east-2, us-west-1, us-west-2

litellm

# RUNNING on http://0.0.0.0:4000

2. 生成虚拟密钥

curl -X POST 'http://0.0.0.0:4000/key/generate' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-d '{}'

预期响应

{
    ...
    "key": "sk-1234ewknldferwedojwojw"
}

3. 测试一下！

curl -X POST 'http://0.0.0.0:4000/bedrock/model/cohere.command-r-v1:0/converse' \
-H 'Authorization: Bearer sk-1234ewknldferwedojwojw' \
-H 'Content-Type: application/json' \
-d '{
    "messages": [
         {"role": "user",
        "content": [{"text": "Hello"}]
    }
    ]
}'

高级 - Bedrock Agents

通过 LiteLLM 代理调用 Bedrock Agents

import os 
import boto3 
from botocore.config import Config

# # Define your proxy endpoint
proxy_endpoint = "http://0.0.0.0:4000/bedrock" # 👈 your proxy base url

# # Create a Config object with the proxy
# Custom headers
custom_headers = {
    'litellm_user_api_key': 'Bearer sk-1234', # 👈 your proxy api key
}


os.environ["AWS_ACCESS_KEY_ID"] = "my-fake-key-id"
os.environ["AWS_SECRET_ACCESS_KEY"] = "my-fake-access-key"


# Create the client
runtime_client = boto3.client(
    service_name="bedrock-agent-runtime", 
    region_name="us-west-2", 
    endpoint_url=proxy_endpoint
)

# Custom header injection
def inject_custom_headers(request, **kwargs):
    request.headers.update(custom_headers)

# Attach the event to inject custom headers before the request is sent
runtime_client.meta.events.register('before-send.*.*', inject_custom_headers)


response = runtime_client.invoke_agent(
            agentId="L1RT58GYRW",
            agentAliasId="MFPSBCXYTW",
            sessionId="12345",
            inputText="Who do you know?"
        )

completion = ""

for event in response.get("completion"):
    chunk = event["chunk"]
    completion += chunk["bytes"].decode()

print(completion)