护栏 - 快速入门

在 LiteLLM Proxy (AI Gateway) 上设置 Prompt Injection 检测、PII 屏蔽

1. 在 LiteLLM config.yaml 中定义 guardrails

在 guardrails 部分设置您的 guardrails

model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: general-guard
    litellm_params:
      guardrail: aim
      mode: [pre_call, post_call]
      api_key: os.environ/AIM_API_KEY
      api_base: os.environ/AIM_API_BASE
      default_on: true # Optional
  
  - guardrail_name: "aporia-pre-guard"
    litellm_params:
      guardrail: aporia  # supported values: "aporia", "lakera"
      mode: "during_call"
      api_key: os.environ/APORIA_API_KEY_1
      api_base: os.environ/APORIA_API_BASE_1
  - guardrail_name: "aporia-post-guard"
    litellm_params:
      guardrail: aporia  # supported values: "aporia", "lakera"
      mode: "post_call"
      api_key: os.environ/APORIA_API_KEY_2
      api_base: os.environ/APORIA_API_BASE_2
    guardrail_info: # Optional field, info is returned on GET /guardrails/list
      # you can enter any fields under info for consumers of your guardrail
      params:
        - name: "toxicity_score"
          type: "float"
          description: "Score between 0-1 indicating content toxicity level"
        - name: "pii_detection"
          type: "boolean"

# Example Presidio guardrail config with entity actions + confidence score thresholds
  - guardrail_name: "presidio-pii"
    litellm_params:
      guardrail: presidio
      mode: "pre_call"
      presidio_language: "en"
      pii_entities_config:
        CREDIT_CARD: "MASK"
        EMAIL_ADDRESS: "MASK"
        US_SSN: "MASK"
      presidio_score_thresholds:  # minimum confidence scores for keeping detections
        CREDIT_CARD: 0.8
        EMAIL_ADDRESS: 0.6

# Example Pillar Security config via Generic Guardrail API
  - guardrail_name: "pillar-security"
    litellm_params:
      guardrail: generic_guardrail_api
      mode: [pre_call, post_call]
      api_base: https://api.pillar.security/api/v1/integrations/litellm
      api_key: os.environ/PILLAR_API_KEY
      additional_provider_specific_params:
        plr_mask: true
        plr_evidence: true
        plr_scanners: true

`mode` (事件钩子) 的支持值

pre_call 在 LLM 调用之前运行，针对输入
post_call 在 LLM 调用之后运行，针对输入和输出
during_call 在 LLM 调用期间运行，针对输入。与 pre_call 相同，但与 LLM 调用并行运行。在 guardrail 检查完成之前不会返回响应
运行多种模式的上述值列表，例如 mode: [pre_call, post_call]

负载均衡 Guardrails

需要跨多个帐户或区域分配 guardrail 请求吗？请参阅 Guardrail 负载均衡以了解详情

跨多个 AWS Bedrock 帐户进行负载均衡（适用于速率限制管理）
guardrail 实例之间的加权分配
多区域 guardrail 部署

2. 启动 LiteLLM Gateway

litellm --config config.yaml --detailed_debug

3. 测试请求

Langchain、OpenAI SDK 用法示例

失败调用
成功调用

由于请求中的 ishaan@berri.ai 是 PII，预计此测试会失败

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi my email is ishaan@berri.ai"}
    ],
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

失败时的预期响应

{
  "error": {
    "message": {
      "error": "Violated guardrail policy",
      "aporia_ai_response": {
        "action": "block",
        "revised_prompt": null,
        "revised_response": "Aporia detected and blocked PII",
        "explain_log": null
      }
    },
    "type": "None",
    "param": "None",
    "code": "400"
  }
}

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi what is the weather"}
    ],
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

默认开启 Guardrails

在 guardrail 配置中设置 default_on: true，以在每个请求上运行 guardrail。如果您希望在每个请求上运行 guardrail，而无需用户指定它，这将很有用。

注意：即使用户指定了不同的 guardrail 或空 guardrails 数组，这些也会运行。

guardrails:
  - guardrail_name: "aporia-pre-guard"
    litellm_params:
      guardrail: aporia
      mode: "pre_call"
      default_on: true

测试请求

在此请求中，guardrail aporia-pre-guard 将在每个请求上运行，因为设置了 default_on: true。

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi my email is ishaan@berri.ai"}
    ]
  }'

预期响应

您的响应头将包含 x-litellm-applied-guardrails 以及应用的 guardrail

x-litellm-applied-guardrails: aporia-pre-guard

Guardrail 策略

需要更多控制权？使用 Guardrail 策略来

将 guardrails 分组为可重用的策略
为特定团队、密钥或模型启用/禁用 guardrails
从现有策略继承并覆盖特定的 guardrails

客户端使用 Guardrails

自行测试 (OSS)

将 guardrails 传递到您的请求体中进行测试

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi my email is ishaan@berri.ai"}
    ],
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

暴露给您的用户 (Enterprise)

遵循此简单的流程来实现和调整 guardrails

1. 查看可用 Guardrails

首先，检查可用的 guardrails 及其参数

调用 /guardrails/list 以查看可用的 guardrails 和 guardrail 信息（支持的参数、描述等）

curl -X GET 'http://0.0.0.0:4000/guardrails/list'

预期响应

{
    "guardrails": [
        {
        "guardrail_name": "aporia-post-guard",
        "guardrail_info": {
            "params": [
            {
                "name": "toxicity_score",
                "type": "float",
                "description": "Score between 0-1 indicating content toxicity level"
            },
            {
                "name": "pii_detection",
                "type": "boolean"
            }
            ]
        }
        }
    ]
}

此配置将返回上述 /guardrails/list 响应。guardrail_info 字段是可选的，您可以将任何字段添加到 info 下方供 guardrail 的使用者使用

- guardrail_name: "aporia-post-guard"
    litellm_params:
      guardrail: aporia  # supported values: "aporia", "lakera"
      mode: "post_call"
      api_key: os.environ/APORIA_API_KEY_2
      api_base: os.environ/APORIA_API_BASE_2
    guardrail_info: # Optional field, info is returned on GET /guardrails/list
      # you can enter any fields under info for consumers of your guardrail
      params:
        - name: "toxicity_score"
          type: "float"
          description: "Score between 0-1 indicating content toxicity level"
        - name: "pii_detection"
          type: "boolean"

2. 应用 Guardrails

将选定的 guardrails 添加到您的聊天完成请求中

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "your message"}],
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

3. 使用 Mock LLM 完成项进行测试

发送 mock_response 以在不进行 LLM 调用时测试 guardrails。有关 mock_response 的更多信息，请参阅此处

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi my email is ishaan@berri.ai"}
    ],
    "mock_response": "This is a mock response",
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

4. ✨ 将动态参数传递给 Guardrail

信息

✨ 此功能仅适用于 Enterprise 获取免费试用

使用此功能将其他参数传递给 guardrail API 调用。例如，成功阈值等。有关更多详细信息，请参阅 guardrails 规范

OpenAI Python v1.0.0+
Curl 请求

设置 guardrails={"aporia-pre-guard": {"extra_body": {"success_threshold": 0.9}}} 以将其他参数传递给 guardrail

在此示例中，success_threshold=0.9 传递给 aporia-pre-guard guardrail 请求体

import openai
client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ],
    extra_body={
      "guardrails": [
        "aporia-pre-guard": {
          "extra_body": {
            "success_threshold": 0.9
          }
        }
      ]
    }

)

print(response)

curl --location 'http://0.0.0.0:4000/chat/completions' \
    --header 'Content-Type: application/json' \
    --data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
        "role": "user",
        "content": "what llm are you"
        }
    ],
    "guardrails": [
      "aporia-pre-guard": {
        "extra_body": {
          "success_threshold": 0.9
        }
      }
    ]
}'

Proxy 管理员控制

监控 Guardrails

监控哪些 guardrails 已执行以及它们是成功还是失败。例如，guardrail 出现异常并导致我们不希望失败的请求失败

:::

设置

将 LiteLLM 连接到支持的日志记录提供程序
发出带有 guardrails 参数的请求
检查您的日志记录提供程序以获取 guardrail 跟踪

追踪的 Guardrail 成功案例

追踪的 Guardrail 失败案例

✨ 每个 API 密钥控制 Guardrails

信息

✨ 此功能仅适用于 Enterprise 获取免费试用

使用此功能控制每个 API 密钥运行哪些 guardrails。在此教程中，我们只想让以下 guardrails 为 1 个 API 密钥运行

guardrails: ["aporia-pre-guard", "aporia-post-guard"]

步骤 1 创建带有 guardrail 设置的 Key

/key/generate
/key/update

curl -X POST 'http://0.0.0.0:4000/key/generate' \
    -H 'Authorization: Bearer sk-1234' \
    -H 'Content-Type: application/json' \
    -d '{
            "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
        }
    }'

curl --location 'http://0.0.0.0:4000/key/update' \
    --header 'Authorization: Bearer sk-1234' \
    --header 'Content-Type: application/json' \
    --data '{
        "key": "sk-jNm1Zar7XfNdZXp49Z1kSQ",
        "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
        }
}'

步骤 2 使用新的 Key 进行测试

curl --location 'http://0.0.0.0:4000/chat/completions' \
    --header 'Authorization: Bearer sk-jNm1Zar7XfNdZXp49Z1kSQ' \
    --header 'Content-Type: application/json' \
    --data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
        "role": "user",
        "content": "my email is ishaan@berri.ai"
        }
    ]
}'

✨ 基于标签的 Guardrail 模式

信息

✨ 此功能仅适用于 Enterprise 获取免费试用

根据 user-agent 标头运行 guardrails。这对于在 OpenWebUI 上运行预调用检查但在 Claude CLI 中仅屏蔽日志很有用。

model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: "guardrails_ai-guard"
    litellm_params:
      guardrail: guardrails_ai
      guard_name: "pii_detect" # 👈 Guardrail AI guard name
      mode:
        tags:
            "User-Agent: claude-cli": "logging_only"                 # Claude CLI - only mask in logs
        default: "pre_call"               # Default mode when no tags match
      api_base: os.environ/GUARDRAILS_AI_API_BASE # 👈 Guardrails AI API Base. Defaults to "http://0.0.0.0:8000"
      default_on: true # run on every request

✨ 模型级别 Guardrails

信息

✨ 此功能仅适用于 Enterprise 获取免费试用

这非常适合您拥有本地和托管模型的情况，并且只想运行防止将 PII 发送到托管模型的情况。

model_list:
  - model_name: claude-sonnet-4
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY
      api_base: https://api.anthropic.com/v1
      guardrails: ["azure-text-moderation"]
  - model_name: openai-gpt-4o
    litellm_params:
      model: openai/gpt-4o

guardrails:
  - guardrail_name: "presidio-pii"
    litellm_params:
      guardrail: presidio  # supported values: "aporia", "bedrock", "lakera", "presidio"
      mode: "pre_call"
      presidio_language: "en"  # optional: set default language for PII analysis
      pii_entities_config:
        PERSON: "BLOCK"  # Will mask credit card numbers
  - guardrail_name: azure-text-moderation
    litellm_params:
      guardrail: azure/text_moderations
      mode: "post_call" 
      api_key: os.environ/AZURE_GUARDRAIL_API_KEY
      api_base: os.environ/AZURE_GUARDRAIL_API_BASE 

✨ 禁止团队开启/关闭 guardrails

信息

✨ 此功能仅适用于 Enterprise 获取免费试用

1. 禁止团队修改 guardrails

curl -X POST 'http://0.0.0.0:4000/team/update' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-d '{
    "team_id": "4198d93c-d375-4c83-8d5a-71e7c5473e50",
    "metadata": {"guardrails": {"modify_guardrails": false}}
}'

2. 尝试禁用某个调用的 guardrails

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_VIRTUAL_KEY' \
--data '{
"model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "user",
        "content": "Think of 10 random colors."
      }
    ],
    "metadata": {"guardrails": {"hide_secrets": false}}
}'

3. 获取 403 错误

{
    "error": {
        "message": {
            "error": "Your team does not have permission to modify guardrails."
        },
        "type": "auth_error",
        "param": "None",
        "code": 403
    }
}

预计不会在您的回调的服务器日志中看到 +1 412-612-9992。

信息

由于 api key=sk-jNm1Zar7XfNdZXp49Z1kSQ 具有 "permissions": {"pii_masking": true}，因此 pii_masking guardrail 在此请求上运行。

规范

YAML 上的 `guardrails` 配置

guardrails:
  - guardrail_name: string     # Required: Name of the guardrail
    litellm_params:            # Required: Configuration parameters
      guardrail: string        # Required: One of "aporia", "bedrock", "guardrails_ai", "lakera", "presidio", "hide-secrets"
      mode: Union[string, List[string], Mode]             # Required: One or more of "pre_call", "post_call", "during_call", "logging_only"
      api_key: string          # Required: API key for the guardrail service
      api_base: string         # Optional: Base URL for the guardrail service
      default_on: boolean      # Optional: Default False. When set to True, will run on every request, does not need client to specify guardrail in request
    guardrail_info:            # Optional[Dict]: Additional information about the guardrail
      

模式规范

from litellm.types.guardrails import Mode

mode = Mode(
    tags={"User-Agent: claude-cli": "logging_only"},
    default="logging_only"
)

`guardrails` 请求参数

可以将 guardrails 参数传递到任何 LiteLLM Proxy 端点（/chat/completions、/completions、/embeddings）。

格式选项

简单列表格式

"guardrails": [
    "aporia-pre-guard",
    "aporia-post-guard"
]

高级字典格式

在这种格式下，字典键是您想要运行的 guardrail_name

"guardrails": {
    "aporia-pre-guard": {
        "extra_body": {
            "success_threshold": 0.9,
            "other_param": "value"
        }
    }
}

类型定义

guardrails: Union[
    List[str],                              # Simple list of guardrail names
    Dict[str, DynamicGuardrailParams]       # Advanced configuration
]

class DynamicGuardrailParams:
    extra_body: Dict[str, Any]              # Additional parameters for the guardrail

1. 在 LiteLLM config.yaml 中定义 guardrails​

mode (事件钩子) 的支持值​

负载均衡 Guardrails​

2. 启动 LiteLLM Gateway​

3. 测试请求​

默认开启 Guardrails​

Guardrail 策略​

客户端使用 Guardrails​

自行测试 (OSS)​

暴露给您的用户 (Enterprise)​

1. 查看可用 Guardrails​

2. 应用 Guardrails​

3. 使用 Mock LLM 完成项进行测试​

4. ✨ 将动态参数传递给 Guardrail​

Proxy 管理员控制​

监控 Guardrails​

设置​

追踪的 Guardrail 成功案例​

追踪的 Guardrail 失败案例​

✨ 每个 API 密钥控制 Guardrails​

✨ 基于标签的 Guardrail 模式​

✨ 模型级别 Guardrails​

✨ 禁止团队开启/关闭 guardrails​

1. 禁止团队修改 guardrails​

2. 尝试禁用某个调用的 guardrails​

3. 获取 403 错误​

规范​

YAML 上的 guardrails 配置​

guardrails 请求参数​

格式选项​

类型定义​

1. 在 LiteLLM config.yaml 中定义 guardrails

`mode` (事件钩子) 的支持值

负载均衡 Guardrails

2. 启动 LiteLLM Gateway

3. 测试请求

默认开启 Guardrails

Guardrail 策略

客户端使用 Guardrails

自行测试 (OSS)

暴露给您的用户 (Enterprise)

1. 查看可用 Guardrails

2. 应用 Guardrails

3. 使用 Mock LLM 完成项进行测试

4. ✨ 将动态参数传递给 Guardrail

Proxy 管理员控制

监控 Guardrails

设置

追踪的 Guardrail 成功案例

追踪的 Guardrail 失败案例

✨ 每个 API 密钥控制 Guardrails

✨ 基于标签的 Guardrail 模式

✨ 模型级别 Guardrails

✨ 禁止团队开启/关闭 guardrails

1. 禁止团队修改 guardrails

2. 尝试禁用某个调用的 guardrails

3. 获取 403 错误

规范

YAML 上的 `guardrails` 配置

`guardrails` 请求参数

格式选项

类型定义