跳至主要内容

护栏 - 快速入门

在 LiteLLM 代理（AI 网关）上设置提示词注入检测及 PII（个人身份信息）掩码

1. 在您的 LiteLLM config.yaml 中定义护栏​

在 guardrails 部分下设置您的护栏

model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: general-guard
    litellm_params:
      guardrail: cato_networks
      mode: [pre_call, post_call]
      api_key: os.environ/CATO_API_KEY
      api_base: os.environ/CATO_API_BASE
      default_on: true # Optional
  
  - guardrail_name: "aporia-pre-guard"
    litellm_params:
      guardrail: aporia  # supported values: "aporia", "lakera"
      mode: "during_call"
      api_key: os.environ/APORIA_API_KEY_1
      api_base: os.environ/APORIA_API_BASE_1
  - guardrail_name: "aporia-post-guard"
    litellm_params:
      guardrail: aporia  # supported values: "aporia", "lakera"
      mode: "post_call"
      api_key: os.environ/APORIA_API_KEY_2
      api_base: os.environ/APORIA_API_BASE_2
    guardrail_info: # Optional field, info is returned on GET /guardrails/list
      # you can enter any fields under info for consumers of your guardrail
      params:
        - name: "toxicity_score"
          type: "float"
          description: "Score between 0-1 indicating content toxicity level"
        - name: "pii_detection"
          type: "boolean"

# Example Presidio guardrail config with entity actions + confidence score thresholds
  - guardrail_name: "presidio-pii"
    litellm_params:
      guardrail: presidio
      mode: "pre_call"
      presidio_language: "en"
      pii_entities_config:
        CREDIT_CARD: "MASK"
        EMAIL_ADDRESS: "MASK"
        US_SSN: "MASK"
      presidio_score_thresholds:  # minimum confidence scores for keeping detections
        CREDIT_CARD: 0.8
        EMAIL_ADDRESS: 0.6

# Example Pillar Security config via Generic Guardrail API
  - guardrail_name: "pillar-security"
    litellm_params:
      guardrail: generic_guardrail_api
      mode: [pre_call, post_call]
      api_base: https://api.pillar.security/api/v1/integrations/litellm
      api_key: os.environ/PILLAR_API_KEY
      additional_provider_specific_params:
        plr_mask: true
        plr_evidence: true
        plr_scanners: true

对于通用护栏 API，您还可以设置静态标头（headers：每次请求发送的键/值对）和动态标头（extra_headers：要转发的客户端标头名称列表）。请参阅 通用护栏 API - 静态和动态标头。

mode（事件钩子）的受支持值​

  • pre_call 在 LLM 调用之前运行，针对输入
  • post_call 在 LLM 调用之后运行，针对输入和输出
  • during_call 在 LLM 调用期间运行，针对输入。与 pre_call 相同，但与 LLM 调用并行运行。在 guardrail 检查完成之前不会返回响应
  • 上述值的列表，用于运行多种模式，例如 mode: [pre_call, post_call]

在护栏评估中跳过系统消息​

您可以阻止统一护栏扫描 role: system 内容，同时仍将完整的 messages 列表发送给模型。

全局 — 在 litellm_settings 中

litellm_settings:
  skip_system_message_in_guardrail: true

针对单个护栏 — 在该护栏的 litellm_params 下：设置 skip_system_message_in_guardrail: true 或 false。如果省略，则使用全局 litellm_settings 值；针对单个护栏设置 false 会强制包含系统消息，即使全局标志为 true。

通过 LiteLLM UI — 在 LiteLLM 管理仪表板中创建或编辑护栏时，设置在护栏中跳过系统消息（创建时在“基本信息”下，或在编辑/护栏设置流程中）

UI 选项效果
使用全局默认值使用代理配置中的 litellm_settings.skip_system_message_in_guardrail
是 — 从护栏扫描中排除设置单个护栏的 skip_system_message_in_guardrail: true
否 — 始终包含在扫描中设置单个护栏的 skip_system_message_in_guardrail: false（覆盖全局跳过设置）

适用范围： 仅限统一护栏路径（实现 apply_guardrail 并通过 LiteLLM 消息转换层的提供商），适用于 OpenAI 聊天补全 (/v1/chat/completions) 和 Anthropic 消息 (/v1/messages)。示例包括 Presidio、Bedrock 护栏、litellm_content_filter、OpenAI 审核、通用护栏 API 以及定义了 apply_guardrail 的自定义代码护栏。

不适用范围： 仅通过原始请求上的直接钩子运行的护栏（例如 Lakera v2、Aporia、DynamoAI、Javelin、Lasso、Pangea、Model Armor、Azure 内容安全钩子、Guardrails AI、AIM、Cato Networks、工具权限、MCP 安全）。在这些端点使用相同的转换层之前，它也不适用于其他路由（例如响应 API、嵌入、语音）。

护栏负载均衡​

需要跨多个账户或区域分发护栏请求？有关详情，请参阅 护栏负载均衡

  • 跨多个 AWS Bedrock 账户进行负载均衡（有助于速率限制管理）
  • 跨护栏实例的加权分发
  • 多区域护栏部署

2. 启动 LiteLLM Gateway​

litellm --config config.yaml --detailed_debug

3. 测试请求​

Langchain、OpenAI SDK 用法示例

由于请求中的 ishaan@berri.ai 是 PII，预计此测试会失败

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi my email is ishaan@berri.ai"}
    ],
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

失败时的预期响应

{
  "error": {
    "message": {
      "error": "Violated guardrail policy",
      "aporia_ai_response": {
        "action": "block",
        "revised_prompt": null,
        "revised_response": "Aporia detected and blocked PII",
        "explain_log": null
      }
    },
    "type": "None",
    "param": "None",
    "code": "400"
  }
}

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi what is the weather"}
    ],
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

默认开启的护栏​

在护栏配置中设置 default_on: true，即可在每次请求时运行该护栏。如果您希望在每次请求时运行护栏而无需用户手动指定，这非常有用。

注意： 即使用户指定了不同的护栏或空的护栏数组，这些护栏仍会运行。

guardrails:
  - guardrail_name: "aporia-pre-guard"
    litellm_params:
      guardrail: aporia
      mode: "pre_call"
      default_on: true

测试请求

在此请求中，由于设置了 default_on: true，护栏 aporia-pre-guard 将在每次请求时运行。

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi my email is ishaan@berri.ai"}
    ]
  }'

预期响应

您的响应标头将包含 x-litellm-applied-guardrails，其中显示已应用的护栏

x-litellm-applied-guardrails: aporia-pre-guard

护栏策略​

需要更多控制权？请使用 护栏策略 来实现：

  • 将护栏组合成可重用的策略
  • 为特定团队、密钥或模型启用/禁用护栏
  • 从现有策略继承并覆盖特定护栏

在客户端使用护栏​

自行测试 (OSS)​

将 guardrails 传递到您的请求体中进行测试

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi my email is ishaan@berri.ai"}
    ],
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

向您的用户公开 (Enterprise)​

遵循此简单工作流程来实施和调整护栏

1. 查看可用护栏​

首先，检查哪些护栏可用及其参数

调用 /guardrails/list 以查看可用护栏及护栏信息（支持的参数、描述等）

curl -X GET 'http://0.0.0.0:4000/guardrails/list'

预期响应

{
    "guardrails": [
        {
        "guardrail_name": "aporia-post-guard",
        "guardrail_info": {
            "params": [
            {
                "name": "toxicity_score",
                "type": "float",
                "description": "Score between 0-1 indicating content toxicity level"
            },
            {
                "name": "pii_detection",
                "type": "boolean"
            }
            ]
        }
        }
    ]
}

此配置将返回上述 /guardrails/list 响应。guardrail_info 字段是可选的，您可以为护栏的使用者在 info 下添加任何字段

- guardrail_name: "aporia-post-guard"
    litellm_params:
      guardrail: aporia  # supported values: "aporia", "lakera"
      mode: "post_call"
      api_key: os.environ/APORIA_API_KEY_2
      api_base: os.environ/APORIA_API_BASE_2
    guardrail_info: # Optional field, info is returned on GET /guardrails/list
      # you can enter any fields under info for consumers of your guardrail
      params:
        - name: "toxicity_score"
          type: "float"
          description: "Score between 0-1 indicating content toxicity level"
        - name: "pii_detection"
          type: "boolean"

2. 应用护栏​

将选定的护栏添加到您的聊天补全请求中

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "your message"}],
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

3. 使用模拟 LLM 补全进行测试​

发送 mock_response 以在不进行 LLM 调用的情况下测试护栏。有关 mock_response 的更多信息，请查看 此处

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi my email is ishaan@berri.ai"}
    ],
    "mock_response": "This is a mock response",
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

4. ✨ 向护栏传递动态参数​

信息

✨ 这是企业版专属功能 获取免费试用

使用此功能可向护栏 API 调用传递额外参数，例如成功阈值。查看 guardrails 规范了解详情

设置 guardrails={"aporia-pre-guard": {"extra_body": {"success_threshold": 0.9}}} 以向护栏传递额外参数

在此示例中，success_threshold=0.9 被传递到 aporia-pre-guard 护栏的请求体中

import openai
client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ],
    extra_body={
      "guardrails": {
        "aporia-pre-guard": {
          "extra_body": {
            "success_threshold": 0.9
          }
        }
      }
    }

)

print(response)
curl --location 'http://0.0.0.0:4000/chat/completions' \
    --header 'Content-Type: application/json' \
    --data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
        "role": "user",
        "content": "what llm are you"
        }
    ],
    "guardrails": {
      "aporia-pre-guard": {
        "extra_body": {
          "success_threshold": 0.9
        }
      }
    }
}'

代理管理控制​

监控护栏​

监控哪些护栏已执行以及它们是通过还是失败。例如，当护栏出现异常并拦截了我们本不打算拦截的请求时。

:::

设置​

  1. 将 LiteLLM 连接到受支持的日志记录提供商
  2. 使用 guardrails 参数发出请求
  3. 检查您的日志记录提供商中的护栏追踪记录

追踪护栏成功​

追踪护栏失败​

✨ 按 API 密钥控制护栏​

信息

✨ 这是企业版专属功能 获取免费试用

使用此功能可控制每个 API 密钥运行哪些护栏。在本教程中，我们仅希望为 1 个 API 密钥运行以下护栏

  • guardrails: ["aporia-pre-guard", "aporia-post-guard"]

步骤 1 创建带有 guardrail 设置的 Key

curl -X POST 'http://0.0.0.0:4000/key/generate' \
    -H 'Authorization: Bearer sk-1234' \
    -H 'Content-Type: application/json' \
    -d '{
            "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
    }'
curl --location 'http://0.0.0.0:4000/key/update' \
    --header 'Authorization: Bearer sk-1234' \
    --header 'Content-Type: application/json' \
    --data '{
        "key": "sk-jNm1Zar7XfNdZXp49Z1kSQ",
        "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
}'

步骤 2 使用新的 Key 进行测试

curl --location 'http://0.0.0.0:4000/chat/completions' \
    --header 'Authorization: Bearer sk-jNm1Zar7XfNdZXp49Z1kSQ' \
    --header 'Content-Type: application/json' \
    --data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
        "role": "user",
        "content": "my email is ishaan@berri.ai"
        }
    ]
}'

✨ 基于标签的护栏模式​

信息

✨ 这是企业版专属功能 获取免费试用

基于 User-Agent 标头运行护栏。这对于在 OpenWebUI 上运行调用前检查，但在 Claude CLI 的日志中仅进行掩码处理非常有用。

default 和标签值都可以是单个模式字符串或模式列表。

model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: "guardrails_ai-guard"
    litellm_params:
      guardrail: guardrails_ai
      guard_name: "pii_detect" # 👈 Guardrail AI guard name
      mode:
        tags:
            "User-Agent: claude-cli": "logging_only"                 # Claude CLI - only mask in logs
        default: "pre_call"               # Default mode when no tags match
      api_base: os.environ/GUARDRAILS_AI_API_BASE # 👈 Guardrails AI API Base. Defaults to "http://0.0.0.0:8000"
      default_on: true # run on every request
Per guardrailmodel_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: "guardrails_ai-guard"
    litellm_params:
      guardrail: guardrails_ai
      guard_name: "pii_detect"
      mode:
        tags:
            "User-Agent: claude-cli": "logging_only"
        default: ["pre_call", "post_call"]  # Run on both pre and post call when no tags match
      api_base: os.environ/GUARDRAILS_AI_API_BASE
      default_on: true
model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: "guardrails_ai-guard"
    litellm_params:
      guardrail: guardrails_ai
      guard_name: "pii_detect"
      mode:
        tags:
            "User-Agent: claude-cli": ["pre_call", "post_call"]  # Run both pre and post call for claude-cli
        default: "logging_only"  # Default to logging only when no tags match
      api_base: os.environ/GUARDRAILS_AI_API_BASE
      default_on: true

✨ 模型级护栏​

信息

✨ 这是企业版专属功能 获取免费试用

这非常适用于您同时拥有本地模型和托管模型的情况，并且只想防止将 PII 发送到托管模型。

model_list:
  - model_name: claude-sonnet-4
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY
      api_base: https://api.anthropic.com/v1
      guardrails: ["azure-text-moderation"]
  - model_name: openai-gpt-4o
    litellm_params:
      model: openai/gpt-4o

guardrails:
  - guardrail_name: "presidio-pii"
    litellm_params:
      guardrail: presidio  # supported values: "aporia", "bedrock", "lakera", "presidio"
      mode: "pre_call"
      presidio_language: "en"  # optional: set default language for PII analysis
      pii_entities_config:
        PERSON: "BLOCK"  # Will mask credit card numbers
  - guardrail_name: azure-text-moderation
    litellm_params:
      guardrail: azure/text_moderations
      mode: "post_call" 
      api_key: os.environ/AZURE_GUARDRAIL_API_KEY
      api_base: os.environ/AZURE_GUARDRAIL_API_BASE

✨ 禁止团队开启/关闭护栏​

信息

✨ 这是企业版专属功能 获取免费试用

1. 禁止团队修改护栏​

curl -X POST 'http://0.0.0.0:4000/team/update' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-d '{
    "team_id": "4198d93c-d375-4c83-8d5a-71e7c5473e50",
    "metadata": {"guardrails": {"modify_guardrails": false}}
}'

2. 尝试为调用禁用护栏​

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer $LITELLM_VIRTUAL_KEY' \
--data '{
"model": "gpt-3.5-turbo",
    "messages": [
      {
        "role": "user",
        "content": "Think of 10 random colors."
      }
    ],
    "metadata": {"guardrails": {"hide_secrets": false}}
}'

3. 获得 403 错误​

{
    "error": {
        "message": {
            "error": "Your team does not have permission to modify guardrails."
        },
        "type": "auth_error",
        "param": "None",
        "code": 403
    }
}

预期不会在回调的服务器日志中看到 +1 412-612-9992。

信息

此请求运行了 pii_masking 护栏，因为 api key=sk-jNm1Zar7XfNdZXp49Z1kSQ 拥有 "permissions": {"pii_masking": true}

规范​

YAML 中的 guardrails 配置​

guardrails:
  - guardrail_name: string     # Required: Name of the guardrail
    litellm_params:            # Required: Configuration parameters
      guardrail: string        # Required: One of "aporia", "bedrock", "guardrails_ai", "lakera", "presidio", "hide-secrets"
      mode: Union[string, List[string], Mode]             # Required: One or more of "pre_call", "post_call", "during_call", "logging_only"
      api_key: string          # Required: API key for the guardrail service
      api_base: string         # Optional: Base URL for the guardrail service
      default_on: boolean      # Optional: Default False. When set to True, will run on every request, does not need client to specify guardrail in request
    guardrail_info:            # Optional[Dict]: Additional information about the guardrail

模式规范

default 和标签值均接受单个字符串或字符串列表。

from litellm.types.guardrails import Mode

# Single default mode
mode = Mode(
    tags={"User-Agent: claude-cli": "logging_only"},
    default="logging_only"
)

# Multiple default modes
mode = Mode(
    tags={"User-Agent: claude-cli": "logging_only"},
    default=["pre_call", "post_call"]
)

# Multiple modes on a tag value
mode = Mode(
    tags={"User-Agent: claude-cli": ["pre_call", "post_call"]},
    default="logging_only"
)

guardrails 请求参数​

guardrails 参数可以传递给任何 LiteLLM 代理端点 (/chat/completions, /completions, /embeddings)。

格式选项​

  1. 简单列表格式
"guardrails": [
    "aporia-pre-guard",
    "aporia-post-guard"
]
  1. 高级字典格式

在此格式中，字典键为您想要运行的 guardrail_name

"guardrails": {
    "aporia-pre-guard": {
        "extra_body": {
            "success_threshold": 0.9,
            "other_param": "value"
        }
    }
}

类型定义​

guardrails: Union[
    List[str],                              # Simple list of guardrail names
    Dict[str, DynamicGuardrailParams]       # Advanced configuration
]

class DynamicGuardrailParams:
    extra_body: Dict[str, Any]              # Additional parameters for the guardrail
🚅
LiteLLM 企业版
SSO/SAML、审计日志、支出跟踪、多团队管理和护栏 —— 专为生产环境构建。
了解更多 →