跳到主要内容

v1.56.3

Krrish Dholakia
Ishaan Jaffer

护栏, 日志记录, 虚拟密钥管理, 新模型

信息

在此获取 LiteLLM 企业版 7 天免费试用。here。

无需致电

新功能​

✨ 记录护栏跟踪​

跟踪护栏失败率，以及护栏是否异常导致请求失败。在此开始

跟踪护栏成功​

跟踪护栏失败​

/guardrails/list​

/guardrails/list 允许客户端查看可用的护栏 + 支持的护栏参数

curl -X GET 'http://0.0.0.0:4000/guardrails/list'

预期响应

{
    "guardrails": [
        {
        "guardrail_name": "aporia-post-guard",
        "guardrail_info": {
            "params": [
            {
                "name": "toxicity_score",
                "type": "float",
                "description": "Score between 0-1 indicating content toxicity level"
            },
            {
                "name": "pii_detection",
                "type": "boolean"
            }
            ]
        }
        }
    ]
}

✨ 使用模拟 LLM 的护栏​

发送 mock_response 来测试护栏，而无需进行 LLM 调用。有关 mock_response 的更多信息，请见此处

curl -i https://:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "hi my email is ishaan@berri.ai"}
    ],
    "mock_response": "This is a mock response",
    "guardrails": ["aporia-pre-guard", "aporia-post-guard"]
  }'

将密钥分配给用户​

您现在可以通过代理 UI 将密钥分配给用户

新模型​

  • openrouter/openai/o1
  • vertex_ai/mistral-large@2411

修复​