Slack - 记录 LLM 输入/输出、异常

信息

我们希望了解如何改进回调！会见 LiteLLM 创始人或加入我们的 discord

前提条件

步骤 1

pip install litellm

步骤 2

从 https://api.slack.com/messaging/webhooks 获取 slack webhook URL

快速开始

创建自定义回调以记录到 slack

我们创建一个自定义回调，用于记录到 slack webhooks，详情请参阅 litellm 上的 自定义回调

def send_slack_alert(
        kwargs,
        completion_response,
        start_time,
        end_time,
):
    print(
        "in custom slack callback func"
    )
    import requests
    import json

    # Define the Slack webhook URL
    # get it from https://api.slack.com/messaging/webhooks
    slack_webhook_url = os.environ['SLACK_WEBHOOK_URL']   # "https://hooks.slack.com/services/<>/<>/<>"

    # Remove api_key from kwargs under litellm_params
    if kwargs.get('litellm_params'):
        kwargs['litellm_params'].pop('api_key', None)
        if kwargs['litellm_params'].get('metadata'):
            kwargs['litellm_params']['metadata'].pop('deployment', None)
    # Remove deployment under metadata
    if kwargs.get('metadata'):
        kwargs['metadata'].pop('deployment', None)
    # Prevent api_key from being logged
    if kwargs.get('api_key'):
        kwargs.pop('api_key', None)

    # Define the text payload, send data available in litellm custom_callbacks
    text_payload = f"""LiteLLM Logging: kwargs: {str(kwargs)}\n\n, response: {str(completion_response)}\n\n, start time{str(start_time)} end time: {str(end_time)}
    """
    payload = {
        "text": text_payload
    }

    # Set the headers
    headers = {
        "Content-type": "application/json"
    }

    # Make the POST request
    response = requests.post(slack_webhook_url, json=payload, headers=headers)

    # Check the response status
    if response.status_code == 200:
        print("Message sent successfully to Slack!")
    else:
        print(f"Failed to send message to Slack. Status code: {response.status_code}")
        print(response.json())

将回调传递给 LiteLLM

litellm.success_callback = [send_slack_alert]

import litellm
litellm.success_callback = [send_slack_alert] # log success
litellm.failure_callback = [send_slack_alert] # log exceptions

# this will raise an exception
response = litellm.completion(
    model="gpt-2",
    messages=[
        {
            "role": "user",
            "content": "Hi 👋 - i'm openai"
        }
    ]
)

支持 & 与创始人交流

• 安排演示 👋
• 社区 Discord 💭
• 我们的电话号码 📞 +1 (770) 8783-106 / ‭+1 (412) 618-6238‬
• 我们的电子邮件 ✉️ ishaan@berri.ai / krrish@berri.ai