Arize AI

AI 可观测性和评估平台

提示

这是社区维护的，如果您遇到错误，请提交一个 issue https://github.com/BerriAI/litellm

先决条件

在 Arize AI 创建账户

快速入门

只需 2 行代码，即可使用 arize 立即记录您在 所有提供商上的响应

您也可以使用 instrumentor 选项代替回调，您可以在此处找到它。

litellm.callbacks = ["arize"]

import litellm
import os

os.environ["ARIZE_SPACE_KEY"] = ""
os.environ["ARIZE_API_KEY"] = ""

# LLM API Keys
os.environ['OPENAI_API_KEY']=""

# set arize as a callback, litellm will send the data to arize
litellm.callbacks = ["arize"]
 
# openai call
response = litellm.completion(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ]
)

与 LiteLLM Proxy 一起使用

1. 设置 config.yaml

model_list:
  - model_name: gpt-4
    litellm_params:
      model: openai/fake
      api_key: fake-key
      api_base: https://exampleopenaiendpoint-production.up.railway.app/

litellm_settings:
  callbacks: ["arize"]

general_settings:
  master_key: "sk-1234" # can also be set as an environment variable

environment_variables:
    ARIZE_SPACE_KEY: "d0*****"
    ARIZE_API_KEY: "141a****"
    ARIZE_ENDPOINT: "https://otlp.arize.com/v1" # OPTIONAL - your custom arize GRPC api endpoint
    ARIZE_HTTP_ENDPOINT: "https://otlp.arize.com/v1" # OPTIONAL - your custom arize HTTP api endpoint. Set either this or ARIZE_ENDPOINT or Neither (defaults to https://otlp.arize.com/v1 on grpc)

2. 启动代理

litellm --config config.yaml

3. 测试一下！

curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{ "model": "gpt-4", "messages": [{"role": "user", "content": "Hi 👋 - i'm openai"}]}'

按请求传递 Arize Space/Key

支持的参数

• arize_api_key
• arize_space_key

SDK

import litellm
import os

# LLM API Keys
os.environ['OPENAI_API_KEY']=""

# set arize as a callback, litellm will send the data to arize
litellm.callbacks = ["arize"]
 
# openai call
response = litellm.completion(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ],
  arize_api_key=os.getenv("ARIZE_SPACE_2_API_KEY"),
  arize_space_key=os.getenv("ARIZE_SPACE_2_KEY"),
)

代理

1. 设置 config.yaml

model_list:
  - model_name: gpt-4
    litellm_params:
      model: openai/fake
      api_key: fake-key
      api_base: https://exampleopenaiendpoint-production.up.railway.app/

litellm_settings:
  callbacks: ["arize"]

general_settings:
  master_key: "sk-1234" # can also be set as an environment variable

2. 启动代理

litellm --config /path/to/config.yaml

3. 测试一下！

CURL

curl -X POST 'http://0.0.0.0:4000/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
  "model": "gpt-4",
  "messages": [{"role": "user", "content": "Hi 👋 - i'm openai"}],
  "arize_api_key": "ARIZE_SPACE_2_API_KEY",
  "arize_space_key": "ARIZE_SPACE_2_KEY"
}'

OpenAI Python

import openai
client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ],
    extra_body={
      "arize_api_key": "ARIZE_SPACE_2_API_KEY",
      "arize_space_key": "ARIZE_SPACE_2_KEY"
    }
)

print(response)

支持和与创始人交流

• 安排演示 👋
• 社区 Discord 💭
• 我们的电话 📞 +1 (770) 8783-106 / ‭+1 (412) 618-6238‬
• 我们的邮箱 ✉️ ishaan@berri.ai / krrish@berri.ai