计费

根据内部团队、外部客户的使用情况计费

🚨 要求

• 设置 Lago，用于基于使用情况的计费。我们建议按照他们的 Stripe 教程

步骤

• 将代理连接到 Lago
• 设置要计费的 ID（客户、内部用户、团队）
• 开始！

快速入门

根据内部团队的使用情况计费

1. 将代理连接到 Lago

在代理的 config.yaml 中将 'lago' 设置为回调

model_list:
  - model_name: fake-openai-endpoint
    litellm_params:
      model: openai/fake
      api_key: fake-key
      api_base: https://exampleopenaiendpoint-production.up.railway.app/

litellm_settings:
  callbacks: ["lago"] # 👈 KEY CHANGE

general_settings:
  master_key: sk-1234

将 Lago 密钥添加到环境变量中

export LAGO_API_BASE="https://:3000" # self-host - https://docs.getlago.com/guide/self-hosted/docker#run-the-app
export LAGO_API_KEY="3e29d607-de54-49aa-a019-ecf585729070" # Get key - https://docs.getlago.com/guide/self-hosted/docker#find-your-api-key
export LAGO_API_EVENT_CODE="openai_tokens" # name of lago billing code
export LAGO_API_CHARGE_BY="team_id" # 👈 Charges 'team_id' attached to proxy key

启动代理

litellm --config /path/to/config.yaml

2. 为内部团队创建密钥

curl 'http://0.0.0.0:4000/key/generate' \
--header 'Authorization: Bearer sk-1234' \
--header 'Content-Type: application/json' \
--data-raw '{"team_id": "my-unique-id"}' # 👈 Internal Team's ID

响应对象

{
  "key": "sk-tXL0wt5-lOOVK9sfY2UacA",
}

3. 开始计费！

Curl

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer sk-tXL0wt5-lOOVK9sfY2UacA' \ # 👈 Team's Key
--data ' {
      "model": "fake-openai-endpoint",
      "messages": [
        {
          "role": "user",
          "content": "what llm are you"
        }
      ],
    }
'

OpenAI Python SDK

import openai
client = openai.OpenAI(
    api_key="sk-tXL0wt5-lOOVK9sfY2UacA", # 👈 Team's Key
    base_url="http://0.0.0.0:4000"
)

# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [
    {
        "role": "user",
        "content": "this is a test request, write a short poem"
    }
])

print(response)

Langchain

from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
    SystemMessagePromptTemplate,
)
from langchain.schema import HumanMessage, SystemMessage
import os 

os.environ["OPENAI_API_KEY"] = "sk-tXL0wt5-lOOVK9sfY2UacA" # 👈 Team's Key

chat = ChatOpenAI(
    openai_api_base="http://0.0.0.0:4000",
    model = "gpt-3.5-turbo",
    temperature=0.1,
)

messages = [
    SystemMessage(
        content="You are a helpful assistant that im using to make a test request to."
    ),
    HumanMessage(
        content="test from litellm. tell me why it's amazing in 1 sentence"
    ),
]
response = chat(messages)

print(response)

在 Lago 上查看结果

高级 - Lago 日志对象

这是 LiteLLM 将记录到 Lago 的内容

{
    "event": {
      "transaction_id": "<generated_unique_id>",
      "external_customer_id": <selected_id>, # either 'end_user_id', 'user_id', or 'team_id'. Default 'end_user_id'. 
      "code": os.getenv("LAGO_API_EVENT_CODE"), 
      "properties": {
          "input_tokens": <number>,
          "output_tokens": <number>,
          "model": <string>,
          "response_cost": <number>, # 👈 LITELLM CALCULATED RESPONSE COST - https://github.com/BerriAI/litellm/blob/d43f75150a65f91f60dc2c0c9462ce3ffc713c1f/litellm/utils.py#L1473
      }
    }
}

高级 - 为客户、内部用户计费

对于

• 客户（在 /chat/completion 调用中通过 'user' 参数传递的 ID）= 'end_user_id'
• 内部用户（创建密钥时设置的 ID）= 'user_id'
• 团队（创建密钥时设置的 ID）= 'team_id'

客户计费

1. 将 'LAGO_API_CHARGE_BY' 设置为 'end_user_id'

   export LAGO_API_CHARGE_BY="end_user_id"

2. 测试一下！

   Curl

   curl --location 'http://0.0.0.0:4000/chat/completions' \
   --header 'Content-Type: application/json' \
   --data ' {
         "model": "gpt-3.5-turbo",
         "messages": [
           {
             "role": "user",
             "content": "what llm are you"
           }
         ],
         "user": "my_customer_id" # 👈 whatever your customer id is
       }
   '

   OpenAI Python SDK

   import openai
   client = openai.OpenAI(
       api_key="anything",
       base_url="http://0.0.0.0:4000"
   )
   
   # request sent to model set on litellm proxy, `litellm --model`
   response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [
       {
           "role": "user",
           "content": "this is a test request, write a short poem"
       }
   ], user="my_customer_id") # 👈 whatever your customer id is
   
   print(response)

   Langchain

   from langchain.chat_models import ChatOpenAI
   from langchain.prompts.chat import (
       ChatPromptTemplate,
       HumanMessagePromptTemplate,
       SystemMessagePromptTemplate,
   )
   from langchain.schema import HumanMessage, SystemMessage
   import os 
   
   os.environ["OPENAI_API_KEY"] = "anything"
   
   chat = ChatOpenAI(
       openai_api_base="http://0.0.0.0:4000",
       model = "gpt-3.5-turbo",
       temperature=0.1,
       extra_body={
           "user": "my_customer_id"  # 👈 whatever your customer id is
       }
   )
   
   messages = [
       SystemMessage(
           content="You are a helpful assistant that im using to make a test request to."
       ),
       HumanMessage(
           content="test from litellm. tell me why it's amazing in 1 sentence"
       ),
   ]
   response = chat(messages)
   
   print(response)

内部用户计费

1. 将 'LAGO_API_CHARGE_BY' 设置为 'user_id'

export LAGO_API_CHARGE_BY="user_id"

2. 为该用户创建一个密钥

curl 'http://0.0.0.0:4000/key/generate' \
--header 'Authorization: Bearer <your-master-key>' \
--header 'Content-Type: application/json' \
--data-raw '{"user_id": "my-unique-id"}' # 👈 Internal User's id

响应对象

{
  "key": "sk-tXL0wt5-lOOVK9sfY2UacA",
}

3. 使用该密钥进行 API 调用

import openai
client = openai.OpenAI(
    api_key="sk-tXL0wt5-lOOVK9sfY2UacA", # 👈 Generated key
    base_url="http://0.0.0.0:4000"
)

# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(model="gpt-3.5-turbo", messages = [
    {
        "role": "user",
        "content": "this is a test request, write a short poem"
    }
])

print(response)