跳到主内容

OpenAI

LiteLLM 支持 OpenAI Chat + Embedding 调用。

所需的 API 密钥​

import os 
os.environ["OPENAI_API_KEY"] = "your-api-key"

使用方法​

import os 
from litellm import completion

os.environ["OPENAI_API_KEY"] = "your-api-key"

# openai call
response = completion(
    model = "gpt-4o", 
    messages=[{ "content": "Hello, how are you?","role": "user"}]
)

使用方法 - LiteLLM 代理服务器​

以下是使用 LiteLLM 代理服务器调用 OpenAI 模型的方法

1. 在环境中保存密钥​

export OPENAI_API_KEY=""

2. 启动代理​

model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-3.5-turbo                          # The `openai/` prefix will call openai.chat.completions.create
      api_key: os.environ/OPENAI_API_KEY
  - model_name: gpt-3.5-turbo-instruct
    litellm_params:
      model: text-completion-openai/gpt-3.5-turbo-instruct # The `text-completion-openai/` prefix will call openai.completions.create
      api_key: os.environ/OPENAI_API_KEY

3. 测试​

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
      "model": "gpt-3.5-turbo",
      "messages": [
        {
          "role": "user",
          "content": "what llm are you"
        }
      ]
    }
'

可选密钥 - OpenAI 组织、OpenAI API Base​

import os 
os.environ["OPENAI_ORGANIZATION"] = "your-org-id"       # OPTIONAL
os.environ["OPENAI_BASE_URL"] = "https://your_host/v1"     # OPTIONAL

OpenAI Chat 补全模型​

模型名称函数调用
gpt-4.1response = completion(model="gpt-4.1", messages=messages)
gpt-4.1-miniresponse = completion(model="gpt-4.1-mini", messages=messages)
gpt-4.1-nanoresponse = completion(model="gpt-4.1-nano", messages=messages)
o4-miniresponse = completion(model="o4-mini", messages=messages)
o3-miniresponse = completion(model="o3-mini", messages=messages)
o3response = completion(model="o3", messages=messages)
o1-miniresponse = completion(model="o1-mini", messages=messages)
o1-previewresponse = completion(model="o1-preview", messages=messages)
gpt-4o-miniresponse = completion(model="gpt-4o-mini", messages=messages)
gpt-4o-mini-2024-07-18response = completion(model="gpt-4o-mini-2024-07-18", messages=messages)
gpt-4oresponse = completion(model="gpt-4o", messages=messages)
gpt-4o-2024-08-06response = completion(model="gpt-4o-2024-08-06", messages=messages)
gpt-4o-2024-05-13response = completion(model="gpt-4o-2024-05-13", messages=messages)
gpt-4-turboresponse = completion(model="gpt-4-turbo", messages=messages)
gpt-4-turbo-previewresponse = completion(model="gpt-4-0125-preview", messages=messages)
gpt-4-0125-previewresponse = completion(model="gpt-4-0125-preview", messages=messages)
gpt-4-1106-previewresponse = completion(model="gpt-4-1106-preview", messages=messages)
gpt-3.5-turbo-1106response = completion(model="gpt-3.5-turbo-1106", messages=messages)
gpt-3.5-turboresponse = completion(model="gpt-3.5-turbo", messages=messages)
gpt-3.5-turbo-0301response = completion(model="gpt-3.5-turbo-0301", messages=messages)
gpt-3.5-turbo-0613response = completion(model="gpt-3.5-turbo-0613", messages=messages)
gpt-3.5-turbo-16kresponse = completion(model="gpt-3.5-turbo-16k", messages=messages)
gpt-3.5-turbo-16k-0613response = completion(model="gpt-3.5-turbo-16k-0613", messages=messages)
gpt-4response = completion(model="gpt-4", messages=messages)
gpt-4-0314response = completion(model="gpt-4-0314", messages=messages)
gpt-4-0613response = completion(model="gpt-4-0613", messages=messages)
gpt-4-32kresponse = completion(model="gpt-4-32k", messages=messages)
gpt-4-32k-0314response = completion(model="gpt-4-32k-0314", messages=messages)
gpt-4-32k-0613response = completion(model="gpt-4-32k-0613", messages=messages)

这些也支持 OPENAI_BASE_URL 环境变量，可用于指定自定义 API 端点。

OpenAI 视觉模型​

模型名称函数调用
gpt-4oresponse = completion(model="gpt-4o", messages=messages)
gpt-4-turboresponse = completion(model="gpt-4-turbo", messages=messages)
gpt-4-vision-previewresponse = completion(model="gpt-4-vision-preview", messages=messages)

使用方法​

import os 
from litellm import completion

os.environ["OPENAI_API_KEY"] = "your-api-key"

# openai call
response = completion(
    model = "gpt-4-vision-preview", 
    messages=[
        {
            "role": "user",
            "content": [
                            {
                                "type": "text",
                                "text": "What’s in this image?"
                            },
                            {
                                "type": "image_url",
                                "image_url": {
                                "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"
                                }
                            }
                        ]
        }
    ],
)

PDF 文件解析​

OpenAI 新增了一种 file 消息类型，允许您传入 PDF 文件并将其解析为结构化输出。了解更多

import base64
from litellm import completion

with open("draconomicon.pdf", "rb") as f:
    data = f.read()

base64_string = base64.b64encode(data).decode("utf-8")

completion = completion(
    model="gpt-4o",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "file",
                    "file": {
                        "filename": "draconomicon.pdf",
                        "file_data": f"data:application/pdf;base64,{base64_string}",
                    }
                },
                {
                    "type": "text",
                    "text": "What is the first dragon in the book?",
                }
            ],
        },
    ],
)

print(completion.choices[0].message.content)

OpenAI 微调模型​

模型名称函数调用
微调 gpt-4-0613response = completion(model="ft:gpt-4-0613", messages=messages)
微调 gpt-4o-2024-05-13response = completion(model="ft:gpt-4o-2024-05-13", messages=messages)
微调 gpt-3.5-turbo-0125response = completion(model="ft:gpt-3.5-turbo-0125", messages=messages)
微调 gpt-3.5-turbo-1106response = completion(model="ft:gpt-3.5-turbo-1106", messages=messages)
微调 gpt-3.5-turbo-0613response = completion(model="ft:gpt-3.5-turbo-0613", messages=messages)

OpenAI 音频转录​

LiteLLM 支持 OpenAI 音频转录端点。

支持的模型

模型名称函数调用
whisper-1response = completion(model="whisper-1", file=audio_file)
gpt-4o-transcriberesponse = completion(model="gpt-4o-transcribe", file=audio_file)
gpt-4o-mini-transcriberesponse = completion(model="gpt-4o-mini-transcribe", file=audio_file)
from litellm import transcription
import os 

# set api keys 
os.environ["OPENAI_API_KEY"] = ""
audio_file = open("/path/to/audio.mp3", "rb")

response = transcription(model="gpt-4o-transcribe", file=audio_file)

print(f"response: {response}")

高级​

获取 OpenAI API 响应头​

设置 litellm.return_response_headers = True 以获取 OpenAI 的原始响应头

您始终可以从 litellm.completion() 和 litellm.embedding() 函数中获取 _response_headers 字段

litellm.return_response_headers = True

# /chat/completion
response = completion(
    model="gpt-4o-mini",
    messages=[
        {
            "role": "user",
            "content": "hi",
        }
    ],
)
print(f"response: {response}")
print("_response_headers=", response._response_headers)
预期的 OpenAI 响应头
{
  "date": "Sat, 20 Jul 2024 22:05:23 GMT",
  "content-type": "application/json",
  "transfer-encoding": "chunked",
  "connection": "keep-alive",
  "access-control-allow-origin": "*",
  "openai-model": "text-embedding-ada-002",
  "openai-organization": "*****",
  "openai-processing-ms": "20",
  "openai-version": "2020-10-01",
  "strict-transport-security": "max-age=15552000; includeSubDomains; preload",
  "x-ratelimit-limit-requests": "5000",
  "x-ratelimit-limit-tokens": "5000000",
  "x-ratelimit-remaining-requests": "4999",
  "x-ratelimit-remaining-tokens": "4999999",
  "x-ratelimit-reset-requests": "12ms",
  "x-ratelimit-reset-tokens": "0s",
  "x-request-id": "req_cc37487bfd336358231a17034bcfb4d9",
  "cf-cache-status": "DYNAMIC",
  "set-cookie": "__cf_bm=E_FJY8fdAIMBzBE2RZI2.OkMIO3lf8Hz.ydBQJ9m3q8-1721513123-1.0.1.1-6OK0zXvtd5s9Jgqfz66cU9gzQYpcuh_RLaUZ9dOgxR9Qeq4oJlu.04C09hOTCFn7Hg.k.2tiKLOX24szUE2shw; path=/; expires=Sat, 20-Jul-24 22:35:23 GMT; domain=.api.openai.com; HttpOnly; Secure; SameSite=None, *cfuvid=SDndIImxiO3U0aBcVtoy1TBQqYeQtVDo1L6*Nlpp7EU-1721513123215-0.0.1.1-604800000; path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None",
  "x-content-type-options": "nosniff",
  "server": "cloudflare",
  "cf-ray": "8a66409b4f8acee9-SJC",
  "content-encoding": "br",
  "alt-svc": "h3=\":443\"; ma=86400"
}

并行函数调用​

此处可查看使用 litellm 进行并行函数调用的详细演练 此处

import litellm
import json
# set openai api key
import os
os.environ['OPENAI_API_KEY'] = "" # litellm reads OPENAI_API_KEY from .env and sends the request
# Example dummy function hard coded to return the same weather
# In production, this could be your backend API or an external API
def get_current_weather(location, unit="fahrenheit"):
    """Get the current weather in a given location"""
    if "tokyo" in location.lower():
        return json.dumps({"location": "Tokyo", "temperature": "10", "unit": "celsius"})
    elif "san francisco" in location.lower():
        return json.dumps({"location": "San Francisco", "temperature": "72", "unit": "fahrenheit"})
    elif "paris" in location.lower():
        return json.dumps({"location": "Paris", "temperature": "22", "unit": "celsius"})
    else:
        return json.dumps({"location": location, "temperature": "unknown"})

messages = [{"role": "user", "content": "What's the weather like in San Francisco, Tokyo, and Paris?"}]
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["location"],
            },
        },
    }
]

response = litellm.completion(
    model="gpt-3.5-turbo-1106",
    messages=messages,
    tools=tools,
    tool_choice="auto",  # auto is default, but we'll be explicit
)
print("\nLLM Response1:\n", response)
response_message = response.choices[0].message
tool_calls = response.choices[0].message.tool_calls

设置 completion 调用的 extra_headers​

import os 
from litellm import completion

os.environ["OPENAI_API_KEY"] = "your-api-key"

response = completion(
    model = "gpt-3.5-turbo", 
    messages=[{ "content": "Hello, how are you?","role": "user"}],
    extra_headers={"AI-Resource Group": "ishaan-resource"}
)

设置 completion 调用的 Organization-ID​

可通过以下任一方式设置

  • 环境变量 OPENAI_ORGANIZATION
  • litellm.completion(model=model, organization="your-organization-id") 的参数
  • 设置为 litellm.organization="your-organization-id"
import os 
from litellm import completion

os.environ["OPENAI_API_KEY"] = "your-api-key"
os.environ["OPENAI_ORGANIZATION"] = "your-org-id" # OPTIONAL

response = completion(
    model = "gpt-3.5-turbo", 
    messages=[{ "content": "Hello, how are you?","role": "user"}]
)

设置 ssl_verify=False​

通过设置您自己的 httpx.Client 来完成

  • 对于 litellm.completion，设置 litellm.client_session=httpx.Client(verify=False)
  • 对于 litellm.acompletion，设置 litellm.aclient_session=AsyncClient.Client(verify=False)
import litellm, httpx

# for completion
litellm.client_session = httpx.Client(verify=False)
response = litellm.completion(
    model="gpt-3.5-turbo",
    messages=messages,
)

# for acompletion
litellm.aclient_session = httpx.AsyncClient(verify=False)
response = litellm.acompletion(
    model="gpt-3.5-turbo",
    messages=messages,
)

将 OpenAI 代理与 LiteLLM 一起使用​

import os 
import litellm
from litellm import completion

os.environ["OPENAI_API_KEY"] = ""

# set custom api base to your proxy
# either set .env or litellm.api_base
# os.environ["OPENAI_BASE_URL"] = "https://your_host/v1"
litellm.api_base = "https://your_host/v1"


messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion("openai/your-model-name", messages)

如果您需要动态设置 api_base，只需在 completions 中传入即可 - completions(...,api_base="your-proxy-api-base")

更多信息请查看 设置 API Base/密钥

转发代理请求的组织 ID​

使用 forward_openai_org_id 参数将客户端的 openai 组织 ID 转发到 OpenAI。

  1. 设置 config.yaml
model_list:
  - model_name: "gpt-3.5-turbo"
    litellm_params:
      model: gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY

general_settings:
    forward_openai_org_id: true # 👈 KEY CHANGE
  1. 启动代理
litellm --config config.yaml --detailed_debug

# RUNNING on http://0.0.0.0:4000
  1. 进行 OpenAI 调用
from openai import OpenAI
client = OpenAI(
    api_key="sk-1234",
    organization="my-special-org",
    base_url="http://0.0.0.0:4000"
)

client.chat.completions.create(model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Hello world"}])

在您的日志中应能看到转发的组织 ID

LiteLLM:DEBUG: utils.py:255 - Request to litellm:
LiteLLM:DEBUG: utils.py:255 - litellm.acompletion(... organization='my-special-org',)