跳到主要内容

结构化输出（JSON模式）

快速开始​

from litellm import completion
import os 

os.environ["OPENAI_API_KEY"] = ""

response = completion(
  model="gpt-4o-mini",
  response_format={ "type": "json_object" },
  messages=[
    {"role": "system", "content": "You are a helpful assistant designed to output JSON."},
    {"role": "user", "content": "Who won the world series in 2020?"}
  ]
)
print(response.choices[0].message.content)

检查模型支持​

1. 检查模型是否支持 response_format​

调用 litellm.get_supported_openai_params 来检查模型/提供商是否支持 response_format。

from litellm import get_supported_openai_params

params = get_supported_openai_params(model="anthropic.claude-3", custom_llm_provider="bedrock")

assert "response_format" in params

2. 检查模型是否支持 json_schema​

这用于检查您是否可以传入

  • response_format={ "type": "json_schema", "json_schema": … , "strict": true }
  • response_format=<Pydantic Model>
from litellm import supports_response_schema

assert supports_response_schema(model="gemini-1.5-pro-preview-0215", custom_llm_provider="bedrock")

请查看 model_prices_and_context_window.json，获取支持 response_schema 的模型完整列表。

传入 'json_schema'​

要使用结构化输出，只需指定

response_format: { "type": "json_schema", "json_schema": … , "strict": true }

适用于

  • OpenAI 模型
  • Azure OpenAI 模型
  • xAI 模型 (Grok-2 或更高版本)
  • Google AI Studio - Gemini 模型
  • Vertex AI 模型 (Gemini + Anthropic)
  • Bedrock 模型
  • Anthropic API 模型
  • Groq 模型
  • Ollama 模型
  • Databricks 模型
import os
from litellm import completion 
from pydantic import BaseModel

# add to env var 
os.environ["OPENAI_API_KEY"] = ""

messages = [{"role": "user", "content": "List 5 important events in the XIX century"}]

class CalendarEvent(BaseModel):
  name: str
  date: str
  participants: list[str]

class EventsList(BaseModel):
    events: list[CalendarEvent]

resp = completion(
    model="gpt-4o-2024-08-06",
    messages=messages,
    response_format=EventsList
)

print("Received={}".format(resp))

验证 JSON Schema​

并非所有 Vertex 模型都支持将 json_schema 传递给它们 (例如 gemini-1.5-flash)。为了解决这个问题，LiteLLM 支持客户端验证 json schema。

litellm.enable_json_schema_validation=True

如果设置了 litellm.enable_json_schema_validation=True，LiteLLM 将使用 jsonvalidator 验证 json 响应。

查看代码

# !gcloud auth application-default login - run this to add vertex credentials to your env
import litellm, os
from litellm import completion 
from pydantic import BaseModel 


messages=[
        {"role": "system", "content": "Extract the event information."},
        {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."},
    ]

litellm.enable_json_schema_validation = True
litellm.set_verbose = True # see the raw request made by litellm

class CalendarEvent(BaseModel):
  name: str
  date: str
  participants: list[str]

resp = completion(
    model="gemini/gemini-1.5-pro",
    messages=messages,
    response_format=CalendarEvent,
)

print("Received={}".format(resp))