跳到主要内容

Meta Llama

属性详细信息
描述Meta 的 Llama API 提供访问 Meta 的大型语言模型系列的能力。
LiteLLM 上的提供商路由meta_llama/
支持的端点/chat/completions, /completions, /responses
API 参考Llama API 参考 ↗

必需变量​

环境变量
os.environ["LLAMA_API_KEY"] = ""  # your Meta Llama API key

支持的模型​

信息

此处列出的所有模型 https://llama.developer.meta.com/docs/models/ 都受支持。我们积极维护模型列表、token 窗口等信息 此处。

模型 ID输入上下文长度输出上下文长度输入模态输出模态
Llama-4-Scout-17B-16E-Instruct-FP8128k4028文本, 图像文本
Llama-4-Maverick-17B-128E-Instruct-FP8128k4028文本, 图像文本
Llama-3.3-70B-Instruct128k4028文本文本
Llama-3.3-8B-Instruct128k4028文本文本

用法 - LiteLLM Python SDK​

非流式​

Meta Llama 非流式续写
import os
import litellm
from litellm import completion

os.environ["LLAMA_API_KEY"] = ""  # your Meta Llama API key

messages = [{"content": "Hello, how are you?", "role": "user"}]

# Meta Llama call
response = completion(model="meta_llama/Llama-3.3-70B-Instruct", messages=messages)

流式​

Meta Llama 流式续写
import os
import litellm
from litellm import completion

os.environ["LLAMA_API_KEY"] = ""  # your Meta Llama API key

messages = [{"content": "Hello, how are you?", "role": "user"}]

# Meta Llama call with streaming
response = completion(
    model="meta_llama/Llama-3.3-70B-Instruct",
    messages=messages,
    stream=True
)

for chunk in response:
    print(chunk)

用法 - LiteLLM Proxy​

将以下内容添加到您的 LiteLLM Proxy 配置文件中

config.yaml
model_list:
  - model_name: meta_llama/Llama-3.3-70B-Instruct
    litellm_params:
      model: meta_llama/Llama-3.3-70B-Instruct
      api_key: os.environ/LLAMA_API_KEY

  - model_name: meta_llama/Llama-3.3-8B-Instruct
    litellm_params:
      model: meta_llama/Llama-3.3-8B-Instruct
      api_key: os.environ/LLAMA_API_KEY

启动您的 LiteLLM Proxy 服务器

启动 LiteLLM Proxy
litellm --config config.yaml

# RUNNING on http://0.0.0.0:4000
通过 Proxy 使用 Meta Llama - 非流式
from openai import OpenAI

# Initialize client with your proxy URL
client = OpenAI(
    base_url="https://:4000",  # Your proxy URL
    api_key="your-proxy-api-key"       # Your proxy API key
)

# Non-streaming response
response = client.chat.completions.create(
    model="meta_llama/Llama-3.3-70B-Instruct",
    messages=[{"role": "user", "content": "Write a short poem about AI."}]
)

print(response.choices[0].message.content)
通过 Proxy 使用 Meta Llama - 流式
from openai import OpenAI

# Initialize client with your proxy URL
client = OpenAI(
    base_url="https://:4000",  # Your proxy URL
    api_key="your-proxy-api-key"       # Your proxy API key
)

# Streaming response
response = client.chat.completions.create(
    model="meta_llama/Llama-3.3-70B-Instruct",
    messages=[{"role": "user", "content": "Write a short poem about AI."}],
    stream=True
)

for chunk in response:
    if chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="")

有关使用 LiteLLM Proxy 的更详细信息，请参阅 LiteLLM Proxy 文档。