Codestral API[Mistral AI]

Codestral 在部分代码补全插件中可用，但也支持直接查询。详情请参阅文档。

API 密钥

# env variable
os.environ['CODESTRAL_API_KEY']

FIM / 补全

信息

Mistral 官方 API 文档：https://docs.mistral.org.cn/api/#operation/createFIMCompletion

非流式

示例用法

import os
import litellm

os.environ['CODESTRAL_API_KEY']

response = await litellm.atext_completion(
    model="text-completion-codestral/codestral-2405",
    prompt="def is_odd(n): \n return n % 2 == 1 \ndef test_is_odd():", 
    suffix="return True",                                              # optional
    temperature=0,                                                     # optional
    top_p=1,                                                           # optional
    max_tokens=10,                                                     # optional
    min_tokens=10,                                                     # optional
    seed=10,                                                           # optional
    stop=["return"],                                                   # optional
)

预期响应

{
  "id": "b41e0df599f94bc1a46ea9fcdbc2aabe",
  "object": "text_completion",
  "created": 1589478378,
  "model": "codestral-latest",
  "choices": [
    {
      "text": "\n assert is_odd(1)\n assert",
      "index": 0,
      "logprobs": null,
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 5,
    "completion_tokens": 7,
    "total_tokens": 12
  }
}

流式

示例用法 - 流式

import os
import litellm

os.environ['CODESTRAL_API_KEY']

response = await litellm.atext_completion(
    model="text-completion-codestral/codestral-2405",
    prompt="def is_odd(n): \n return n % 2 == 1 \ndef test_is_odd():",
    suffix="return True",    # optional
    temperature=0,           # optional
    top_p=1,                 # optional
    stream=True,                
    seed=10,                 # optional
    stop=["return"],         # optional
)

async for chunk in response:
    print(chunk)

预期响应

{
  "id": "726025d3e2d645d09d475bb0d29e3640",
  "object": "text_completion",
  "created": 1718659669,
  "choices": [
    {
      "text": "This",
      "index": 0,
      "logprobs": null,
      "finish_reason": null
    }
  ],
  "model": "codestral-2405", 
}

支持的模型

此处列出的所有模型 https://docs.mistral.org.cn/platform/endpoints 均受支持。我们在此处 https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json 积极维护模型列表、定价、token 窗口等信息。

模型名称	函数调用
Codestral 最新版	completion(model="text-completion-codestral/codestral-latest", messages)
Codestral 2405	completion(model="text-completion-codestral/codestral-2405", messages)

聊天补全

信息

Mistral 官方 API 文档：https://docs.mistral.org.cn/api/#operation/createChatCompletion

非流式

示例用法

import os
import litellm

os.environ['CODESTRAL_API_KEY']

response = await litellm.acompletion(
    model="codestral/codestral-latest",
    messages=[
        {
            "role": "user",
            "content": "Hey, how's it going?",
        }
    ],
    temperature=0.0,       # optional
    top_p=1,               # optional
    max_tokens=10,         # optional
    safe_prompt=False,     # optional
    seed=12,               # optional
)

预期响应

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "codestral/codestral-latest",
  "system_fingerprint": None,
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "\n\nHello there, how may I assist you today?",
    },
    "logprobs": null,
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 12,
    "total_tokens": 21
  }
}

流式

示例用法 - 流式

import os
import litellm

os.environ['CODESTRAL_API_KEY']

response = await litellm.acompletion(
    model="codestral/codestral-latest",
    messages=[
        {
            "role": "user",
            "content": "Hey, how's it going?",
        }
    ],
    stream=True,           # optional
    temperature=0.0,       # optional
    top_p=1,               # optional
    max_tokens=10,         # optional
    safe_prompt=False,     # optional
    seed=12,               # optional
)
async for chunk in response:
    print(chunk)

预期响应

{
    "id":"chatcmpl-123",
    "object":"chat.completion.chunk",
    "created":1694268190,
    "model": "codestral/codestral-latest",
    "system_fingerprint": None, 
    "choices":[
        {
            "index":0,
            "delta":{"role":"assistant","content":"gm"},
            "logprobs":null,
        "   finish_reason":null
        }
    ]
}

支持的模型

此处列出的所有模型 https://docs.mistral.org.cn/platform/endpoints 均受支持。我们在此处 https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json 积极维护模型列表、定价、token 窗口等信息。

模型名称	函数调用
Codestral 最新版	completion(model="codestral/codestral-latest", messages)
Codestral 2405	completion(model="codestral/codestral-2405", messages)