CodeLlama - 代码填充

本教程展示了如何调用 CodeLlama（托管在 Huggingface PRO 推理端点上）进行代码填充。

这是一项特定于代码模型的专门任务。该模型经过训练，可以生成与现有前缀和后缀最匹配的代码（包括注释）。

这项任务适用于 CodeLlama 模型的基础版本和指令版本，包括 7B 和 13B。它不适用于任何 34B 模型或 Python 版本。

用法

import os
from litellm import longer_context_model_fallback_dict, ContextWindowExceededError, completion

os.environ["HUGGINGFACE_API_KEY"] = "your-hf-token" # https://hugging-face.cn/docs/hub/security-tokens

## CREATE THE PROMPT
prompt_prefix = 'def remove_non_ascii(s: str) -> str:\n    """ '
prompt_suffix = "\n    return result"

### set <pre> <suf> to indicate the string before and after the part you want codellama to fill 
prompt = f"<PRE> {prompt_prefix} <SUF>{prompt_suffix} <MID>"

messages = [{"content": prompt, "role": "user"}]
model = "huggingface/codellama/CodeLlama-34b-Instruct-hf" # specify huggingface as the provider 'huggingface/'
response = completion(model=model, messages=messages, max_tokens=500)

输出

def remove_non_ascii(s: str) -> str:
    """ Remove non-ASCII characters from a string.

    Args:
        s (str): The string to remove non-ASCII characters from.

    Returns:
        str: The string with non-ASCII characters removed.
    """
    result = ""
    for c in s:
        if ord(c) < 128:
            result += c
    return result