超GPT3.5性能,无限长文本,超强RAG三件套,MiniCPM3-4B模型分享

MiniCPM3-4B是由面壁智能与清华大学自然语言处理实验室合作开发的一款高性能端侧AI模型,它是MiniCPM系列的第三代产品,具有4亿参数量。

MiniCPM3-4B模型在性能上超过了Phi-3.5-mini-Instruct和GPT-3.5-Turbo-0125,并且与多款70亿至90亿参数的AI模型相媲美。

MiniCPM3-4B在多项指标上都有显著提升,包括词汇表大小、模型层数和隐藏层节点的增加,使其处理能力更为出色。

MiniCPM3-4B支持32k的上下文窗口设计,理论上可以处理无限的上下文信息,这对于需要处理大量数据和复杂查询的用户来说是一个巨大的优势。

MiniCPM3-4B还支持更高效的代码执行和函数调用,使开发者能够更快速地实现复杂的任务。

此外,面壁智能还发布了针对RAG场景的微调版MiniCPM3-RAG-LoRA模型,以及RAG套件MiniCPM-Embedding模型和MiniCPM-Reranker模型。

github项目地址:https://github.com/OpenBMB/MiniCPM。

一、环境安装

1、python环境

建议安装python版本在3.10以上。

2、pip库安装

pip install torch==2.3.0+cu118 torchvision==0.18.0+cu118 torchaudio==2.3.0 --extra-index-url https://download.pytorch.org/whl/cu118

pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

pip install datamodel_code_generator -i https://pypi.tuna.tsinghua.edu.cn/simple

3、MiniCPM3-4B模型下载

git lfs install

git clone https://modelscope.cn/models/OpenBMB/MiniCPM3-4B 4、MiniCPM3-RAG-LoRA模型下载

git lfs install

git clone https://modelscope.cn/models/OpenBMB/MiniCPM3-RAG-LoRA 5、MiniCPM-Reranker模型下载

git lfs install

git clone https://modelscope.cn/models/OpenBMB/MiniCPM-Reranker 6、MiniCPM-Embedding模型下载

git lfs install

git clone https://modelscope.cn/models/OpenBMB/MiniCPM-Embedding

二**、功能测试**

1、运行测试

(1)python代码调用测试

复制代码
import torch
from modelscope import AutoModelForCausalLM, AutoModel, AutoTokenizer, snapshot_download
from transformers import AutoModelForSequenceClassification
from peft import PeftModel
import torch.nn.functional as F
import numpy as np

def MiniCPM3_4B_inference(message, model_path="OpenBMB/MiniCPM3-4B", device="cuda"):
    tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
    model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16, device_map=device, trust_remote_code=True)

    messages = [{"role": "user", "content": message}]
    model_inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(device)

    model_outputs = model.generate(
        model_inputs,
        max_new_tokens=1024,
        top_p=0.7,
        temperature=0.7,
        repetition_penalty=1.02
    )

    output_token_ids = [model_outputs[i][len(model_inputs[i]):] for i in range(len(model_inputs))]
    responses = tokenizer.batch_decode(output_token_ids, skip_special_tokens=True)[0]
    return responses

def MiniCPM3_RAG_LoRA_inference(instruction, passages_list, base_model_dir="OpenBMB/MiniCPM3-4B", lora_model_dir="OpenBMB/MiniCPM3-RAG-LoRA"):
    base_model_dir = snapshot_download(base_model_dir)
    lora_model_dir = snapshot_download(lora_model_dir)

    model = AutoModelForCausalLM.from_pretrained(base_model_dir, device_map="auto", torch_dtype=torch.bfloat16).eval()
    tokenizer = AutoTokenizer.from_pretrained(lora_model_dir)
    model = PeftModel.from_pretrained(model, lora_model_dir)

    passages = '\n'.join(passages_list)
    input_text = 'Background:\n' + passages + '\n\n' + instruction

    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": input_text},
    ]
    prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
    outputs = model.chat(tokenizer, prompt, temperature=0.8, top_p=0.8)
    return outputs[0]

def MiniCPM_Embedding_inference(queries, passages, model_name="OpenBMB/MiniCPM-Embedding", device="cuda"):
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModel.from_pretrained(model_name, trust_remote_code=True, attn_implementation="flash_attention_2", torch_dtype=torch.float16).to(device)
    model.eval()

    def weighted_mean_pooling(hidden, attention_mask):
        attention_mask_ = attention_mask * attention_mask.cumsum(dim=1)
        s = torch.sum(hidden * attention_mask_.unsqueeze(-1).float(), dim=1)
        d = attention_mask_.sum(dim=1, keepdim=True).float()
        reps = s / d
        return reps

    @torch.no_grad()
    def encode(input_texts):
        batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt', return_attention_mask=True).to(device)
        outputs = model(**batch_dict)
        attention_mask = batch_dict["attention_mask"]
        hidden = outputs.last_hidden_state
        reps = weighted_mean_pooling(hidden, attention_mask)
        embeddings = F.normalize(reps, p=2, dim=1).detach().cpu().numpy()
        return embeddings

    INSTRUCTION = "Query: "
    queries = [INSTRUCTION + query for query in queries]

    embeddings_query = encode(queries)
    embeddings_doc = encode(passages)

    scores = (embeddings_query @ embeddings_doc.T)
    return scores.tolist()

def MiniCPM_Reranker_rerank(queries, passages, model_name='OpenBMB/MiniCPM-Reranker', device="cuda", max_len_q=512, max_len_d=512):
    model_name = snapshot_download(model_name)
    tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
    tokenizer.padding_side = "right"
    model = AutoModelForSequenceClassification.from_pretrained(model_name, trust_remote_code=True, attn_implementation="flash_attention_2", torch_dtype=torch.float16).to(device)
    model.eval()

    def tokenize_our(query, doc):
        input_id_query = tokenizer.encode(query, add_special_tokens=False, max_length=max_len_q, truncation=True)
        input_id_doc = tokenizer.encode(doc, add_special_tokens=False, max_length=max_len_d, truncation=True)
        pad_input = {"input_ids": [tokenizer.bos_token_id] + input_id_query + [tokenizer.eos_token_id] + input_id_doc}
        return tokenizer.pad(
            pad_input,
            padding="max_length",
            max_length=max_len_q + max_len_d + 2,
            return_tensors="pt",
        )

    @torch.no_grad()
    def rerank(input_query, input_docs):
        tokenized_inputs = [tokenize_our(input_query, input_doc).to(device) for input_doc in input_docs]
        input_ids = {
            "input_ids": [tokenized_input["input_ids"] for tokenized_input in tokenized_inputs],
            "attention_mask": [tokenized_input["attention_mask"] for tokenized_input in tokenized_inputs]
        }

        for k in input_ids:
            input_ids[k] = torch.stack(input_ids[k]).to(device)
        outputs = model(**input_ids)
        score = outputs.logits
        return score.float().detach().cpu().numpy()

    INSTRUCTION = "Query: "
    queries = [INSTRUCTION + query for query in queries]
    scores = [rerank(query, docs) for query, docs in zip(queries, passages)]
    return np.array(scores)

def main():
    # Example use cases
    response_4B = MiniCPM3_4B_inference("推荐5个北京的景点。")
    print(f"MiniCPM3-4B Response: {response_4B}")

    instruction = "Q: What is the name of the lead character in the novel 'The Silent Watcher'?\nA:"
    passages_list = [
        "In the novel 'The Silent Watcher,' the lead character is named Alex Carter. Alex is a private detective who uncovers a series of mysterious events in a small town.",
        "Set in a quiet town, 'The Silent Watcher' follows Alex Carter, a former police officer turned private investigator, as he unravels the town's dark secrets.",
        "'The Silent Watcher' revolves around Alex Carter's journey as he confronts his past while solving complex cases in his hometown."
    ]
    response_RAG_LoRA = MiniCPM3_RAG_LoRA_inference(instruction, passages_list)
    print(f"MiniCPM3-RAG-LoRA Response: {response_RAG_LoRA}")

    queries = ["China capital?"]
    passages = ["beijing", "shanghai"]
    scores_embedding = MiniCPM_Embedding_inference(queries, passages)
    print(f"MiniCPM-Embedding Scores: {scores_embedding}")

    rerank_queries = ["China capital?"]
    rerank_passages = [["beijing", "shanghai"]]
    scores_reranker = MiniCPM_Reranker_rerank(rerank_queries, rerank_passages)
    print(f"MiniCPM-Reranker Scores: {scores_reranker}")

if __name__ == "__main__":
    main()

未完......

更多详细的欢迎关注:杰哥新技术

相关推荐
Jay Kay35 分钟前
TensorFlow源码深度阅读指南
人工智能·python·tensorflow
FF-Studio38 分钟前
【硬核数学 · LLM篇】3.1 Transformer之心:自注意力机制的线性代数解构《从零构建机器学习、深度学习到LLM的数学认知》
人工智能·pytorch·深度学习·线性代数·机器学习·数学建模·transformer
会的全对٩(ˊᗜˋ*)و1 小时前
【数据挖掘】数据挖掘综合案例—银行精准营销
人工智能·经验分享·python·数据挖掘
云渚钓月梦未杳1 小时前
深度学习03 人工神经网络ANN
人工智能·深度学习
在美的苦命程序员1 小时前
中文语境下的视频生成革命:百度 MuseSteamer 的“产品级落地”启示录
人工智能·百度
kngines1 小时前
【字节跳动】数据挖掘面试题0007:Kmeans原理,何时停止迭代
人工智能·数据挖掘·kmeans
Kali_071 小时前
使用 Mathematical_Expression 从零开始实现数学题目的作答小游戏【可复制代码】
java·人工智能·免费
贾全1 小时前
第十章:HIL-SERL 真实机器人训练实战
人工智能·深度学习·算法·机器学习·机器人
每日摸鱼大王1 小时前
互联网摸鱼日报(2025-07-01)
人工智能
GIS小天2 小时前
AI+预测3D新模型百十个定位预测+胆码预测+去和尾2025年7月4日第128弹
人工智能·算法·机器学习·彩票