超GPT3.5性能,无限长文本,超强RAG三件套,MiniCPM3-4B模型分享

MiniCPM3-4B是由面壁智能与清华大学自然语言处理实验室合作开发的一款高性能端侧AI模型,它是MiniCPM系列的第三代产品,具有4亿参数量。

MiniCPM3-4B模型在性能上超过了Phi-3.5-mini-Instruct和GPT-3.5-Turbo-0125,并且与多款70亿至90亿参数的AI模型相媲美。

MiniCPM3-4B在多项指标上都有显著提升,包括词汇表大小、模型层数和隐藏层节点的增加,使其处理能力更为出色。

MiniCPM3-4B支持32k的上下文窗口设计,理论上可以处理无限的上下文信息,这对于需要处理大量数据和复杂查询的用户来说是一个巨大的优势。

MiniCPM3-4B还支持更高效的代码执行和函数调用,使开发者能够更快速地实现复杂的任务。

此外,面壁智能还发布了针对RAG场景的微调版MiniCPM3-RAG-LoRA模型,以及RAG套件MiniCPM-Embedding模型和MiniCPM-Reranker模型。

github项目地址:https://github.com/OpenBMB/MiniCPM。

一、环境安装

1、python环境

建议安装python版本在3.10以上。

2、pip库安装

pip install torch==2.3.0+cu118 torchvision==0.18.0+cu118 torchaudio==2.3.0 --extra-index-url https://download.pytorch.org/whl/cu118

pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

pip install datamodel_code_generator -i https://pypi.tuna.tsinghua.edu.cn/simple

3、MiniCPM3-4B模型下载:

git lfs install

git clone https://modelscope.cn/models/OpenBMB/MiniCPM3-4B 4、MiniCPM3-RAG-LoRA模型下载:

git lfs install

git clone https://modelscope.cn/models/OpenBMB/MiniCPM3-RAG-LoRA 5、MiniCPM-Reranker模型下载:

git lfs install

git clone https://modelscope.cn/models/OpenBMB/MiniCPM-Reranker 6、MiniCPM-Embedding模型下载:

git lfs install

git clone https://modelscope.cn/models/OpenBMB/MiniCPM-Embedding

二**、功能测试**

1、运行测试:

(1)python代码调用测试

复制代码
import torch
from modelscope import AutoModelForCausalLM, AutoModel, AutoTokenizer, snapshot_download
from transformers import AutoModelForSequenceClassification
from peft import PeftModel
import torch.nn.functional as F
import numpy as np

def MiniCPM3_4B_inference(message, model_path="OpenBMB/MiniCPM3-4B", device="cuda"):
    tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
    model = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16, device_map=device, trust_remote_code=True)

    messages = [{"role": "user", "content": message}]
    model_inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(device)

    model_outputs = model.generate(
        model_inputs,
        max_new_tokens=1024,
        top_p=0.7,
        temperature=0.7,
        repetition_penalty=1.02
    )

    output_token_ids = [model_outputs[i][len(model_inputs[i]):] for i in range(len(model_inputs))]
    responses = tokenizer.batch_decode(output_token_ids, skip_special_tokens=True)[0]
    return responses

def MiniCPM3_RAG_LoRA_inference(instruction, passages_list, base_model_dir="OpenBMB/MiniCPM3-4B", lora_model_dir="OpenBMB/MiniCPM3-RAG-LoRA"):
    base_model_dir = snapshot_download(base_model_dir)
    lora_model_dir = snapshot_download(lora_model_dir)

    model = AutoModelForCausalLM.from_pretrained(base_model_dir, device_map="auto", torch_dtype=torch.bfloat16).eval()
    tokenizer = AutoTokenizer.from_pretrained(lora_model_dir)
    model = PeftModel.from_pretrained(model, lora_model_dir)

    passages = '\n'.join(passages_list)
    input_text = 'Background:\n' + passages + '\n\n' + instruction

    messages = [
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": input_text},
    ]
    prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
    outputs = model.chat(tokenizer, prompt, temperature=0.8, top_p=0.8)
    return outputs[0]

def MiniCPM_Embedding_inference(queries, passages, model_name="OpenBMB/MiniCPM-Embedding", device="cuda"):
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    model = AutoModel.from_pretrained(model_name, trust_remote_code=True, attn_implementation="flash_attention_2", torch_dtype=torch.float16).to(device)
    model.eval()

    def weighted_mean_pooling(hidden, attention_mask):
        attention_mask_ = attention_mask * attention_mask.cumsum(dim=1)
        s = torch.sum(hidden * attention_mask_.unsqueeze(-1).float(), dim=1)
        d = attention_mask_.sum(dim=1, keepdim=True).float()
        reps = s / d
        return reps

    @torch.no_grad()
    def encode(input_texts):
        batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt', return_attention_mask=True).to(device)
        outputs = model(**batch_dict)
        attention_mask = batch_dict["attention_mask"]
        hidden = outputs.last_hidden_state
        reps = weighted_mean_pooling(hidden, attention_mask)
        embeddings = F.normalize(reps, p=2, dim=1).detach().cpu().numpy()
        return embeddings

    INSTRUCTION = "Query: "
    queries = [INSTRUCTION + query for query in queries]

    embeddings_query = encode(queries)
    embeddings_doc = encode(passages)

    scores = (embeddings_query @ embeddings_doc.T)
    return scores.tolist()

def MiniCPM_Reranker_rerank(queries, passages, model_name='OpenBMB/MiniCPM-Reranker', device="cuda", max_len_q=512, max_len_d=512):
    model_name = snapshot_download(model_name)
    tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
    tokenizer.padding_side = "right"
    model = AutoModelForSequenceClassification.from_pretrained(model_name, trust_remote_code=True, attn_implementation="flash_attention_2", torch_dtype=torch.float16).to(device)
    model.eval()

    def tokenize_our(query, doc):
        input_id_query = tokenizer.encode(query, add_special_tokens=False, max_length=max_len_q, truncation=True)
        input_id_doc = tokenizer.encode(doc, add_special_tokens=False, max_length=max_len_d, truncation=True)
        pad_input = {"input_ids": [tokenizer.bos_token_id] + input_id_query + [tokenizer.eos_token_id] + input_id_doc}
        return tokenizer.pad(
            pad_input,
            padding="max_length",
            max_length=max_len_q + max_len_d + 2,
            return_tensors="pt",
        )

    @torch.no_grad()
    def rerank(input_query, input_docs):
        tokenized_inputs = [tokenize_our(input_query, input_doc).to(device) for input_doc in input_docs]
        input_ids = {
            "input_ids": [tokenized_input["input_ids"] for tokenized_input in tokenized_inputs],
            "attention_mask": [tokenized_input["attention_mask"] for tokenized_input in tokenized_inputs]
        }

        for k in input_ids:
            input_ids[k] = torch.stack(input_ids[k]).to(device)
        outputs = model(**input_ids)
        score = outputs.logits
        return score.float().detach().cpu().numpy()

    INSTRUCTION = "Query: "
    queries = [INSTRUCTION + query for query in queries]
    scores = [rerank(query, docs) for query, docs in zip(queries, passages)]
    return np.array(scores)

def main():
    # Example use cases
    response_4B = MiniCPM3_4B_inference("推荐5个北京的景点。")
    print(f"MiniCPM3-4B Response: {response_4B}")

    instruction = "Q: What is the name of the lead character in the novel 'The Silent Watcher'?\nA:"
    passages_list = [
        "In the novel 'The Silent Watcher,' the lead character is named Alex Carter. Alex is a private detective who uncovers a series of mysterious events in a small town.",
        "Set in a quiet town, 'The Silent Watcher' follows Alex Carter, a former police officer turned private investigator, as he unravels the town's dark secrets.",
        "'The Silent Watcher' revolves around Alex Carter's journey as he confronts his past while solving complex cases in his hometown."
    ]
    response_RAG_LoRA = MiniCPM3_RAG_LoRA_inference(instruction, passages_list)
    print(f"MiniCPM3-RAG-LoRA Response: {response_RAG_LoRA}")

    queries = ["China capital?"]
    passages = ["beijing", "shanghai"]
    scores_embedding = MiniCPM_Embedding_inference(queries, passages)
    print(f"MiniCPM-Embedding Scores: {scores_embedding}")

    rerank_queries = ["China capital?"]
    rerank_passages = [["beijing", "shanghai"]]
    scores_reranker = MiniCPM_Reranker_rerank(rerank_queries, rerank_passages)
    print(f"MiniCPM-Reranker Scores: {scores_reranker}")

if __name__ == "__main__":
    main()

未完......

更多详细的欢迎关注:杰哥新技术

相关推荐
可乐ea26 分钟前
MCP 和 CLI 差 42 倍:一次 GitHub 任务的上下文成本实测
人工智能·github·工具调用·ai智能体·openrouter·mcp协议·上下文优化
kyriewen40 分钟前
编辑弹窗点了取消,列表里的数据怎么还是变了?
前端·javascript·人工智能
泯泷1 小时前
Multica 真正打动我的,不是更多 Agent,而是一种新的团队协作方式
人工智能·llm·agent
电科_银尘1 小时前
【深度学习项目实战·22】灰度图像着色:用小型U-Net学习从灰度到RGB
人工智能·深度学习·学习
阿里云大数据AI技术1 小时前
云栖实录|倍思科技 × 阿里云:让VOC洞察从“生成报告”走向“业务可用”
人工智能·elasticsearch·agent
ZhengEnCi2 小时前
AI 规则越多越费 Token:无关规则让每次回答多耗 46% 的规则体检提示词
aigc·ai编程·claude
AllData公司负责人2 小时前
第九篇:AI大模型数据开发底座系列说明书:AllData集成BISHENG开源项目建设大模型应用开发底座,涵盖可视化编排/智能助手/一键API/模型接入!
人工智能·云原生·数据分析·视觉检测·知识图谱·大模型应用·bisheng
数字新视界2 小时前
数据中心DCIM管理系统助力提升机房管理效率与安全性
大数据·人工智能·物联网·信息与通信·动环监控系统
阿里云大数据AI技术2 小时前
阿里云 EMR Serverless StarRocks(Stella)× Paimon:让多模态数据湖从“存得下”走向“找得到、用得上”
人工智能
class心平气和2 小时前
AI 赋能测试流程:从基础概念到测试全流程实战
人工智能