大模型学习与实践笔记(十四)

使用 OpenCompass 评测 InternLM2-Chat-7B 模型使用 LMDeploy 0.2.0 部署后在 C-Eval 数据集上的性能

步骤1:下载internLM2-Chat-7B 模型,并进行挂载

以下命令将internlm2-7b模型挂载到当前目录下:

bash 复制代码
ln -s /share/model_repos/internlm2-7b/ ./

步骤2:编译安装LMdeploy0.2.0

复制代码
pip install 'lmdeploy[all]==v0.2.0'

步骤3:使用LMdeploy 将模型internLM2-Chat-7B 进行转换

bash 复制代码
lmdeploy convert internlm2-chat-7b /root/model/Shanghai_AI_Laboratory/internlm2-chat-7b

运行日志:

bash 复制代码
(internlm-demo) root@intern-studio:~/deploy# lmdeploy convert internlm2-chat-7b /root/model/Shanghai_AI_Laboratory/internlm2-chat-7b
create workspace in directory workspace
copy triton model templates from "/root/.conda/envs/internlm-demo/lib/python3.10/site-packages/lmdeploy/serve/turbomind/triton_models" to "workspace/triton_models"
copy service_docker_up.sh from "/root/.conda/envs/internlm-demo/lib/python3.10/site-packages/lmdeploy/serve/turbomind/service_docker_up.sh" to "workspace"
model_name             internlm2-chat-7b
model_format           None
inferred_model_format  internlm2
model_path             /root/model/Shanghai_AI_Laboratory/internlm2-chat-7b
tokenizer_path         /root/model/Shanghai_AI_Laboratory/internlm2-chat-7b/tokenizer.model
output_format          fp16
01/29 17:36:32 - lmdeploy - WARNING - Can not find tokenizer.json. It may take long time to initialize the tokenizer.
*** splitting layers.0.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.0.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.0.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.0.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.0.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.1.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.1.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.1.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.1.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.1.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.2.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.2.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.2.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.2.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.2.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.3.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.3.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.3.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.3.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.3.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.4.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.4.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.4.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.4.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.4.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.5.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.5.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.5.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.5.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.5.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.6.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.6.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.6.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.6.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.6.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.7.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.7.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.7.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.7.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.7.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.8.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.8.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.8.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.8.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.8.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.9.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                           
*** splitting layers.9.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                               
*** splitting layers.9.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.9.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.9.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                           
*** splitting layers.10.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.10.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.10.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.10.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.10.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.11.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.11.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.11.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.11.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.11.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.12.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.12.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.12.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.12.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.12.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.13.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.13.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.13.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.13.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.13.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.14.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.14.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.14.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.14.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.14.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.15.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.15.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.15.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.15.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.15.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.16.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.16.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.16.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.16.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.16.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.17.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.17.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.17.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.17.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.17.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.18.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.18.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.18.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.18.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.18.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.19.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.19.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.19.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.19.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.19.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.20.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.20.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.20.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.20.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.20.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.21.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.21.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.21.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.21.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.21.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.22.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.22.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.22.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.22.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.22.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.23.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.23.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.23.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.23.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.23.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.24.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.24.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.24.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.24.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.24.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.25.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.25.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.25.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.25.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.25.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.26.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.26.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.26.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.26.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.26.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.27.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.27.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.27.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.27.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.27.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.28.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.28.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.28.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.28.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.28.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.29.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.29.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.29.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.29.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.29.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.30.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.30.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.30.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.30.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.30.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
*** splitting layers.31.attention.w_qkv.weight, shape=torch.Size([4096, 6144]), split_dim=-1, tp=1                                                                                                                                          
*** splitting layers.31.attention.wo.weight, shape=torch.Size([4096, 4096]), split_dim=0, tp=1                                                                                                                                              
*** splitting layers.31.feed_forward.w1.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.31.feed_forward.w3.weight, shape=torch.Size([4096, 14336]), split_dim=-1, tp=1                                                                                                                                         
*** splitting layers.31.feed_forward.w2.weight, shape=torch.Size([14336, 4096]), split_dim=0, tp=1                                                                                                                                          
Convert to turbomind format: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 32/32 [00:27<00:00,  1.18it/s

步骤4:模型结果测评

首先新建config文件,其中参数"/root/deploy/workspace/"表示LMdeploy转换后的模型地址。

bash 复制代码
from mmengine.config import read_base
from opencompass.models.turbomind import TurboMindModel

with read_base():
 # choose a list of datasets   
 from .datasets.ceval.ceval_gen_5f30c7 import ceval_datasets 
 # and output the results in a choosen format
 from .summarizers.medium import summarizer

datasets = sum((v for k, v in locals().items() if k.endswith('_datasets')), [])

internlm_meta_template = dict(round=[
 dict(role='HUMAN', begin='<|User|>:', end='\n'),
 dict(role='BOT', begin='<|Bot|>:', end='<eoa>\n', generate=True),
],
 eos_token_id=103028)

# config for internlm-chat-7b
internlm2_chat_7b = dict(
 type=TurboMindModel,
 abbr='internlm2-chat-7b-turbomind',
 path='/root/deploy/workspace/',
 engine_config=dict(session_len=512,
 max_batch_size=2,
 rope_scaling_factor=1.0),
 gen_config=dict(top_k=1,
 top_p=0.8,
 temperature=1.0,
 max_new_tokens=100),
 max_out_len=100,
 max_seq_len=512,
 batch_size=2,
 concurrency=1,
 meta_template=internlm_meta_template,
 run_cfg=dict(num_gpus=1, num_procs=1),
)
models = [internlm2_chat_7b]

在opencompass 目录下运行:

bash 复制代码
python run.py configs/eval_turbomind.py

同样可以添加--debug ,输出日志信息。

bash 复制代码
python run.py configs/eval_turbomind.py --debug

过程日志如下:

相关推荐
ZWZhangYu1 小时前
LangChain 构建向量数据库和检索器
数据库·langchain·easyui
千宇宙航1 小时前
闲庭信步使用SV搭建图像测试平台:第三十一课——基于神经网络的手写数字识别
图像处理·人工智能·深度学习·神经网络·计算机视觉·fpga开发
onceco1 小时前
领域LLM九讲——第5讲 为什么选择OpenManus而不是QwenAgent(附LLM免费api邀请码)
人工智能·python·深度学习·语言模型·自然语言处理·自动化
天水幼麟2 小时前
动手学深度学习-学习笔记(总)
笔记·深度学习·学习
jndingxin4 小时前
OpenCV CUDA模块设备层-----高效地计算两个 uint 类型值的带权重平均值
人工智能·opencv·计算机视觉
天水幼麟4 小时前
动手学深度学习-学习笔记【二】(基础知识)
笔记·深度学习·学习
Sweet锦4 小时前
零基础保姆级本地化部署文心大模型4.5开源系列
人工智能·语言模型·文心一言
hie988945 小时前
MATLAB锂离子电池伪二维(P2D)模型实现
人工智能·算法·matlab
晨同学03275 小时前
opencv的颜色通道问题 & rgb & bgr
人工智能·opencv·计算机视觉
蓝婷儿6 小时前
Python 机器学习核心入门与实战进阶 Day 3 - 决策树 & 随机森林模型实战
人工智能·python·机器学习