开启想象翅膀：轻松实现文本生成模型的创作应用，支持LLaMA、ChatGLM、UDA、GPT2、Seq2Seq、BART、T5、SongNet等模型，开箱即用
1.介绍
TextGen实现了多种文本生成模型，包括：LLaMA、ChatGLM、UDA、GPT2、Seq2Seq、BART、T5、SongNet等模型，开箱即用。
1.1 最新更新
2023/06/15\] v1.0.0版本: 新增ChatGLM/LLaMA/Bloom模型的多轮对话微调训练，并发布医疗问诊LoRA模型[shibing624/ziya-llama-13b-medical-lora](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fziya-llama-13b-medical-lora "https://huggingface.co/shibing624/ziya-llama-13b-medical-lora")。详见[Release-v1.0.0](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2FMedicalGPT%2Freleases%2Ftag%2F1.0.0 "https://github.com/shibing624/MedicalGPT/releases/tag/1.0.0")
\[2023/06/02\] v0.2.7版本: 新增ChatGLM/LLaMA/Bloom模型的SFT微调训练，并发布适用于通用对话和中文纠错的LoRA模型。详见[Release-v0.2.7](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2FMedicalGPT%2Freleases%2Ftag%2F0.2.7 "https://github.com/shibing624/MedicalGPT/releases/tag/0.2.7")
## 1.2 相关介绍
* [ChatGLM](https://link.juejin.cn?target=textgen%2Fchatglm "textgen/chatglm")：本项目基于PyTorch实现了ChatGLM-6B模型LoRA微调训练和预测，可以用于句子纠错、对话等文本生成任务
* [LLaMA](https://link.juejin.cn?target=textgen%2Fllama "textgen/llama")：本项目基于PyTorch实现了LLaMA模型LoRA微调训练和预测，可以用于对话生成任务和领域微调训练
* [BLOOM](https://link.juejin.cn?target=textgen%2Fbloom "textgen/bloom")：本项目基于PyTorch实现了BLOOM模型LoRA微调训练和预测，可以用于对话生成任务和领域微调训练
* [UDA/EDA](https://link.juejin.cn?target=textgen%2Faugment%2Fword_level_augment.py "textgen/augment/word_level_augment.py")：本项目实现了UDA(非核心词替换)、EDA和Back Translation(回译)算法，基于TF-IDF将句子中部分不重要词替换为同义词，随机词插入、删除、替换等方法，产生新的文本，实现了文本扩增
* [Seq2Seq](https://link.juejin.cn?target=textgen%2Fseq2seq "textgen/seq2seq")：本项目基于PyTorch实现了Seq2Seq、ConvSeq2Seq、BART模型的训练和预测，可以用于文本翻译、对话生成、摘要生成等文本生成任务
* [T5](https://link.juejin.cn?target=textgen%2Ft5 "textgen/t5")：本项目基于PyTorch实现了T5和CopyT5模型训练和预测，可以用于文本翻译、对话生成、对联生成、文案撰写等文本生成任务
* [GPT2](https://link.juejin.cn?target=textgen%2Flanguage_modeling "textgen/language_modeling")：本项目基于PyTorch实现了GTP2模型训练和预测，可以用于文章生成、对联生成等文本生成任务
* [SongNet](https://link.juejin.cn?target=textgen%2Flanguage_modeling%2Fsongnet_model.py "textgen/language_modeling/songnet_model.py")：本项目基于PyTorch实现了SongNet模型训练和预测，可以用于规范格式的诗词、歌词等文本生成任务
* [TGLS](https://link.juejin.cn?target=textgen%2Funsup_generation "textgen/unsup_generation")：本项目实现了[TGLS](https://link.juejin.cn?target=https%3A%2F%2Fwww.jiqizhixin.com%2Farticles%2F2020-08-11-5 "https://www.jiqizhixin.com/articles/2020-08-11-5")无监督相似文本生成模型，是一种"先搜索后学习"的文本生成方法，通过反复迭代学习候选集，最终模型能生成类似候选集的高质量相似文本
### 1.2.1 支持模型
release基于`textgen`训练的中文模型，模型已经release到HuggingFace models，指定模型名称`textgen`会自动下载模型，可直接使用。
| Model | Arch | Introduction | Train Script | Predict Script |
|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [shibing624/t5-chinese-couplet](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Ft5-chinese-couplet "https://huggingface.co/shibing624/t5-chinese-couplet") | T5 | fine-tuned中文对联后的模型 | [对联生成模型调研](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fdocs%2F%25E5%25AF%25B9%25E8%2581%2594%25E7%2594%259F%25E6%2588%2590%25E6%25A8%25A1%25E5%259E%258B%25E5%25AF%25B9%25E6%25AF%2594.md "https://github.com/shibing624/textgen/blob/main/docs/%E5%AF%B9%E8%81%94%E7%94%9F%E6%88%90%E6%A8%A1%E5%9E%8B%E5%AF%B9%E6%AF%94.md") | [predict script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Ft5%2Ft5_couplet_demo.py "https://github.com/shibing624/textgen/blob/main/examples/t5/t5_couplet_demo.py") |
| [shibing624/songnet-base-chinese-songci](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fsongnet-base-chinese-songci "https://huggingface.co/shibing624/songnet-base-chinese-songci") | SongNet | fine-tuned宋词后的模型 | [training script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fsongnet%2Ftraining_zh_songnet_demo.py "https://github.com/shibing624/textgen/blob/main/examples/songnet/training_zh_songnet_demo.py") | [predict script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fsongnet%2Fsongnet_songci_demo.py "https://github.com/shibing624/textgen/blob/main/examples/songnet/songnet_songci_demo.py") |
| [shibing624/songnet-base-chinese-couplet](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fsongnet-base-chinese-couplet "https://huggingface.co/shibing624/songnet-base-chinese-couplet") | SongNet | fine-tuned对联后的模型 | [training script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fsongnet%2Ftraining_zh_songnet_demo.py "https://github.com/shibing624/textgen/blob/main/examples/songnet/training_zh_songnet_demo.py") | [predict script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fsongnet%2Fsongnet_couplet_demo.py "https://github.com/shibing624/textgen/blob/main/examples/songnet/songnet_couplet_demo.py") |
| [shibing624/chatglm-6b-csc-zh-lora](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fchatglm-6b-csc-zh-lora "https://huggingface.co/shibing624/chatglm-6b-csc-zh-lora") | ChatGLM-6B | 在27万中文拼写纠错数据[shibing624/CSC](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fdatasets%2Fshibing624%2FCSC "https://huggingface.co/datasets/shibing624/CSC")上微调了一版ChatGLM-6B，纠错效果有提升，发布微调后的LoRA权重 | [training script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fchatglm%2Ftraining_chatglm_csc_demo.py "https://github.com/shibing624/textgen/blob/main/examples/chatglm/training_chatglm_csc_demo.py") | [predict script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fchatglm%2Fcsc_demo.py "https://github.com/shibing624/textgen/blob/main/examples/chatglm/csc_demo.py") |
| [shibing624/chatglm-6b-belle-zh-lora](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fchatglm-6b-belle-zh-lora "https://huggingface.co/shibing624/chatglm-6b-belle-zh-lora") | ChatGLM-6B | 在100万条中文ChatGPT指令Belle数据集[BelleGroup/train_1M_CN](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fdatasets%2FBelleGroup%2Ftrain_1M_CN "https://huggingface.co/datasets/BelleGroup/train_1M_CN")上微调了一版ChatGLM-6B，问答效果有提升，发布微调后的LoRA权重 | [training script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fchatglm%2Ftraining_chatglm_hfdataset_demo.py "https://github.com/shibing624/textgen/blob/main/examples/chatglm/training_chatglm_hfdataset_demo.py") | [predict script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fchatglm%2Ftraining_chatglm_hfdataset_demo.py "https://github.com/shibing624/textgen/blob/main/examples/chatglm/training_chatglm_hfdataset_demo.py") |
| [shibing624/llama-13b-belle-zh-lora](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fllama-13b-belle-zh-lora "https://huggingface.co/shibing624/llama-13b-belle-zh-lora") | LLaMA-13B | 在100万条中文ChatGPT指令Belle数据集[BelleGroup/train_1M_CN](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fdatasets%2FBelleGroup%2Ftrain_1M_CN "https://huggingface.co/datasets/BelleGroup/train_1M_CN")上微调了一版Llama-13B，问答效果有提升，发布微调后的LoRA权重 | [training script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fllama%2Ftraining_llama_hfdataset_demo.py "https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_hfdataset_demo.py") | [predict script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fllama%2Ftraining_llama_hfdataset_demo.py "https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_hfdataset_demo.py") |
| [shibing624/chinese-alpaca-plus-7b-hf](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fchinese-alpaca-plus-7b-hf "https://huggingface.co/shibing624/chinese-alpaca-plus-7b-hf") | LLaMA-7B | [中文LLaMA-Plus, Alpaca-Plus 7B版本](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fymcui%2FChinese-LLaMA-Alpaca%2Freleases%2Ftag%2Fv3.0 "https://github.com/ymcui/Chinese-LLaMA-Alpaca/releases/tag/v3.0")，在LLaMA-7B上扩充了中文词表并继续预训练120G文本（通用领域），在4M指令数据集上微调后得到的中文Alpaca-plus模型 | [training script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fllama%2Ftraining_llama_demo.py "https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_demo.py") | [predict script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fllama%2Ftraining_llama_demo.py "https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_demo.py") |
| [shibing624/chinese-alpaca-plus-13b-hf](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fchinese-alpaca-plus-13b-hf "https://huggingface.co/shibing624/chinese-alpaca-plus-13b-hf") | LLaMA-13B | [中文LLaMA-Plus, Alpaca-Plus 13B版本](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fymcui%2FChinese-LLaMA-Alpaca%2Freleases%2Ftag%2Fv3.1 "https://github.com/ymcui/Chinese-LLaMA-Alpaca/releases/tag/v3.1")，在LLaMA-13B上扩充了中文词表并继续预训练120G文本（通用领域），在4.3M指令数据集上微调后得到的中文Alpaca-plus模型 | [training script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fllama%2Ftraining_llama_demo.py "https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_demo.py") | [predict script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fllama%2Ftraining_llama_demo.py "https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_demo.py") |
| [shibing624/ziya-llama-13b-medical-lora](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fziya-llama-13b-medical-lora "https://huggingface.co/shibing624/ziya-llama-13b-medical-lora") | LLaMA-13B | 在240万条中英文医疗数据集[shibing624/medical](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fdatasets%2Fshibing624%2Fmedical "https://huggingface.co/datasets/shibing624/medical")上微调了一版Ziya-LLaMA-13B模型，医疗问答效果有提升，发布微调后的LoRA权重 | [training script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fllama%2Ftraining_llama_demo.py "https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_demo.py") | [predict script](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fllama%2Ftraining_llama_demo.py "https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_demo.py") |
### 1.2.2 模型评价得分
| Model | Arch | Introduction | Score |
|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------|
| [LLaMA-7B-Chinese-Alpaca](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fziqingyang%2Fchinese-alpaca-lora-7b "https://huggingface.co/ziqingyang/chinese-alpaca-lora-7b") | LLaMA-7B | 复用[ymcui/Chinese-LLaMA-Alpaca](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fymcui%2FChinese-LLaMA-Alpaca%2Fblob%2Fmain%2Fexamples%2FREADME.md "https://github.com/ymcui/Chinese-LLaMA-Alpaca/blob/main/examples/README.md")的评估case和得分 | 4.92 |
| [LLaMA-13B-Chinese-Alpaca](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fziqingyang%2Fchinese-alpaca-lora-13b "https://huggingface.co/ziqingyang/chinese-alpaca-lora-13b") | LLaMA-13B | 复用[ymcui/Chinese-LLaMA-Alpaca](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fymcui%2FChinese-LLaMA-Alpaca%2Fblob%2Fmain%2Fexamples%2FREADME.md "https://github.com/ymcui/Chinese-LLaMA-Alpaca/blob/main/examples/README.md")的评估case和得分 | 7.05 |
| [ChatGLM-6B](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2FTHUDM%2Fchatglm-6b "https://huggingface.co/THUDM/chatglm-6b") | ChatGLM-6B | 基于原生`THUDM/chatglm-6b`评估测试集得分 | 7.16 |
| [ChatGLM-6B-v1.1](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2FTHUDM%2Fchatglm-6b "https://huggingface.co/THUDM/chatglm-6b") | ChatGLM-6B | 基于原生`THUDM/chatglm-6b`v1.1英文优化版模型评估测试集得分 | **7.18** |
| [shibing624/chatglm-6b-belle-zh-lora](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fchatglm-6b-belle-zh-lora "https://huggingface.co/shibing624/chatglm-6b-belle-zh-lora") | ChatGLM-6B | 基于`THUDM/chatglm-6b`加载`shibing624/chatglm-6b-belle-zh-lora`LoRA模型后评估测试集得分 | 7.03 |
| [facat/alpaca-lora-cn-13b](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Ffacat%2Falpaca-lora-cn-13b "https://huggingface.co/facat/alpaca-lora-cn-13b") | LLaMA-13B | 基于`decapoda-research/llama-13b-hf`加载`facat/alpaca-lora-cn-13b`LoRA模型后评估测试集并标注得分 | 4.13 |
| [Chinese-Vicuna/Chinese-Vicuna-lora-13b-belle-and-guanaco](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2FChinese-Vicuna%2FChinese-Vicuna-lora-13b-belle-and-guanaco "https://huggingface.co/Chinese-Vicuna/Chinese-Vicuna-lora-13b-belle-and-guanaco") | LLaMA-13B | 基于`decapoda-research/llama-13b-hf`加载`Chinese-Vicuna/Chinese-Vicuna-lora-13b-belle-and-guanaco`LoRA模型后评估测试集并标注得分 | 3.98 |
| [shibing624/chinese-alpaca-plus-7b-hf](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fchinese-alpaca-plus-7b-hf "https://huggingface.co/shibing624/chinese-alpaca-plus-7b-hf") | LLaMA-7B | 使用[ymcui/Chinese-LLaMA-Alpaca 合并模型方法](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fymcui%2FChinese-LLaMA-Alpaca%2Fwiki%2F%25E6%2589%258B%25E5%258A%25A8%25E6%25A8%25A1%25E5%259E%258B%25E5%2590%2588%25E5%25B9%25B6%25E4%25B8%258E%25E8%25BD%25AC%25E6%258D%25A2%23%25E5%25A4%259Alora%25E6%259D%2583%25E9%2587%258D%25E5%2590%2588%25E5%25B9%25B6%25E9%2580%2582%25E7%2594%25A8%25E4%25BA%258Echinese-alpaca-plus "https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki/%E6%89%8B%E5%8A%A8%E6%A8%A1%E5%9E%8B%E5%90%88%E5%B9%B6%E4%B8%8E%E8%BD%AC%E6%8D%A2#%E5%A4%9Alora%E6%9D%83%E9%87%8D%E5%90%88%E5%B9%B6%E9%80%82%E7%94%A8%E4%BA%8Echinese-alpaca-plus")合并HF权重后，评估测试集并标注得分 | 6.93 |
| [shibing624/chinese-alpaca-plus-13b-hf](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fshibing624%2Fchinese-alpaca-plus-13b-hf "https://huggingface.co/shibing624/chinese-alpaca-plus-13b-hf") | LLaMA-13B | 使用[ymcui/Chinese-LLaMA-Alpaca 合并模型方法](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fymcui%2FChinese-LLaMA-Alpaca%2Fwiki%2F%25E6%2589%258B%25E5%258A%25A8%25E6%25A8%25A1%25E5%259E%258B%25E5%2590%2588%25E5%25B9%25B6%25E4%25B8%258E%25E8%25BD%25AC%25E6%258D%25A2%23%25E5%25A4%259Alora%25E6%259D%2583%25E9%2587%258D%25E5%2590%2588%25E5%25B9%25B6%25E9%2580%2582%25E7%2594%25A8%25E4%25BA%258Echinese-alpaca-plus "https://github.com/ymcui/Chinese-LLaMA-Alpaca/wiki/%E6%89%8B%E5%8A%A8%E6%A8%A1%E5%9E%8B%E5%90%88%E5%B9%B6%E4%B8%8E%E8%BD%AC%E6%8D%A2#%E5%A4%9Alora%E6%9D%83%E9%87%8D%E5%90%88%E5%B9%B6%E9%80%82%E7%94%A8%E4%BA%8Echinese-alpaca-plus")合并HF权重后，评估测试集并标注得分 | 7.07 |
| [TheBloke/vicuna-13B-1.1-HF](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2FTheBloke%2Fvicuna-13B-1.1-HF "https://huggingface.co/TheBloke/vicuna-13B-1.1-HF") | LLaMA-13B | 使用原生vicuna-13B-1.1合并后的模型，评估测试集并标注得分 | 5.13 |
| [IDEA-CCNL/Ziya-LLaMA-13B-v1](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2FIDEA-CCNL%2FZiya-LLaMA-13B-v1 "https://huggingface.co/IDEA-CCNL/Ziya-LLaMA-13B-v1") | LLaMA-13B | 使用姜子牙通用大模型V1，评估测试集并标注得分 | 6.63 |
说明：
* 评估case，详见在线文档：中文LLM-benchmark多任务评估集(腾讯文档) [docs.qq.com/sheet/DUUps...](https://link.juejin.cn?target=https%3A%2F%2Fdocs.qq.com%2Fsheet%2FDUUpsREtWbFBsUVJE%3Ftab%3Dr7io7g "https://docs.qq.com/sheet/DUUpsREtWbFBsUVJE?tab=r7io7g") 感谢韩俊明、[杨家铭](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fyangjiam "https://github.com/yangjiam")等同学的标注
* 评估任务类型包括：知识问答，开放式问答，数值计算，诗词、音乐、体育，娱乐，写文章，文本翻译，代码编程，伦理、拒答类，多轮问答，Score 评分是前100条（10分制）的平均分数，人工打分，越高越好
* 评估数量少，任务类型不够全面，评分之间的大小关系有一些参考价值，分数的绝对值没太大参考价值
* 评估脚本：[tests/test_benchmark.py](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Ftests%2Ftest_benchmark.py "https://github.com/shibing624/textgen/blob/main/tests/test_benchmark.py") ，使用fp16预测，无int量化处理，运行脚本可复现评估结果，但生成结果具有随机性，受解码超参、随机种子等因素影响。评测并非绝对严谨，测试结果仅供晾晒参考
* 结论：ChatGLM-6B、LLaMA-13B的中文衍生模型（包括alpaca-plus, vicuna, ziya）的表现属于第一梯队，原版LLaMA-7B的表现整体稍差些
* LLaMA-13B-Chinese-Alpaca是在原版LLaMA上扩充了中文词表，并融入了约20G的通用中文语料后的指令微调模型，表明了LLaMA的底座优秀，具有强大的语言迁移能力
* ChatGLM这种原生的中文预训练模型更理解中文语义，且在中文知识问答、开放式问答得分高
* LLaMA系列模型数值计算、中英翻译、代码编程类得分高
* 经过中文预训练和SFT微调后的Chinese-LLaMA模型在中文诗词、娱乐、伦理类得分相较原版LLaMA有提升
# 2.demo展示
HuggingFace Demo: [huggingface.co/spaces/shib...](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fspaces%2Fshibing624%2Fchinese-couplet-generate "https://huggingface.co/spaces/shibing624/chinese-couplet-generate")
run example: [examples/gradio_demo.py](https://link.juejin.cn?target=examples%2Fgradio_demo.py "examples/gradio_demo.py") to see the demo:
```shell
python examples/gradio_demo.py
```
model trained by [examples/t5/T5_Finetune_Chinese_Couplet.ipynb](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Ft5%2FT5_Finetune_Chinese_Couplet.ipynb "https://github.com/shibing624/textgen/blob/main/examples/t5/T5_Finetune_Chinese_Couplet.ipynb")
## 2.1 安装
```shell
pip install -U textgen
```
or
install develop version:
```shell
pip install torch # conda install pytorch
git clone https://github.com/shibing624/textgen.git
cd textgen
python setup.py install
```
## 2.2 ChatGLM-6B 模型
### 2.2.1 使用 ChatGLM-6B 微调后的模型
example: [examples/chatglm/inference_demo.py](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fchatglm%2Finference_demo.py "https://github.com/shibing624/textgen/blob/main/examples/chatglm/inference_demo.py")
```python
from textgen import ChatGlmModel
model = ChatGlmModel("chatglm", "THUDM/chatglm-6b", peft_name="shibing624/chatglm-6b-csc-zh-lora")
r = model.predict(["对下面中文拼写纠错：\n少先队员因该为老人让坐。\n答："])
print(r) # ['少先队员应该为老人让座。\n错误字：因，坐']
```
PS：由于使用了开发中的peft库，可能由于版本更新，导致LoRA模型加载失败，建议使用下面的训练方法，自己训练LoRA模型。
### 2.2.2 训练 ChatGLM-6B 微调模型
1. 支持自定义训练数据集和训练参数，数据集格式参考[examples/data/zh_csc_test.tsv](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fdata%2Fzh_csc_test.tsv "https://github.com/shibing624/textgen/blob/main/examples/data/zh_csc_test.tsv")或者[examples/data/json_files/belle_10.json](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fdata%2Fjson_files%2Fbelle_10.json "https://github.com/shibing624/textgen/blob/main/examples/data/json_files/belle_10.json")
2. 支持AdaLoRA、LoRA、P_Tuning、Prefix_Tuning等部分参数微调方法，也支持全参微调
3. 支持多卡训练，支持混合精度训练
example: [examples/chatglm/training_chatglm_demo.py](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fchatglm%2Ftraining_chatglm_demo.py "https://github.com/shibing624/textgen/blob/main/examples/chatglm/training_chatglm_demo.py")
单卡训练：
```shell
cd examples/chatglm
CUDA_VISIBLE_DEVICES=0 python training_chatglm_demo.py --do_train --do_predict --num_epochs 1 --output_dir outputs_chatglm
```
多卡训练：
```shell
cd examples/chatglm
CUDA_VISIBLE_DEVICES=0,1 torchrun --nproc_per_node 2 training_chatglm_demo.py --do_train --do_predict --num_epochs 20
```
### 2.2.3 基于微调(LoRA)模型继续训练
如果需要基于Lora模型继续训练，可以使用下面的脚本合并模型为新的base model，再微调训练即可。
执行以下命令：
```shell
python -m textgen/chatglm/merge_peft_adapter.py \
--base_model_name_or_path path_to_original_base_model_dir \
--peft_model_path path_to_peft_model_dir \
--output_dir path_to_output_dir
```
参数说明：
```css
--base_model_name_or_path：存放HF格式的底座模型权重和配置文件的目录
--peft_model_path：存放PEFT格式的微调模型权重和配置文件的目录
--output_dir：指定保存全量模型权重的目录，默认为./merged
```
## 2.3 LLaMA 模型
### 2.3.1 使用 LLaMA 微调后的模型
example: [examples/gpt/inference_demo.py](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fgpt%2Finference_demo.py "https://github.com/shibing624/textgen/blob/main/examples/gpt/inference_demo.py")
show code example and result
```python
import sys
sys.path.append('../..')
from textgen import GptModel
def generate_prompt(instruction):
return f"""Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:{instruction}\n\n### Response:"""
model = GptModel("llama", "decapoda-research/llama-7b-hf", peft_name="ziqingyang/chinese-alpaca-lora-7b")
predict_sentence = generate_prompt("问：用一句话描述地球为什么是独一无二的。\n答：")
r = model.predict([predict_sentence])
print(r) # ['地球是唯一一颗拥有生命的行星。']
```
### 2.3.2训练 LLaMA 微调模型
1. 支持自定义训练数据集和训练参数，数据集格式参考[examples/data/zh_csc_test.tsv](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fdata%2Fzh_csc_test.tsv "https://github.com/shibing624/textgen/blob/main/examples/data/zh_csc_test.tsv")或者[shibing624/alpaca-zh](https://link.juejin.cn?target=https%3A%2F%2Fhuggingface.co%2Fdatasets%2Fshibing624%2Falpaca-zh "https://huggingface.co/datasets/shibing624/alpaca-zh")
2. 支持AdaLoRA、LoRA、P_Tuning、Prefix_Tuning等部分参数微调方法，也支持全参微调
3. 支持多卡训练，支持混合精度训练，使用方法同上（ChatGLM多卡训练）
example: [examples/llama/training_llama_demo.py](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fllama%2Ftraining_llama_demo.py "https://github.com/shibing624/textgen/blob/main/examples/llama/training_llama_demo.py")
### 2.3.3 基于微调(LoRA)模型继续训练
如果需要基于Lora模型继续训练，可以使用下面的脚本合并模型为新的base model，再微调训练即可。
单LoRA权重合并（适用于 Chinese-LLaMA, Chinese-LLaMA-Plus, Chinese-Alpaca）
执行以下命令：
```shell
python -m textgen/gpt/merge_peft_adapter.py \
--base_model_name_or_path path_to_original_base_model_dir \
--peft_model_path path_to_chinese_llama_or_alpaca_lora \
--output_type [pth|huggingface]
--output_dir path_to_output_dir
```
参数说明：
```diff
--base_model_name_or_path：存放HF格式的底座模型权重和配置文件的目录
--peft_model_path：中文LLaMA/Alpaca LoRA解压后文件所在目录，也可使用HF上的Lora模型名称，如`ziqingyang/chinese-alpaca-lora-7b`会自动下载对应模型
--output_type: 指定输出格式，可为pth或huggingface。若不指定，默认为huggingface
--output_dir：指定保存全量模型权重的目录，默认为./merged
--offload_dir（可选）：对于低内存用户需要指定一个offload缓存路径
```
### 2.3.4 训练领域模型
Note: 为了全面的介绍训练医疗大模型的过程，把4阶段训练方法(Pretraining, Supervised Finetuning, Reward Modeling and Reinforcement Learning)单独新建了一个repo：[shibing624/MedicalGPT](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2FMedicalGPT "https://github.com/shibing624/MedicalGPT")，请移步该repo查看训练方法。
## 2.4 BLOOM 模型
### 2.4.1训练 BLOOM 微调模型
example: [examples/bloom/training_bloom_demo.py](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fbloom%2Ftraining_bloom_demo.py "https://github.com/shibing624/textgen/blob/main/examples/bloom/training_bloom_demo.py")
## 2.5 ConvSeq2Seq 模型
训练并预测ConvSeq2Seq模型：
example: [examples/seq2sesq/training_convseq2seq_model_demo.py](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fseq2seq%2Ftraining_convseq2seq_model_demo.py "https://github.com/shibing624/textgen/blob/main/examples/seq2seq/training_convseq2seq_model_demo.py")
show code example and result
```python
import argparse
from loguru import logger
import sys
sys.path.append('../..')
from textgen.seq2seq.conv_seq2seq_model import ConvSeq2SeqModel
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--train_file', default='../data/zh_dialog.tsv', type=str, help='Training data file')
parser.add_argument('--do_train', action='store_true', help='Whether to run training.')
parser.add_argument('--do_predict', action='store_true', help='Whether to run predict.')
parser.add_argument('--output_dir', default='./outputs/convseq2seq_zh/', type=str, help='Model output directory')
parser.add_argument('--max_seq_length', default=50, type=int, help='Max sequence length')
parser.add_argument('--num_epochs', default=200, type=int, help='Number of training epochs')
parser.add_argument('--batch_size', default=32, type=int, help='Batch size')
args = parser.parse_args()
logger.info(args)
if args.do_train:
logger.info('Loading data...')
model = ConvSeq2SeqModel(epochs=args.num_epochs, batch_size=args.batch_size,
model_dir=args.output_dir, max_length=args.max_seq_length)
model.train_model(args.train_file)
print(model.eval_model(args.train_file))
if args.do_predict:
model = ConvSeq2SeqModel(epochs=args.num_epochs, batch_size=args.batch_size,
model_dir=args.output_dir, max_length=args.max_seq_length)
sentences = ["什么是ai", "你是什么类型的计算机", "你知道热力学吗"]
print("inputs:", sentences)
print('outputs:', model.predict(sentences))
if __name__ == '__main__':
main()
```
output:
```bash
inputs: ["什么是ai", "你是什么类型的计算机", "你知道热力学吗"]
outputs: ['人工智能是工程和科学的分支,致力于构建思维的机器。', '我的程序运行在python,所以我在任何运脑上工作！', '我不能错热是一个疯狂的人工智能"200年。']
```
## 2.6 BART 模型
训练并预测BART模型：
example: [examples/seq2sesq/training_bartseq2seq_zh_demo.py](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Fseq2seq%2Ftraining_bartseq2seq_zh_demo.py "https://github.com/shibing624/textgen/blob/main/examples/seq2seq/training_bartseq2seq_zh_demo.py")
output:
```shell
inputs: ['什么是ai', '你是什么类型的计算机', '你知道热力学吗']
outputs: ['人工智能是工程和科学的分支,致力于构', '我的程序运行在python,所以我在任何电脑上', '什么是热力学吗？']
```
## 2.7 T5 模型
example: [examples/t5/training_zh_t5_model_demo.py](https://link.juejin.cn?target=https%3A%2F%2Fgithub.com%2Fshibing624%2Ftextgen%2Fblob%2Fmain%2Fexamples%2Ft5%2Ftraining_zh_t5_model_demo.py "https://github.com/shibing624/textgen/blob/main/examples/t5/training_zh_t5_model_demo.py")
show code example and result
```python
import argparse
from loguru import logger
import pandas as pd
import sys
sys.path.append('../..')
from textgen.t5 import T5Model
def load_data(file_path):
data = []
with open(file_path, 'r', encoding='utf-8') as f:
for line in f:
line = line.strip('\n')
terms = line.split('\t')
if len(terms) == 2:
data.append(['QA', terms[0], terms[1]])
else:
logger.warning(f'line error: {line}')
return data
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--train_file', default='../data/zh_dialog.tsv', type=str, help='Training data file')
parser.add_argument('--model_type', default='t5', type=str, help='Transformers model type')
parser.add_argument('--model_name', default='Langboat/mengzi-t5-base', type=str, help='Transformers model or path')
parser.add_argument('--do_train', action='store_true', help='Whether to run training.')
parser.add_argument('--do_predict', action='store_true', help='Whether to run predict.')
parser.add_argument('--output_dir', default='./outputs/mengzi_t5_zh/', type=str, help='Model output directory')
parser.add_argument('--max_seq_length', default=50, type=int, help='Max sequence length')
parser.add_argument('--num_epochs', default=10, type=int, help='Number of training epochs')
parser.add_argument('--batch_size', default=32, type=int, help='Batch size')
args = parser.parse_args()
logger.info(args)
if args.do_train:
logger.info('Loading data...')
# train_data: Pandas DataFrame containing the 3 columns - `prefix`, `input_text`, `target_text`.
# - `prefix`: A string indicating the task to perform. (E.g. `"question"`, `"stsb"`)
# - `input_text`: The input text. `prefix` is prepended to form the full input. (