大模型Qwen3企业业务数据微调之模型导出

1. webui界面导出

微调训练完成,在指定的Output dir目录下,有如下文件:

访问http://192.168.0.24:7860/, 选择"Export"版面,如下:

按上图"Export"就可以执行模型合并和导出,中间有很多错误,下面再具体说。

2. 命令行导出

其实习惯Linux系统上开发和运维的人,是喜欢编写脚本来操作的,如下:

vim export_model.sh

bash 复制代码
#!/bin/bash

export DISABLE_VERSION_CHECK=1

llamafactory-cli export \
    --model_name_or_path /home/honya/.cache/modelscope/hub/models/Qwen/Qwen3.5-9B-Base \
    --adapter_name_or_path /home/honya/llm/LlamaFactory/saves/Qwen3.5-9B-Base/lora/train_2026-03-17-08-13-37 \
    --template qwen3 \
    --export_dir /home/honya/llm/LlamaFactory/saves/my_model/qwen3.5-9b-sft \
    --export_size 2 \
    --export_legacy_format False \
    --trust_remote_code

3. 执行中出现的错误

今天有强大的LLM chat, 有问题看错误日志,把错误日志截取去问AI好了,基本上都能解决。

File "/data/llm/qwen3/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 1380, in from_pretrained raise ValueError( ValueError: Unrecognized model in /home/honya/llm/LlamaFactory/saves/Qwen3.5-9B-Base/lora/train_2026-03-17-08-13-37. Should have a `model_type` key in its config.json,

原因:开始把--model_name_or_path写成训练导出的目录了。

解决:下面2个地址要写清楚,不能反。

--model_name_or_path /home/honya/.cache/modelscope/hub/models/Qwen/Qwen3.5-9B-Base \

--adapter_name_or_path /home/honya/llm/LlamaFactory/saves/Qwen3.5-9B-Base/lora/train_2026-03-17-08-13-37 \

还有,界面上就没找到adapter_name_or_path这个选项,所以界面上执行总是不成功。

Traceback (most recent call last): File "/data/llm/qwen3/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 1360, in from_pretrained config_class = CONFIG_MAPPINGconfig_dict\["model_type"] ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/llm/qwen3/lib/python3.12/site-packages/transformers/models/auto/configuration_auto.py", line 1048, in getitem raise KeyError(key) KeyError: 'qwen3_5'

原因:开始选择的是 --template qwen3_5;

解决:改为qwen3。

File "/data/llm/LlamaFactory/src/llamafactory/model/loader.py", line 142, in load_model patch_config(config, tokenizer, model_args, init_kwargs, is_trainable) File "/data/llm/LlamaFactory/src/llamafactory/model/patcher.py", line 146, in patch_config configure_kv_cache(config, model_args, is_trainable) File "/data/llm/LlamaFactory/src/llamafactory/model/model_utils/kv_cache.py", line 33, in configure_kv_cache setattr(config.text_config, "use_cache", model_args.use_kv_cache) AttributeError: 'dict' object has no attribute 'use_cache'

原因:config.json中配置和库版本问题。

解决:执行命令加参数:

--trust_remote_code

Traceback (most recent call last): File "/data/llm/LlamaFactory/src/llamafactory/model/loader.py", line 79, in load_tokenizer tokenizer = AutoTokenizer.from_pretrained( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/data/llm/qwen3/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 1153, in from_pretrained ValueError: Tokenizer class TokenizersBackend does not exist or is not currently imported.

原因:这个很扯淡,在webui上训练时,transformers不能用最新版本,我降了版本,但是低版本没有TokenizersBackend啊。

解决:pip install tokenizers --upgrade,直接安装到最新版本,最后就成功了:

transformers 5.3.0

成功的日志如下:

WARNING\|logging.py:330 2026-03-18 13:37:46,330 >> The fast path is not available because one of the required library is not installed. Falling back to torch implementation. To install follow https://github.com/fla-org/flash-linear-attention#installation and https://github.com/Dao-AILab/causal-conv1d Loading weights: 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 760/760 00:03\<00:00, 234.64it/s INFO\|utils.py:410 2026-03-18 13:37:50,909 >> Generation config file not found, using a generation config created from the model config. INFO\|configuration_utils.py:965 2026-03-18 13:37:50,909 >> loading configuration file /home/honya/.cache/modelscope/hub/models/Qwen/Qwen3.5-9B-Base/config.json INFO\|configuration_utils.py:1014 2026-03-18 13:37:50,909 >> Generate config GenerationConfig {} INFO\|dynamic_module_utils.py:406 2026-03-18 13:37:50,910 >> Could not locate the custom_generate/generate.py inside /home/honya/.cache/modelscope/hub/models/Qwen/Qwen3.5-9B-Base. INFO\|2026-03-18 13:37:50 llamafactory.model.model_utils.attention:144 >> Using torch SDPA for faster training and inference. INFO\|2026-03-18 13:39:25 llamafactory.model.adapter:144 >> Merged 1 adapter(s). INFO\|2026-03-18 13:39:25 llamafactory.model.adapter:144 >> Loaded adapter(s): /home/honya/llm/LlamaFactory/saves/Qwen3.5-9B-Base/lora/train_2026-03-17-08-13-37 INFO\|2026-03-18 13:39:25 llamafactory.model.loader:144 >> all params: 9,409,813,744 INFO\|2026-03-18 13:39:25 llamafactory.train.tuner:144 >> Convert model dtype to: torch.bfloat16. INFO\|configuration_utils.py:434 2026-03-18 13:39:27,984 >> Configuration saved in /home/honya/llm/LlamaFactory/saves/my_model/qwen3.5-9b-sft/config.json INFO\|configuration_utils.py:803 2026-03-18 13:39:27,985 >> Configuration saved in /home/honya/llm/LlamaFactory/saves/my_model/qwen3.5-9b-sft/generation_config.json Writing model shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 10/10 00:58\<00:00, 5.86s/it INFO\|modeling_utils.py:3446 2026-03-18 13:40:26,659 >> The model is bigger than the maximum size per checkpoint (2GB) and is going to be split in 10 checkpoint shards. You can find where each parameters has been saved in the index located at /home/honya/llm/LlamaFactory/saves/my_model/qwen3.5-9b-sft/model.safetensors.index.json. INFO\|tokenization_utils_base.py:3258 2026-03-18 13:40:26,661 >> chat template saved in /home/honya/llm/LlamaFactory/saves/my_model/qwen3.5-9b-sft/chat_template.jinja INFO\|tokenization_utils_base.py:2091 2026-03-18 13:40:26,661 >> tokenizer config file saved in /home/honya/llm/LlamaFactory/saves/my_model/qwen3.5-9b-sft/tokenizer_config.json INFO\|tokenization_utils_base.py:3258 2026-03-18 13:40:26,942 >> chat template saved in /home/honya/llm/LlamaFactory/saves/my_model/qwen3.5-9b-sft/chat_template.jinja INFO\|tokenization_utils_base.py:2091 2026-03-18 13:40:26,954 >> tokenizer config file saved in /home/honya/llm/LlamaFactory/saves/my_model/qwen3.5-9b-sft/tokenizer_config.json INFO\|processing_utils.py:873 2026-03-18 13:40:28,623 >> processor saved in /home/honya/llm/LlamaFactory/saves/my_model/qwen3.5-9b-sft/processor_config.json INFO\|2026-03-18 13:40:28 llamafactory.train.tuner:144 >> Ollama modelfile saved in /home/honya/llm/LlamaFactory/saves/my_model/qwen3.5-9b-sft/Modelfile

新导出的微调模型在这个目录下:

4. 部署

采用vLLM来部署刚导出的模型,新创建一个虚拟环境:

cd ~/llm

python3 -m venv vLLM

source ./vLLM/bin/activate

pip install vllm

pip list

由于cuda硬件是最新的,依赖库都用最新的,没出现什么错误。

编写一个启动脚本: vim run_vllm.sh

bash 复制代码
#!/bin/bash


# 安装vLLM
# pip install vllm

# 启动服务
rm -f ./nohup_vllm.out 

nohup python -m vllm.entrypoints.openai.api_server \
  --model /home/honya/llm/LlamaFactory/saves/my_model/qwen3.5-9b-sft \
  --served-model-name my-qwen3.5-9b-sft \
  --tensor-parallel-size 1 \
  --trust-remote-code \
  --max-model-len 65536 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --port 38080 \
  --host 0.0.0.0 >./nohup_vllm.out 2>&1 &

要提供openapi兼容的服务接口,故需要vllm.entrypoints.openai.api_server,不要用网上说的默认的;

要主动设置模型id:--served-model-name my-qwen3.5-9b-sft

--max-model-len 65536 要足够大,如果是2048,openclaw连接会提示上下文窗口太小的错误,访问失败。

--model 设置的目录就是我们导出的新模型目录。

执行很慢,执行成功后会有38080端口侦听:

访问swagger文档:http://192.168.0.24:38080/docs

上面的id就是模型ID, 在openclaw配置接入时必须填写正确的。

5. 测试

5.1 编写脚本访问

python 复制代码
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sys
import requests


def main():
    content = sys.argv[1]

    # 注意 URL 变成了 /v1/chat/completions
    url = "http://vllm:38080/v1/chat/completions"

    payload = {
        "model": "my-qwen3.5-9b-sft",  # 这里填你在 vLLM 启动时指定的模型名称
        "messages": [
            {"role": "user", "content": content}
        ],
        "max_tokens": 2048,
        "temperature": 0.7,  # 适当调高温度,让回答更自然
        "top_p": 0.9
    }

    resp = requests.post(url, json=payload)
    result = resp.json()

    # 解析 chat 接口的返回结构
    if "choices" in result and len(result["choices"]) > 0:
        print(result["choices"][0]["message"]["content"])
    else:
        print("Error:", result)


if __name__ == "__main__":
    main()

能正常聊天返回信息就是成功了。

5.2 openclaw接入

在.openclaw/openclaw.json添加配置:

"my-vllm": {

"baseUrl": "http://vllm:38080/v1",

"apiKey": "local-key",

"api": "openai-completions",

"models": [

{

"id": "my-qwen3.5-9b-sft",

"name": "Qwen3.5-9B-SFT",

"reasoning": false,

"input": [

"text"

],

"cost": {

"input": 0,

"output": 0,

"cacheRead": 0,

"cacheWrite": 0

},

"contextWindow": 32768,

"maxTokens": 32768

}

]

}

}

能在openclaw界面上聊天成功,就是接入成功了。

相关推荐
AI算力5 小时前
让安全驾驶有“AI”相伴|腾视科技DMS视频监控一体机,守护每一次出行
人工智能·科技·ai·车载终端·ai算力·车载智能终端·车载dvr
泯泷5 小时前
超越基准:Claude Opus 4.5 的 4 个惊人洞察
人工智能
果果软件6 小时前
2026 后台管理系统选型:若依、JeecgBoot、芋道、NocoBase 和 GewuMind Boot 怎么选?
java·人工智能·后端·ai·开源
Csvn6 小时前
刚开始用 WorkBuddy,第一步该做什么?
人工智能
一水鉴天6 小时前
同一与唯一:AppBus 外壳与 DevOps 内核 之1 20261121(元宝)
人工智能·机器学习
Wellijohn6 小时前
一个 Android 老兵的十年开源路:从 167 star 的滑动控件,到本地优先的 AI Agent
android·人工智能·开源
ZONGYUAN-ROOT6 小时前
【API配额】API 配额监控告警不生效?用 Python 监控读数变化量,抓住静态阈值漏掉的「假重置」
人工智能
Wellijohn7 小时前
我把公司内部的 12 个 CRUD 接口交给了 AI Agent,然后省出了两周
人工智能·python·mcp
伞伞悦读8 小时前
【第92期】逻辑回归的分数概率与阈值的实验与排查
人工智能·python·深度学习·机器学习