ComfyUI中基于Fluxgym训练Flux的Lora模型

1、介绍

Fluxgym训练非常方便，只需要更改一个配置文件内容即可。训练时也不需要提前进行图片裁剪、打标等前置工作。

本文章是介绍在16G以下显存下训练Flux模型的方法。

2、部署项目

（1）下载Fluxgym 和 kohya-ss/sd-scripts

bash 复制代码

git clone https://github.com/cocktailpeanut/fluxgym
cd fluxgym
git clone -b sd3 https://github.com/kohya-ss/sd-scripts

完成之后的文件夹结构应该如下所示：

bash 复制代码

/fluxgym
  app.py
  requirements.txt
  sd-scripts/

（2）创建虚拟环境fluxgym

使用conda创建，

bash 复制代码

conda create -n fluxgym python=3.10
conda activate fluxgym

（3）安装python依赖项

进入sd-scripts文件夹，安装依赖项

bash 复制代码

cd sd-scripts
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

完成之后返回根文件夹(fluxgym)，修改requirements.txt文件，将huggingface.co改为hf-mirror.com：

然后安装依赖项：

bash 复制代码

pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

（4）安装pytorch Nightly

bash 复制代码

pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu121

3、启动项目

指定server_name 和 server_port，启动服务。

cs 复制代码

cd fluxgym
export GRADIO_SERVER_NAME=0.0.0.0
export GRADIO_SERVER_PORT=16080
python app.py

启动成功的日志如下：

打开页面后显示如下：

4、训练模型

（1）模型选择

fluxgym默认支持flux-dev / flux-schnell / bdsqlsz/flux1-dev2pro-single 三种flux模型。

该信息在models.yaml文件里。

不过文件中指定的model全都是大于20G的，在16G显存中无法训练。

我们可以使用flux1-dev-fp8.safetensors模型，链接如下：

F.1-dev/fp8/NF4/GGUF-所需模型文件包-Other-墨幽-LiblibAI

（2）下载模型

该文件比较大，假如公司网络限速的话，可以直接获取下载地址，在linux服务器上下载。

Chrome浏览器上获取下载源地址的方法：

首先点击那14G文件进行下载

然后在浏览器上输入chrome://net-export/，单击开始记录日志，隔个5秒钟左右关闭记录，查看日志，找到liblibai-online.liblib.cloud的链接地址。

最后在linux服务器上直接通过wget命令进行下载。

注意这个是zip包，不是模型文件！！！下载之后需要通过unzip命令解压缩后才能使用。

下载中的日志：

然后在linux上通过wget命令下载该文件。

通过tail -f wget-log.4可以下载进度：

下载完毕后，通过mv命令修改为zip后缀的文件。

然后通过unzip命令解压文件。

然后把里面的文件放到fluxgym对应的目录之下：

mv flux1-dev-fp8.safetensors xx/fluxgym/models/unet

mv clip_l.safetensors xx/fluxgym/models/clip

mv t5xxl_fp8_e4m3fn.safetensors xx/fluxgym/models/clip

mv ae.sft xx/fluxgym/models/vae

修改app.py文件：

将文件中所有的t5xxl_fp16.safetensors替换为t5xxl_fp8_e4m3fn.safetensors。

（3）修改models.yaml文件

在末尾添加如下内容：

flux1-dev-fp8:

repo: .

base: .

license: other

license_name: flux1-dev-fp8-license

file: flux1-dev-fp8.safetensors

然后重新启动项目。

（4）启动训练

1）基本设置

The name of your LoRA：设置Lora的名称
Trigger word/sentence：lora的触发词，结尾处需要增加一个英文的逗号
Base model：基模，选择较小的那个模型
VRAM：选择12G
repeat trains per image：修改为1，默认为10，但是10的效果不一定比1好

上传图片，然后再点击"Add AI captions with Florence-2"，生成图片对应的提示词。首次生成时会自动下载模型，模型大概1.5G。

Max Train Expochs：最多的训练轮次，假如提前收敛则会提前结束。
Expected training steps：自动计算出来的训练步数
Sample Image Prompts：提示词样例，不影响训练结果
Sample Image Every N Steps：不要修改该值
Resize dataset images：训练模型结果对应的分辨率。

注意（使用阶段的剧透）：即使Resize dataset images设置为512*512，但是如果空潜空间图像设置为1024*1024，那么最终生成的还是1024*1024的图像。

2）高级选项

点击Advanced options打开高级选项：

save_every_n_epochs：每N次保存一次模型，总轮次不多的话填1
console_log_file：日志文件位置，比如："/data/work/xiehao/fluxgym/log2/2253"，记得加双引号。
console_log_simple：打勾
fp8_base_unet：打勾，因为我们使用的是fp8的模型

3）训练过程

29张图片，以上参数，会占用10G的显存。

29张图片，768分辨率，会占用12G的显存。

运行中观察Volatile GPU-Util的值，需要大于0，一般是99%或100%。

如果是0，说明停止训练了。

训练完整日志如下：

[2025-01-27 21:26:51] [INFO] Running bash "/data/work/xiehao/fluxgym/outputs/girl-flux/train.sh"
[2025-01-27 21:27:00] [INFO] 2025-01-27 21:27:00 INFO     highvram is enabled / highvramが有効です                                                                                                          train_util.py:4199
[2025-01-27 21:27:00] [INFO] WARNING  cache_latents_to_disk is enabled, so cache_latents is also enabled / cache_latents_to_diskが有効なため、cache_latentsを有効にします               train_util.py:4216
[2025-01-27 21:27:00] [INFO] /data/work/anaconda3/envs/fluxgym/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
[2025-01-27 21:27:00] [INFO] warnings.warn(
[2025-01-27 21:27:01] [INFO] You are using the default legacy behaviour of the <class 'transformers.models.t5.tokenization_t5.T5Tokenizer'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565
[2025-01-27 21:27:01] [INFO] 0%|          | 0/29 [00:00<?, ?it/s]
100%|██████████| 29/29 [00:00<00:00, 125655.80it/s]
[2025-01-27 21:27:01] [INFO] read caption:   0%|          | 0/29 [00:00<?, ?it/s]
read caption: 100%|██████████| 29/29 [00:00<00:00, 4335.74it/s]
[2025-01-27 21:27:01] [INFO] 0%|          | 0/29 [00:00<?, ?it/s]
100%|██████████| 29/29 [00:00<00:00, 679524.11it/s]
[2025-01-27 21:27:01] [INFO] accelerator device: cuda
[2025-01-27 21:27:01] [INFO] FLUX: Block swap enabled. Swapping 18 blocks, double blocks: 9, single blocks: 18.
[2025-01-27 21:27:33] [INFO] import network module: networks.lora_flux
[2025-01-27 21:27:33] [INFO] 0%|          | 0/29 [00:00<?, ?it/s]
100%|██████████| 29/29 [00:00<00:00, 6469.94it/s]
[2025-01-27 21:27:35] [INFO] 0%|          | 0/29 [00:00<?, ?it/s]
100%|██████████| 29/29 [00:00<00:00, 3760.78it/s]
[2025-01-27 21:27:41] [INFO] FLUX: Gradient checkpointing enabled. CPU offload: False
[2025-01-27 21:27:41] [INFO] prepare optimizer, data loader etc.
[2025-01-27 21:27:41] [INFO] override steps. steps for 6 epochs is / 指定エポックまでのステップ数: 174
[2025-01-27 21:27:41] [INFO] enable fp8 training for U-Net.
[2025-01-27 21:28:03] [INFO] running training / 学習開始
[2025-01-27 21:28:03] [INFO] num train images * repeats / 学習画像の数×繰り返し回数: 29
[2025-01-27 21:28:03] [INFO] num reg images / 正則化画像の数: 0
[2025-01-27 21:28:03] [INFO] num batches per epoch / 1epochのバッチ数: 29
[2025-01-27 21:28:03] [INFO] num epochs / epoch数: 6
[2025-01-27 21:28:03] [INFO] batch size per device / バッチサイズ: 1
[2025-01-27 21:28:03] [INFO] gradient accumulation steps / 勾配を合計するステップ数 = 1
[2025-01-27 21:28:03] [INFO] total optimization steps / 学習ステップ数: 174
[2025-01-27 21:28:47] [INFO] steps:   0%|          | 0/174 [00:00<?, ?it/s]
[2025-01-27 21:28:47] [INFO] epoch 1/6
[2025-01-27 21:56:53] [INFO] steps:   1%|          | 1/174 [00:55<2:40:05, 55.52s/it]
steps:   1%|          | 1/174 [00:55<2:40:05, 55.52s/it, avr_loss=0.345]
steps:   1%|          | 2/174 [01:51<2:39:54, 55.78s/it, avr_loss=0.345]
steps:   1%|          | 2/174 [01:51<2:39:54, 55.78s/it, avr_loss=0.375]
steps:   2%|▏         | 3/174 [02:48<2:40:20, 56.26s/it, avr_loss=0.375]
steps:   2%|▏         | 3/174 [02:48<2:40:20, 56.26s/it, avr_loss=0.444]
steps:   2%|▏         | 4/174 [03:47<2:40:50, 56.77s/it, avr_loss=0.444]
steps:   2%|▏         | 4/174 [03:47<2:40:50, 56.77s/it, avr_loss=0.473]
steps:   3%|▎         | 5/174 [04:45<2:40:51, 57.11s/it, avr_loss=0.473]
steps:   3%|▎         | 5/174 [04:45<2:40:51, 57.11s/it, avr_loss=0.49] 
steps:   3%|▎         | 6/174 [05:43<2:40:31, 57.33s/it, avr_loss=0.49]
steps:   3%|▎         | 6/174 [05:43<2:40:31, 57.33s/it, avr_loss=0.479]
steps:   4%|▍         | 7/174 [06:42<2:40:00, 57.49s/it, avr_loss=0.479]
steps:   4%|▍         | 7/174 [06:42<2:40:00, 57.49s/it, avr_loss=0.468]
steps:   5%|▍         | 8/174 [07:40<2:39:22, 57.60s/it, avr_loss=0.468]
steps:   5%|▍         | 8/174 [07:40<2:39:22, 57.60s/it, avr_loss=0.472]
steps:   5%|▌         | 9/174 [08:39<2:38:39, 57.69s/it, avr_loss=0.472]
steps:   5%|▌         | 9/174 [08:39<2:38:39, 57.69s/it, avr_loss=0.466]
steps:   6%|▌         | 10/174 [09:37<2:37:52, 57.76s/it, avr_loss=0.466]
steps:   6%|▌         | 10/174 [09:37<2:37:52, 57.76s/it, avr_loss=0.479]
steps:   6%|▋         | 11/174 [10:35<2:37:03, 57.81s/it, avr_loss=0.479]
steps:   6%|▋         | 11/174 [10:35<2:37:03, 57.81s/it, avr_loss=0.478]
steps:   7%|▋         | 12/174 [11:34<2:36:13, 57.86s/it, avr_loss=0.478]
steps:   7%|▋         | 12/174 [11:34<2:36:13, 57.86s/it, avr_loss=0.473]
steps:   7%|▋         | 13/174 [12:32<2:35:23, 57.91s/it, avr_loss=0.473]
steps:   7%|▋         | 13/174 [12:32<2:35:23, 57.91s/it, avr_loss=0.466]
steps:   8%|▊         | 14/174 [13:31<2:34:30, 57.94s/it, avr_loss=0.466]
steps:   8%|▊         | 14/174 [13:31<2:34:30, 57.94s/it, avr_loss=0.463]
steps:   9%|▊         | 15/174 [14:29<2:33:37, 57.97s/it, avr_loss=0.463]
steps:   9%|▊         | 15/174 [14:29<2:33:37, 57.97s/it, avr_loss=0.461]
steps:   9%|▉         | 16/174 [15:27<2:32:42, 57.99s/it, avr_loss=0.461]
steps:   9%|▉         | 16/174 [15:27<2:32:42, 57.99s/it, avr_loss=0.46] 
steps:  10%|▉         | 17/174 [16:26<2:31:47, 58.01s/it, avr_loss=0.46]
steps:  10%|▉         | 17/174 [16:26<2:31:47, 58.01s/it, avr_loss=0.454]
steps:  10%|█         | 18/174 [17:24<2:30:52, 58.03s/it, avr_loss=0.454]
steps:  10%|█         | 18/174 [17:24<2:30:52, 58.03s/it, avr_loss=0.452]
steps:  11%|█         | 19/174 [18:22<2:29:56, 58.04s/it, avr_loss=0.452]
steps:  11%|█         | 19/174 [18:22<2:29:56, 58.04s/it, avr_loss=0.452]
steps:  11%|█▏        | 20/174 [19:21<2:29:01, 58.06s/it, avr_loss=0.452]
steps:  11%|█▏        | 20/174 [19:21<2:29:01, 58.06s/it, avr_loss=0.451]
steps:  12%|█▏        | 21/174 [20:19<2:28:05, 58.07s/it, avr_loss=0.451]
steps:  12%|█▏        | 21/174 [20:19<2:28:05, 58.07s/it, avr_loss=0.449]
steps:  13%|█▎        | 22/174 [21:17<2:27:08, 58.08s/it, avr_loss=0.449]
steps:  13%|█▎        | 22/174 [21:17<2:27:08, 58.08s/it, avr_loss=0.446]
steps:  13%|█▎        | 23/174 [22:16<2:26:12, 58.10s/it, avr_loss=0.446]
steps:  13%|█▎        | 23/174 [22:16<2:26:12, 58.10s/it, avr_loss=0.444]
steps:  14%|█▍        | 24/174 [23:14<2:25:16, 58.11s/it, avr_loss=0.444]
steps:  14%|█▍        | 24/174 [23:14<2:25:16, 58.11s/it, avr_loss=0.443]
steps:  14%|█▍        | 25/174 [24:12<2:24:19, 58.11s/it, avr_loss=0.443]
steps:  14%|█▍        | 25/174 [24:12<2:24:19, 58.11s/it, avr_loss=0.441]
steps:  15%|█▍        | 26/174 [25:11<2:23:21, 58.12s/it, avr_loss=0.441]
steps:  15%|█▍        | 26/174 [25:11<2:23:21, 58.12s/it, avr_loss=0.437]
steps:  16%|█▌        | 27/174 [26:09<2:22:24, 58.13s/it, avr_loss=0.437]
steps:  16%|█▌        | 27/174 [26:09<2:22:24, 58.13s/it, avr_loss=0.442]
steps:  16%|█▌        | 28/174 [27:07<2:21:27, 58.13s/it, avr_loss=0.442]
steps:  16%|█▌        | 28/174 [27:07<2:21:27, 58.14s/it, avr_loss=0.439]
steps:  17%|█▋        | 29/174 [28:06<2:20:30, 58.14s/it, avr_loss=0.439]
steps:  17%|█▋        | 29/174 [28:06<2:20:30, 58.14s/it, avr_loss=0.436]
[2025-01-27 21:56:53] [INFO] saving checkpoint: /data/work/xiehao/fluxgym/outputs/girl-flux/girl-flux-000001.safetensors
[2025-01-27 21:56:53] [INFO] /data/work/xiehao/fluxgym/sd-scripts/networks/lora_flux.py:861: FutureWarning: Positional args are being deprecated, use kwargs instead. Refer to https://pytorch.org/docs/main/generated/torch.nn.Module.html#torch.nn.Module.state_dict for details.
[2025-01-27 21:56:53] [INFO] return super().state_dict(destination, prefix, keep_vars)
[2025-01-27 21:56:53] [INFO] 
[2025-01-27 21:56:53] [INFO] epoch 2/6
[2025-01-27 22:25:05] [INFO] steps:  17%|█▋        | 30/174 [29:04<2:19:34, 58.16s/it, avr_loss=0.436]
steps:  17%|█▋        | 30/174 [29:04<2:19:34, 58.16s/it, avr_loss=0.438]
steps:  18%|█▊        | 31/174 [30:03<2:18:37, 58.16s/it, avr_loss=0.438]
steps:  18%|█▊        | 31/174 [30:03<2:18:37, 58.16s/it, avr_loss=0.438]
steps:  18%|█▊        | 32/174 [31:01<2:17:40, 58.17s/it, avr_loss=0.438]
steps:  18%|█▊        | 32/174 [31:01<2:17:40, 58.17s/it, avr_loss=0.431]
steps:  19%|█▉        | 33/174 [31:59<2:16:43, 58.18s/it, avr_loss=0.431]
steps:  19%|█▉        | 33/174 [31:59<2:16:43, 58.18s/it, avr_loss=0.428]
steps:  20%|█▉        | 34/174 [32:58<2:15:46, 58.19s/it, avr_loss=0.428]
steps:  20%|█▉        | 34/174 [32:58<2:15:46, 58.19s/it, avr_loss=0.422]
steps:  20%|██        | 35/174 [33:56<2:14:48, 58.19s/it, avr_loss=0.422]
steps:  20%|██        | 35/174 [33:56<2:14:48, 58.19s/it, avr_loss=0.423]
steps:  21%|██        | 36/174 [34:55<2:13:50, 58.19s/it, avr_loss=0.423]
steps:  21%|██        | 36/174 [34:55<2:13:50, 58.19s/it, avr_loss=0.423]
steps:  21%|██▏       | 37/174 [35:53<2:12:52, 58.20s/it, avr_loss=0.423]
steps:  21%|██▏       | 37/174 [35:53<2:12:52, 58.20s/it, avr_loss=0.423]
steps:  22%|██▏       | 38/174 [36:51<2:11:54, 58.20s/it, avr_loss=0.423]
steps:  22%|██▏       | 38/174 [36:51<2:11:54, 58.20s/it, avr_loss=0.421]
steps:  22%|██▏       | 39/174 [37:49<2:10:56, 58.20s/it, avr_loss=0.421]
steps:  22%|██▏       | 39/174 [37:49<2:10:56, 58.20s/it, avr_loss=0.415]
steps:  23%|██▎       | 40/174 [38:47<2:09:58, 58.20s/it, avr_loss=0.415]
steps:  23%|██▎       | 40/174 [38:47<2:09:58, 58.20s/it, avr_loss=0.412]
steps:  24%|██▎       | 41/174 [39:46<2:09:00, 58.20s/it, avr_loss=0.412]
steps:  24%|██▎       | 41/174 [39:46<2:09:00, 58.20s/it, avr_loss=0.413]
steps:  24%|██▍       | 42/174 [40:44<2:08:02, 58.20s/it, avr_loss=0.413]
steps:  24%|██▍       | 42/174 [40:44<2:08:02, 58.20s/it, avr_loss=0.418]
steps:  25%|██▍       | 43/174 [41:42<2:07:04, 58.20s/it, avr_loss=0.418]
steps:  25%|██▍       | 43/174 [41:42<2:07:04, 58.20s/it, avr_loss=0.422]
steps:  25%|██▌       | 44/174 [42:41<2:06:06, 58.21s/it, avr_loss=0.422]
steps:  25%|██▌       | 44/174 [42:41<2:06:06, 58.21s/it, avr_loss=0.419]
steps:  26%|██▌       | 45/174 [43:39<2:05:08, 58.21s/it, avr_loss=0.419]
steps:  26%|██▌       | 45/174 [43:39<2:05:08, 58.21s/it, avr_loss=0.42] 
steps:  26%|██▋       | 46/174 [44:37<2:04:10, 58.21s/it, avr_loss=0.42]
steps:  26%|██▋       | 46/174 [44:37<2:04:10, 58.21s/it, avr_loss=0.421]
steps:  27%|██▋       | 47/174 [45:35<2:03:12, 58.21s/it, avr_loss=0.421]
steps:  27%|██▋       | 47/174 [45:35<2:03:12, 58.21s/it, avr_loss=0.425]
steps:  28%|██▊       | 48/174 [46:34<2:02:14, 58.21s/it, avr_loss=0.425]
steps:  28%|██▊       | 48/174 [46:34<2:02:14, 58.21s/it, avr_loss=0.423]
steps:  28%|██▊       | 49/174 [47:32<2:01:16, 58.21s/it, avr_loss=0.423]
steps:  28%|██▊       | 49/174 [47:32<2:01:16, 58.21s/it, avr_loss=0.427]
steps:  29%|██▊       | 50/174 [48:30<2:00:18, 58.22s/it, avr_loss=0.427]
steps:  29%|██▊       | 50/174 [48:30<2:00:18, 58.22s/it, avr_loss=0.428]
steps:  29%|██▉       | 51/174 [49:29<1:59:20, 58.22s/it, avr_loss=0.428]
steps:  29%|██▉       | 51/174 [49:29<1:59:20, 58.22s/it, avr_loss=0.431]
steps:  30%|██▉       | 52/174 [50:27<1:58:23, 58.22s/it, avr_loss=0.431]
steps:  30%|██▉       | 52/174 [50:27<1:58:23, 58.22s/it, avr_loss=0.429]
steps:  30%|███       | 53/174 [51:26<1:57:25, 58.23s/it, avr_loss=0.429]
steps:  30%|███       | 53/174 [51:26<1:57:25, 58.23s/it, avr_loss=0.427]
steps:  31%|███       | 54/174 [52:24<1:56:27, 58.23s/it, avr_loss=0.427]
steps:  31%|███       | 54/174 [52:24<1:56:27, 58.23s/it, avr_loss=0.427]
steps:  32%|███▏      | 55/174 [53:22<1:55:30, 58.24s/it, avr_loss=0.427]
steps:  32%|███▏      | 55/174 [53:23<1:55:30, 58.24s/it, avr_loss=0.434]
steps:  32%|███▏      | 56/174 [54:21<1:54:32, 58.24s/it, avr_loss=0.434]
steps:  32%|███▏      | 56/174 [54:21<1:54:32, 58.24s/it, avr_loss=0.426]
steps:  33%|███▎      | 57/174 [55:19<1:53:34, 58.25s/it, avr_loss=0.426]
steps:  33%|███▎      | 57/174 [55:19<1:53:34, 58.25s/it, avr_loss=0.428]
steps:  33%|███▎      | 58/174 [56:18<1:52:36, 58.25s/it, avr_loss=0.428]
steps:  33%|███▎      | 58/174 [56:18<1:52:36, 58.25s/it, avr_loss=0.428]
[2025-01-27 22:25:05] [INFO] saving checkpoint: /data/work/xiehao/fluxgym/outputs/girl-flux/girl-flux-000002.safetensors
[2025-01-27 22:25:06] [INFO] 
[2025-01-27 22:25:06] [INFO] epoch 3/6
[2025-01-27 22:53:19] [INFO] steps:  34%|███▍      | 59/174 [57:17<1:51:39, 58.26s/it, avr_loss=0.428]
steps:  34%|███▍      | 59/174 [57:17<1:51:39, 58.26s/it, avr_loss=0.428]
steps:  34%|███▍      | 60/174 [58:15<1:50:41, 58.26s/it, avr_loss=0.428]
steps:  34%|███▍      | 60/174 [58:15<1:50:41, 58.26s/it, avr_loss=0.427]
steps:  35%|███▌      | 61/174 [59:14<1:49:43, 58.26s/it, avr_loss=0.427]
steps:  35%|███▌      | 61/174 [59:14<1:49:43, 58.26s/it, avr_loss=0.426]
steps:  36%|███▌      | 62/174 [1:00:12<1:48:45, 58.26s/it, avr_loss=0.426]
steps:  36%|███▌      | 62/174 [1:00:12<1:48:45, 58.26s/it, avr_loss=0.424]
steps:  36%|███▌      | 63/174 [1:01:10<1:47:47, 58.26s/it, avr_loss=0.424]
steps:  36%|███▌      | 63/174 [1:01:10<1:47:47, 58.26s/it, avr_loss=0.424]
steps:  37%|███▋      | 64/174 [1:02:08<1:46:49, 58.26s/it, avr_loss=0.424]
steps:  37%|███▋      | 64/174 [1:02:08<1:46:49, 58.26s/it, avr_loss=0.422]
steps:  37%|███▋      | 65/174 [1:03:07<1:45:50, 58.26s/it, avr_loss=0.422]
steps:  37%|███▋      | 65/174 [1:03:07<1:45:50, 58.26s/it, avr_loss=0.424]
steps:  38%|███▊      | 66/174 [1:04:05<1:44:52, 58.26s/it, avr_loss=0.424]
steps:  38%|███▊      | 66/174 [1:04:05<1:44:52, 58.26s/it, avr_loss=0.423]
steps:  39%|███▊      | 67/174 [1:05:03<1:43:54, 58.26s/it, avr_loss=0.423]
steps:  39%|███▊      | 67/174 [1:05:03<1:43:54, 58.26s/it, avr_loss=0.425]
steps:  39%|███▉      | 68/174 [1:06:01<1:42:55, 58.26s/it, avr_loss=0.425]
steps:  39%|███▉      | 68/174 [1:06:01<1:42:55, 58.26s/it, avr_loss=0.426]
steps:  40%|███▉      | 69/174 [1:07:00<1:41:57, 58.27s/it, avr_loss=0.426]
steps:  40%|███▉      | 69/174 [1:07:00<1:41:57, 58.27s/it, avr_loss=0.43] 
steps:  40%|████      | 70/174 [1:07:58<1:40:59, 58.27s/it, avr_loss=0.43]
steps:  40%|████      | 70/174 [1:07:58<1:40:59, 58.27s/it, avr_loss=0.431]
steps:  41%|████      | 71/174 [1:08:57<1:40:01, 58.27s/it, avr_loss=0.431]
steps:  41%|████      | 71/174 [1:08:57<1:40:01, 58.27s/it, avr_loss=0.428]
steps:  41%|████▏     | 72/174 [1:09:55<1:39:03, 58.27s/it, avr_loss=0.428]
steps:  41%|████▏     | 72/174 [1:09:55<1:39:03, 58.27s/it, avr_loss=0.422]
steps:  42%|████▏     | 73/174 [1:10:53<1:38:05, 58.27s/it, avr_loss=0.422]
steps:  42%|████▏     | 73/174 [1:10:53<1:38:05, 58.27s/it, avr_loss=0.424]
steps:  43%|████▎     | 74/174 [1:11:52<1:37:07, 58.27s/it, avr_loss=0.424]
steps:  43%|████▎     | 74/174 [1:11:52<1:37:07, 58.27s/it, avr_loss=0.422]
steps:  43%|████▎     | 75/174 [1:12:50<1:36:08, 58.27s/it, avr_loss=0.422]
steps:  43%|████▎     | 75/174 [1:12:50<1:36:08, 58.27s/it, avr_loss=0.421]
steps:  44%|████▎     | 76/174 [1:13:48<1:35:10, 58.27s/it, avr_loss=0.421]
steps:  44%|████▎     | 76/174 [1:13:48<1:35:10, 58.27s/it, avr_loss=0.417]
steps:  44%|████▍     | 77/174 [1:14:47<1:34:12, 58.27s/it, avr_loss=0.417]
steps:  44%|████▍     | 77/174 [1:14:47<1:34:12, 58.27s/it, avr_loss=0.416]
steps:  45%|████▍     | 78/174 [1:15:45<1:33:14, 58.28s/it, avr_loss=0.416]
steps:  45%|████▍     | 78/174 [1:15:45<1:33:14, 58.28s/it, avr_loss=0.413]
steps:  45%|████▌     | 79/174 [1:16:43<1:32:16, 58.28s/it, avr_loss=0.413]
steps:  45%|████▌     | 79/174 [1:16:43<1:32:16, 58.28s/it, avr_loss=0.411]
steps:  46%|████▌     | 80/174 [1:17:42<1:31:18, 58.28s/it, avr_loss=0.411]
steps:  46%|████▌     | 80/174 [1:17:42<1:31:18, 58.28s/it, avr_loss=0.408]
steps:  47%|████▋     | 81/174 [1:18:40<1:30:20, 58.28s/it, avr_loss=0.408]
steps:  47%|████▋     | 81/174 [1:18:40<1:30:20, 58.28s/it, avr_loss=0.41] 
steps:  47%|████▋     | 82/174 [1:19:39<1:29:22, 58.29s/it, avr_loss=0.41]
steps:  47%|████▋     | 82/174 [1:19:39<1:29:22, 58.29s/it, avr_loss=0.412]
steps:  48%|████▊     | 83/174 [1:20:37<1:28:24, 58.29s/it, avr_loss=0.412]
steps:  48%|████▊     | 83/174 [1:20:37<1:28:24, 58.29s/it, avr_loss=0.412]
steps:  48%|████▊     | 84/174 [1:21:36<1:27:26, 58.29s/it, avr_loss=0.412]
steps:  48%|████▊     | 84/174 [1:21:36<1:27:26, 58.29s/it, avr_loss=0.401]
steps:  49%|████▉     | 85/174 [1:22:34<1:26:28, 58.29s/it, avr_loss=0.401]
steps:  49%|████▉     | 85/174 [1:22:34<1:26:28, 58.29s/it, avr_loss=0.4]  
steps:  49%|████▉     | 86/174 [1:23:33<1:25:29, 58.29s/it, avr_loss=0.4]
steps:  49%|████▉     | 86/174 [1:23:33<1:25:29, 58.29s/it, avr_loss=0.401]
steps:  50%|█████     | 87/174 [1:24:31<1:24:31, 58.29s/it, avr_loss=0.401]
steps:  50%|█████     | 87/174 [1:24:31<1:24:31, 58.29s/it, avr_loss=0.409]
[2025-01-27 22:53:19] [INFO] saving checkpoint: /data/work/xiehao/fluxgym/outputs/girl-flux/girl-flux-000003.safetensors
[2025-01-27 22:53:19] [INFO] 
[2025-01-27 22:53:19] [INFO] epoch 4/6
[2025-01-27 23:21:31] [INFO] steps:  51%|█████     | 88/174 [1:25:30<1:23:33, 58.30s/it, avr_loss=0.409]
steps:  51%|█████     | 88/174 [1:25:30<1:23:33, 58.30s/it, avr_loss=0.409]
steps:  51%|█████     | 89/174 [1:26:28<1:22:35, 58.30s/it, avr_loss=0.409]
steps:  51%|█████     | 89/174 [1:26:28<1:22:35, 58.30s/it, avr_loss=0.412]
steps:  52%|█████▏    | 90/174 [1:27:26<1:21:36, 58.30s/it, avr_loss=0.412]
steps:  52%|█████▏    | 90/174 [1:27:26<1:21:36, 58.30s/it, avr_loss=0.42] 
steps:  52%|█████▏    | 91/174 [1:28:24<1:20:38, 58.30s/it, avr_loss=0.42]
steps:  52%|█████▏    | 91/174 [1:28:24<1:20:38, 58.30s/it, avr_loss=0.418]
steps:  53%|█████▎    | 92/174 [1:29:23<1:19:40, 58.30s/it, avr_loss=0.418]
steps:  53%|█████▎    | 92/174 [1:29:23<1:19:40, 58.30s/it, avr_loss=0.417]
steps:  53%|█████▎    | 93/174 [1:30:21<1:18:41, 58.30s/it, avr_loss=0.417]
steps:  53%|█████▎    | 93/174 [1:30:21<1:18:41, 58.30s/it, avr_loss=0.417]
steps:  54%|█████▍    | 94/174 [1:31:19<1:17:43, 58.30s/it, avr_loss=0.417]
steps:  54%|█████▍    | 94/174 [1:31:19<1:17:43, 58.30s/it, avr_loss=0.413]
steps:  55%|█████▍    | 95/174 [1:32:18<1:16:45, 58.30s/it, avr_loss=0.413]
steps:  55%|█████▍    | 95/174 [1:32:18<1:16:45, 58.30s/it, avr_loss=0.413]
steps:  55%|█████▌    | 96/174 [1:33:16<1:15:47, 58.30s/it, avr_loss=0.413]
steps:  55%|█████▌    | 96/174 [1:33:16<1:15:47, 58.30s/it, avr_loss=0.412]
steps:  56%|█████▌    | 97/174 [1:34:14<1:14:48, 58.29s/it, avr_loss=0.412]
steps:  56%|█████▌    | 97/174 [1:34:14<1:14:48, 58.29s/it, avr_loss=0.409]
steps:  56%|█████▋    | 98/174 [1:35:12<1:13:50, 58.30s/it, avr_loss=0.409]
steps:  56%|█████▋    | 98/174 [1:35:12<1:13:50, 58.30s/it, avr_loss=0.407]
steps:  57%|█████▋    | 99/174 [1:36:11<1:12:52, 58.30s/it, avr_loss=0.407]
steps:  57%|█████▋    | 99/174 [1:36:11<1:12:52, 58.30s/it, avr_loss=0.407]
steps:  57%|█████▋    | 100/174 [1:37:09<1:11:54, 58.30s/it, avr_loss=0.407]
steps:  57%|█████▋    | 100/174 [1:37:09<1:11:54, 58.30s/it, avr_loss=0.404]
steps:  58%|█████▊    | 101/174 [1:38:08<1:10:55, 58.30s/it, avr_loss=0.404]
steps:  58%|█████▊    | 101/174 [1:38:08<1:10:55, 58.30s/it, avr_loss=0.405]
steps:  59%|█████▊    | 102/174 [1:39:06<1:09:57, 58.30s/it, avr_loss=0.405]
steps:  59%|█████▊    | 102/174 [1:39:06<1:09:57, 58.30s/it, avr_loss=0.403]
steps:  59%|█████▉    | 103/174 [1:40:05<1:08:59, 58.31s/it, avr_loss=0.403]
steps:  59%|█████▉    | 103/174 [1:40:05<1:08:59, 58.31s/it, avr_loss=0.403]
steps:  60%|█████▉    | 104/174 [1:41:03<1:08:01, 58.31s/it, avr_loss=0.403]
steps:  60%|█████▉    | 104/174 [1:41:03<1:08:01, 58.31s/it, avr_loss=0.406]
steps:  60%|██████    | 105/174 [1:42:02<1:07:03, 58.31s/it, avr_loss=0.406]
steps:  60%|██████    | 105/174 [1:42:02<1:07:03, 58.31s/it, avr_loss=0.411]
steps:  61%|██████    | 106/174 [1:43:00<1:06:04, 58.30s/it, avr_loss=0.411]
steps:  61%|██████    | 106/174 [1:43:00<1:06:04, 58.30s/it, avr_loss=0.414]
steps:  61%|██████▏   | 107/174 [1:43:58<1:05:06, 58.30s/it, avr_loss=0.414]
steps:  61%|██████▏   | 107/174 [1:43:58<1:05:06, 58.30s/it, avr_loss=0.415]
steps:  62%|██████▏   | 108/174 [1:44:56<1:04:08, 58.30s/it, avr_loss=0.415]
steps:  62%|██████▏   | 108/174 [1:44:56<1:04:08, 58.30s/it, avr_loss=0.42] 
steps:  63%|██████▎   | 109/174 [1:45:55<1:03:09, 58.30s/it, avr_loss=0.42]
steps:  63%|██████▎   | 109/174 [1:45:55<1:03:09, 58.30s/it, avr_loss=0.419]
steps:  63%|██████▎   | 110/174 [1:46:53<1:02:11, 58.30s/it, avr_loss=0.419]
steps:  63%|██████▎   | 110/174 [1:46:53<1:02:11, 58.30s/it, avr_loss=0.42] 
steps:  64%|██████▍   | 111/174 [1:47:51<1:01:12, 58.30s/it, avr_loss=0.42]
steps:  64%|██████▍   | 111/174 [1:47:51<1:01:12, 58.30s/it, avr_loss=0.42]
steps:  64%|██████▍   | 112/174 [1:48:49<1:00:14, 58.30s/it, avr_loss=0.42]
steps:  64%|██████▍   | 112/174 [1:48:49<1:00:14, 58.30s/it, avr_loss=0.425]
steps:  65%|██████▍   | 113/174 [1:49:48<59:16, 58.30s/it, avr_loss=0.425]  
steps:  65%|██████▍   | 113/174 [1:49:48<59:16, 58.30s/it, avr_loss=0.433]
steps:  66%|██████▌   | 114/174 [1:50:46<58:18, 58.31s/it, avr_loss=0.433]
steps:  66%|██████▌   | 114/174 [1:50:46<58:18, 58.31s/it, avr_loss=0.433]
steps:  66%|██████▌   | 115/174 [1:51:45<57:20, 58.31s/it, avr_loss=0.433]
steps:  66%|██████▌   | 115/174 [1:51:45<57:20, 58.31s/it, avr_loss=0.432]
steps:  67%|██████▋   | 116/174 [1:52:43<56:21, 58.31s/it, avr_loss=0.432]
steps:  67%|██████▋   | 116/174 [1:52:43<56:21, 58.31s/it, avr_loss=0.429]
[2025-01-27 23:21:31] [INFO] saving checkpoint: /data/work/xiehao/fluxgym/outputs/girl-flux/girl-flux-000004.safetensors
[2025-01-27 23:21:31] [INFO] 
[2025-01-27 23:21:31] [INFO] epoch 5/6
[2025-01-27 23:49:42] [INFO] steps:  67%|██████▋   | 117/174 [1:53:42<55:23, 58.31s/it, avr_loss=0.429]
steps:  67%|██████▋   | 117/174 [1:53:42<55:23, 58.31s/it, avr_loss=0.43] 
steps:  68%|██████▊   | 118/174 [1:54:40<54:25, 58.31s/it, avr_loss=0.43]
steps:  68%|██████▊   | 118/174 [1:54:40<54:25, 58.31s/it, avr_loss=0.432]
steps:  68%|██████▊   | 119/174 [1:55:38<53:26, 58.31s/it, avr_loss=0.432]
steps:  68%|██████▊   | 119/174 [1:55:38<53:26, 58.31s/it, avr_loss=0.425]
steps:  69%|██████▉   | 120/174 [1:56:36<52:28, 58.31s/it, avr_loss=0.425]
steps:  69%|██████▉   | 120/174 [1:56:36<52:28, 58.31s/it, avr_loss=0.424]
steps:  70%|██████▉   | 121/174 [1:57:35<51:30, 58.31s/it, avr_loss=0.424]
steps:  70%|██████▉   | 121/174 [1:57:35<51:30, 58.31s/it, avr_loss=0.431]
steps:  70%|███████   | 122/174 [1:58:33<50:31, 58.31s/it, avr_loss=0.431]
steps:  70%|███████   | 122/174 [1:58:33<50:31, 58.31s/it, avr_loss=0.433]
steps:  71%|███████   | 123/174 [1:59:31<49:33, 58.30s/it, avr_loss=0.433]
steps:  71%|███████   | 123/174 [1:59:31<49:33, 58.30s/it, avr_loss=0.435]
steps:  71%|███████▏  | 124/174 [2:00:29<48:35, 58.30s/it, avr_loss=0.435]
steps:  71%|███████▏  | 124/174 [2:00:29<48:35, 58.30s/it, avr_loss=0.433]
steps:  72%|███████▏  | 125/174 [2:01:27<47:36, 58.30s/it, avr_loss=0.433]
steps:  72%|███████▏  | 125/174 [2:01:27<47:36, 58.30s/it, avr_loss=0.436]
steps:  72%|███████▏  | 126/174 [2:02:26<46:38, 58.30s/it, avr_loss=0.436]
steps:  72%|███████▏  | 126/174 [2:02:26<46:38, 58.30s/it, avr_loss=0.435]
steps:  73%|███████▎  | 127/174 [2:03:24<45:40, 58.30s/it, avr_loss=0.435]
steps:  73%|███████▎  | 127/174 [2:03:24<45:40, 58.30s/it, avr_loss=0.439]
steps:  74%|███████▎  | 128/174 [2:04:22<44:41, 58.30s/it, avr_loss=0.439]
steps:  74%|███████▎  | 128/174 [2:04:22<44:41, 58.30s/it, avr_loss=0.434]
steps:  74%|███████▍  | 129/174 [2:05:20<43:43, 58.30s/it, avr_loss=0.434]
steps:  74%|███████▍  | 129/174 [2:05:20<43:43, 58.30s/it, avr_loss=0.434]
steps:  75%|███████▍  | 130/174 [2:06:19<42:45, 58.30s/it, avr_loss=0.434]
steps:  75%|███████▍  | 130/174 [2:06:19<42:45, 58.30s/it, avr_loss=0.435]
steps:  75%|███████▌  | 131/174 [2:07:17<41:47, 58.30s/it, avr_loss=0.435]
steps:  75%|███████▌  | 131/174 [2:07:17<41:47, 58.30s/it, avr_loss=0.435]
steps:  76%|███████▌  | 132/174 [2:08:16<40:48, 58.31s/it, avr_loss=0.435]
steps:  76%|███████▌  | 132/174 [2:08:16<40:48, 58.31s/it, avr_loss=0.438]
steps:  76%|███████▋  | 133/174 [2:09:14<39:50, 58.31s/it, avr_loss=0.438]
steps:  76%|███████▋  | 133/174 [2:09:14<39:50, 58.31s/it, avr_loss=0.438]
steps:  77%|███████▋  | 134/174 [2:10:13<38:52, 58.31s/it, avr_loss=0.438]
steps:  77%|███████▋  | 134/174 [2:10:13<38:52, 58.31s/it, avr_loss=0.433]
steps:  78%|███████▊  | 135/174 [2:11:11<37:54, 58.31s/it, avr_loss=0.433]
steps:  78%|███████▊  | 135/174 [2:11:11<37:54, 58.31s/it, avr_loss=0.432]
steps:  78%|███████▊  | 136/174 [2:12:09<36:55, 58.31s/it, avr_loss=0.432]
steps:  78%|███████▊  | 136/174 [2:12:09<36:55, 58.31s/it, avr_loss=0.43] 
steps:  79%|███████▊  | 137/174 [2:13:08<35:57, 58.31s/it, avr_loss=0.43]
steps:  79%|███████▊  | 137/174 [2:13:08<35:57, 58.31s/it, avr_loss=0.426]
steps:  79%|███████▉  | 138/174 [2:14:06<34:59, 58.31s/it, avr_loss=0.426]
steps:  79%|███████▉  | 138/174 [2:14:06<34:59, 58.31s/it, avr_loss=0.427]
steps:  80%|███████▉  | 139/174 [2:15:04<34:00, 58.31s/it, avr_loss=0.427]
steps:  80%|███████▉  | 139/174 [2:15:04<34:00, 58.31s/it, avr_loss=0.426]
steps:  80%|████████  | 140/174 [2:16:03<33:02, 58.31s/it, avr_loss=0.426]
steps:  80%|████████  | 140/174 [2:16:03<33:02, 58.31s/it, avr_loss=0.424]
steps:  81%|████████  | 141/174 [2:17:01<32:04, 58.31s/it, avr_loss=0.424]
steps:  81%|████████  | 141/174 [2:17:01<32:04, 58.31s/it, avr_loss=0.42] 
steps:  82%|████████▏ | 142/174 [2:17:59<31:05, 58.31s/it, avr_loss=0.42]
steps:  82%|████████▏ | 142/174 [2:17:59<31:05, 58.31s/it, avr_loss=0.419]
steps:  82%|████████▏ | 143/174 [2:18:58<30:07, 58.31s/it, avr_loss=0.419]
steps:  82%|████████▏ | 143/174 [2:18:58<30:07, 58.31s/it, avr_loss=0.418]
steps:  83%|████████▎ | 144/174 [2:19:56<29:09, 58.31s/it, avr_loss=0.418]
steps:  83%|████████▎ | 144/174 [2:19:56<29:09, 58.31s/it, avr_loss=0.42] 
steps:  83%|████████▎ | 145/174 [2:20:55<28:11, 58.31s/it, avr_loss=0.42]
steps:  83%|████████▎ | 145/174 [2:20:55<28:11, 58.31s/it, avr_loss=0.417]
[2025-01-27 23:49:42] [INFO] saving checkpoint: /data/work/xiehao/fluxgym/outputs/girl-flux/girl-flux-000005.safetensors
[2025-01-27 23:49:42] [INFO] 
[2025-01-27 23:49:42] [INFO] epoch 6/6
[2025-01-28 00:17:58] [INFO] steps:  84%|████████▍ | 146/174 [2:21:53<27:12, 58.31s/it, avr_loss=0.417]
steps:  84%|████████▍ | 146/174 [2:21:53<27:12, 58.31s/it, avr_loss=0.425]
steps:  84%|████████▍ | 147/174 [2:22:52<26:14, 58.32s/it, avr_loss=0.425]
steps:  84%|████████▍ | 147/174 [2:22:52<26:14, 58.32s/it, avr_loss=0.423]
steps:  85%|████████▌ | 148/174 [2:23:50<25:16, 58.32s/it, avr_loss=0.423]
steps:  85%|████████▌ | 148/174 [2:23:50<25:16, 58.32s/it, avr_loss=0.425]
steps:  86%|████████▌ | 149/174 [2:24:49<24:17, 58.32s/it, avr_loss=0.425]
steps:  86%|████████▌ | 149/174 [2:24:49<24:17, 58.32s/it, avr_loss=0.425]
steps:  86%|████████▌ | 150/174 [2:25:47<23:19, 58.32s/it, avr_loss=0.425]
steps:  86%|████████▌ | 150/174 [2:25:47<23:19, 58.32s/it, avr_loss=0.422]
steps:  87%|████████▋ | 151/174 [2:26:46<22:21, 58.32s/it, avr_loss=0.422]
steps:  87%|████████▋ | 151/174 [2:26:46<22:21, 58.32s/it, avr_loss=0.419]
steps:  87%|████████▋ | 152/174 [2:27:44<21:23, 58.32s/it, avr_loss=0.419]
steps:  87%|████████▋ | 152/174 [2:27:44<21:23, 58.32s/it, avr_loss=0.419]
steps:  88%|████████▊ | 153/174 [2:28:43<20:24, 58.32s/it, avr_loss=0.419]
steps:  88%|████████▊ | 153/174 [2:28:43<20:24, 58.32s/it, avr_loss=0.423]
steps:  89%|████████▊ | 154/174 [2:29:41<19:26, 58.32s/it, avr_loss=0.423]
steps:  89%|████████▊ | 154/174 [2:29:41<19:26, 58.32s/it, avr_loss=0.417]
steps:  89%|████████▉ | 155/174 [2:30:39<18:28, 58.32s/it, avr_loss=0.417]
steps:  89%|████████▉ | 155/174 [2:30:39<18:28, 58.32s/it, avr_loss=0.423]
steps:  90%|████████▉ | 156/174 [2:31:38<17:29, 58.32s/it, avr_loss=0.423]
steps:  90%|████████▉ | 156/174 [2:31:38<17:29, 58.32s/it, avr_loss=0.419]
steps:  90%|█████████ | 157/174 [2:32:36<16:31, 58.32s/it, avr_loss=0.419]
steps:  90%|█████████ | 157/174 [2:32:36<16:31, 58.32s/it, avr_loss=0.421]
steps:  91%|█████████ | 158/174 [2:33:35<15:33, 58.32s/it, avr_loss=0.421]
steps:  91%|█████████ | 158/174 [2:33:35<15:33, 58.32s/it, avr_loss=0.424]
steps:  91%|█████████▏| 159/174 [2:34:33<14:34, 58.32s/it, avr_loss=0.424]
steps:  91%|█████████▏| 159/174 [2:34:33<14:34, 58.32s/it, avr_loss=0.423]
steps:  92%|█████████▏| 160/174 [2:35:31<13:36, 58.32s/it, avr_loss=0.423]
steps:  92%|█████████▏| 160/174 [2:35:31<13:36, 58.32s/it, avr_loss=0.424]
steps:  93%|█████████▎| 161/174 [2:36:30<12:38, 58.32s/it, avr_loss=0.424]
steps:  93%|█████████▎| 161/174 [2:36:30<12:38, 58.32s/it, avr_loss=0.42] 
steps:  93%|█████████▎| 162/174 [2:37:28<11:39, 58.33s/it, avr_loss=0.42]
steps:  93%|█████████▎| 162/174 [2:37:28<11:39, 58.33s/it, avr_loss=0.429]
steps:  94%|█████████▎| 163/174 [2:38:27<10:41, 58.33s/it, avr_loss=0.429]
steps:  94%|█████████▎| 163/174 [2:38:27<10:41, 58.33s/it, avr_loss=0.426]
steps:  94%|█████████▍| 164/174 [2:39:25<09:43, 58.33s/it, avr_loss=0.426]
steps:  94%|█████████▍| 164/174 [2:39:25<09:43, 58.33s/it, avr_loss=0.426]
steps:  95%|█████████▍| 165/174 [2:40:24<08:44, 58.33s/it, avr_loss=0.426]
steps:  95%|█████████▍| 165/174 [2:40:24<08:44, 58.33s/it, avr_loss=0.427]
steps:  95%|█████████▌| 166/174 [2:41:22<07:46, 58.33s/it, avr_loss=0.427]
steps:  95%|█████████▌| 166/174 [2:41:22<07:46, 58.33s/it, avr_loss=0.425]
steps:  96%|█████████▌| 167/174 [2:42:21<06:48, 58.33s/it, avr_loss=0.425]
steps:  96%|█████████▌| 167/174 [2:42:21<06:48, 58.33s/it, avr_loss=0.425]
steps:  97%|█████████▋| 168/174 [2:43:19<05:49, 58.33s/it, avr_loss=0.425]
steps:  97%|█████████▋| 168/174 [2:43:19<05:49, 58.33s/it, avr_loss=0.425]
steps:  97%|█████████▋| 169/174 [2:44:18<04:51, 58.33s/it, avr_loss=0.425]
steps:  97%|█████████▋| 169/174 [2:44:18<04:51, 58.33s/it, avr_loss=0.43] 
steps:  98%|█████████▊| 170/174 [2:45:16<03:53, 58.33s/it, avr_loss=0.43]
steps:  98%|█████████▊| 170/174 [2:45:16<03:53, 58.33s/it, avr_loss=0.431]
steps:  98%|█████████▊| 171/174 [2:46:15<02:55, 58.33s/it, avr_loss=0.431]
steps:  98%|█████████▊| 171/174 [2:46:15<02:55, 58.33s/it, avr_loss=0.431]
steps:  99%|█████████▉| 172/174 [2:47:13<01:56, 58.34s/it, avr_loss=0.431]
steps:  99%|█████████▉| 172/174 [2:47:13<01:56, 58.34s/it, avr_loss=0.434]
steps:  99%|█████████▉| 173/174 [2:48:12<00:58, 58.34s/it, avr_loss=0.434]
steps:  99%|█████████▉| 173/174 [2:48:12<00:58, 58.34s/it, avr_loss=0.433]
steps: 100%|██████████| 174/174 [2:49:10<00:00, 58.34s/it, avr_loss=0.433]
steps: 100%|██████████| 174/174 [2:49:10<00:00, 58.34s/it, avr_loss=0.434]
[2025-01-28 00:17:58] [INFO] saving checkpoint: /data/work/xiehao/fluxgym/outputs/girl-flux/girl-flux.safetensors
[2025-01-28 00:17:58] [INFO] steps: 100%|██████████| 174/174 [2:49:10<00:00, 58.34s/it, avr_loss=0.434]
[2025-01-28 00:18:02] [INFO] Command exited successfully
[2025-01-28 00:18:02] [INFO] Runner: <LogsViewRunner nb_logs=54 exit_code=0>

训练中，会在 /data/work/xiehao/fluxgym/outputs/tmallyc-flux 生成对应的lora模型。

本文参考：

Flux"炼丹炉"------fluxgym安装教程-CSDN博客