1. 模型简介
Animate Anyone是一项角色动画视频生成技术,能将静态图像依据指定动作生成动态的角色动画视频。该技术利用扩散模型,以保持图像到视频转换中的时间一致性和内容细节。训练由两阶段组成,对不同组网成分进行微调。具体实现借鉴于MooreThreads/Moore-AnimateAnyone。
2. 环境准备
安装新版本ppdiffusers以及该项目相关依赖。
In [ ]
!pip install https://paddlenlp.bj.bcebos.com/models/community/junnyu/wheels/ppdiffusers-0.24.0-py3-none-any.whl --user
!pip install -r requirements.txt --user
3. 模型下载
运行以下自动下载脚本,下载 AnimateAnyone 推理以及训练初始化模型权重文件,模型权重文件将存储在./pretrained_weights
下。
In [3]
!python scripts/download_weights.py
/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
W0420 15:57:14.067077 131773 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 12.0, Runtime API Version: 11.8
W0420 15:57:14.068542 131773 gpu_resources.cc:164] device: 0, cuDNN Version: 8.9.
Preparing AnimateAnyone pretrained weights...
(…)ity/tsaiyue/AnimateAnyone_PD/config.json: 100%|█| 746/746 [00:00<00:00, 2.56M
(…)AnimateAnyone_PD/denoising_unet.pdparams: 100%|▉| 3.44G/3.44G [00:27<00:00, 1
(…)eAnyone_PD/motion_module_stage2.pdparams: 100%|▉| 909M/909M [00:08<00:00, 108
(…)ue/AnimateAnyone_PD/pose_guider.pdparams: 100%|█| 4.35M/4.35M [00:02<00:00, 1
(…)AnimateAnyone_PD/reference_unet.pdparams: 100%|▉| 3.44G/3.44G [00:30<00:00, 1
(…)e_PD/control_v11p_sd15_openpose.pdparams: 100%|▉| 1.45G/1.45G [00:17<00:00, 8
(…)one_PD/animatediff_mm_sd_v15_v2.pdparams: 100%|▉| 1.82G/1.82G [00:07<00:00, 2
(…)D/denoising_unet_initial4stage1.pdparams: 100%|▉| 3.44G/3.44G [00:39<00:00, 8
Preparing DWPose weights...
4. 两阶段训练
4.1 训练数据准备
训练数据由ubc_fashion和bili_dance两个数据集组成,其中ubc_fashion包含598组数据,bili_dance包含2451组数据,数据获取方式如下:
In [7]
# ubc_fashion数据集下载
!wget https://bj.bcebos.com/paddlenlp/models/community/tsaiyue/ubcNbili_data/ubcNbili_data.tar.gz
# 文件解压
!tar -xzvf ubcNbili_data.tar.gz
# 删除压缩文件
!rm -rf ubcNbili_data.tar.gz
./ubcNbili_data/video_dwpose/BV1Gu4y1e72H_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1dp4y1j7N9_segment_03_part_1_cut.mp4
./ubcNbili_data/video_dwpose/BV1nm4y1Q7PU_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1EY411z7qN_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1Tz4y1x7Nu_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1sK411e7zk_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1vu4m1P7yr_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1jg411G7dT_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1hP4y117v1_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1eN411B7LK_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1G44y1V7KZ_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1sW4y1m755_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1LK411Y7Nx_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1Dm4y1X7JT_segment_34.mp4
./ubcNbili_data/video_dwpose/BV1TY4y1a7Bw_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1mC4y1A7gt_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1nQ4y1X7L2_segment_07.mp4
./ubcNbili_data/video_dwpose/BV19N411p7FJ_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1CY4y1M7Bj_segment_12.mp4
./ubcNbili_data/video_dwpose/BV1vJ411x7af_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1zj411L7v9_segment_03.mp4
./ubcNbili_data/video_dwpose/BV12H4y1L7Qf_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1eN411B7LK_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1bS4y1a71t_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1aH4y177tp_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1YF411v7GN_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Mh411W7GV_segment_03.mp4
./ubcNbili_data/video_dwpose/BV16F4m1u7KC_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1Wp4y1m7AY_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1h34y1e7h4_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1br4y1F7mr_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1QS4y1P7Yu_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1bz411e7JE_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1QM411a7Wy_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1TY4y1a7Bw_segment_00.mp4
./ubcNbili_data/video_dwpose/BV125411R7ru_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1b841117TP_segment_04_part_2_cut.mp4
./ubcNbili_data/video_dwpose/BV1iv411q7EL_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1HF411u7QK_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1uV4y1x7RU_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1nZ4y187Sh_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1K84y1L7UU_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1Tk4y1H7n1_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1SE411R7za_segment_02.mp4
./ubcNbili_data/video_dwpose/BV14V4y1Y78o_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1eJ4m1W7k4_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Qy421a7Jf_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1YF411v7GN_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1Dk4y1m725_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1nQ4y1X7L2_segment_11.mp4
./ubcNbili_data/video_dwpose/BV1QK4y1h7GJ_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1mJ4m1Y7zy_segment_01.mp4
./ubcNbili_data/video_dwpose/BV135411R7af_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1sr4y1n7u1_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1Mg411U71D_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1i3411J72N_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1ns4y1C7sK_segment_11.mp4
./ubcNbili_data/video_dwpose/BV1nZ4y1y77w_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1gh411M711_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1LZ42127Gj_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1Ds4y1t7Ab_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1bT4y1H7FH_segment_04_part_1_cut.mp4
./ubcNbili_data/video_dwpose/BV1B44y157Vq_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1LK421y7kY_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1vr4y1m78d_segment_11.mp4
./ubcNbili_data/video_dwpose/BV1MF411n7cP_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1xm411D7qF_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1s14y1u7GZ_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1sW4y1m755_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1dc411C7KP_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1vJ411x7af_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1ur4y1r7pU_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1Me4y1u7NH_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1PU4y1Y7SN_segment_02.mp4
./ubcNbili_data/video_dwpose/BV14Q4y1J7Ko_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1Tk4y1H7n1_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1iW4y1Y7HD_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1mp4y1Z7NJ_segment_13.mp4
./ubcNbili_data/video_dwpose/BV1Au4y1M7b7_segment_04_part_1_cut.mp4
./ubcNbili_data/video_dwpose/BV1ua4y167kX_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1PC411W7ag_segment_02.mp4
./ubcNbili_data/video_dwpose/BV19A411J7YL_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1s8411v7i6_segment_11.mp4
./ubcNbili_data/video_dwpose/BV1aF4m157ZE_segment_01.mp4
./ubcNbili_data/video_dwpose/BV14A41137wj_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1Zu4y137eF_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1CY4y1M7Bj_segment_25.mp4
./ubcNbili_data/video_dwpose/BV1FT4y1x7wx_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1pg411V74C_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1b44y1F7BA_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1894y1P7w8_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1H8411b7ML_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1zo4y1x77S_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1bj411V7zk_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1hP4y117v1_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1UC4y1M72W_segment_03.mp4
./ubcNbili_data/video_dwpose/BV19L4y1n7H1_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1ua411X7aW_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1M24y1d7Qc_segment_03.mp4
./ubcNbili_data/video_dwpose/BV123411Y7SC_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1Zu4m1A7DM_segment_03.mp4
./ubcNbili_data/video_dwpose/BV19A411J7YL_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1LK421y7kY_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1FD4y1w7Pi_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1R3411A7g4_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1zN411u7sa_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1wp4y1d7nH_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1UC4y1M72W_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1ua411X7aW_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1XT4y1K79d_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1Ew411g7MB_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1FD4y1w7Pi_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1ip4y1T7iE_segment_03_part_1_cut.mp4
./ubcNbili_data/video_dwpose/BV1BZ421a7Zq_segment_01_part_2_cut.mp4
./ubcNbili_data/video_dwpose/BV1ur4y1r7pU_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1Tt411c7wy_segment_07.mp4
./ubcNbili_data/video_dwpose/BV1Ze4y1D7Es_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1Jg4y1R7Rh_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1GK421k7jh_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1Pf4y177j7_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1CY4y1M7Bj_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1JF41117Fz_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1564y1o75r_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1mw411Q76N_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1Dm4y1X7JT_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1tG411G7kG_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1YF411v7GN_segment_12.mp4
./ubcNbili_data/video_dwpose/BV1uS4y1N7rW_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1Wv4y147o1_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Pf4y177j7_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1UC4y1M72W_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1cQ4y1p78p_segment_07.mp4
./ubcNbili_data/video_dwpose/BV1qt4y187ZE_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1gW4y1c7f3_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1D24y1P7TT_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1ZZ4y1c7ff_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1sp4y1L7rV_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1DT4y1G7bP_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1V34y1u7DX_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Wp4y1m7AY_segment_01.mp4
./ubcNbili_data/video_dwpose/BV123411Y7SC_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1X8411B7v6_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1vr4y1m78d_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1b54y1M7uu_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1ua4y167kX_segment_07.mp4
./ubcNbili_data/video_dwpose/BV1mx4y1f7eT_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1fo4y1t7Da_segment_11.mp4
./ubcNbili_data/video_dwpose/BV1B44y157Vq_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1sK411e7zk_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1nM4y1p7YP_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1Po4y1Q7yB_segment_11.mp4
./ubcNbili_data/video_dwpose/BV1Me4y1u7NH_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1Dm4y1X7JT_segment_28.mp4
./ubcNbili_data/video_dwpose/BV1Dm4y1X7JT_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1QS4y1P7Yu_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1eJ4m1W7k4_segment_00_part_1_cut.mp4
./ubcNbili_data/video_dwpose/BV1cE411j7RL_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1mp4y1Z7NJ_segment_10.mp4
./ubcNbili_data/video_dwpose/BV1Ni4y1B79Q_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1TZ4y1c7Te_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1Gu4y1e72H_segment_20.mp4
./ubcNbili_data/video_dwpose/BV1As4y1t7vr_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1ju4m137Mh_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1Ze4y1D7Es_segment_14.mp4
./ubcNbili_data/video_dwpose/BV1ns4y1C7sK_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1Gc41147a4_segment_07.mp4
./ubcNbili_data/video_dwpose/BV1uK411e7b8_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1bT4y1H7FH_segment_03_part_1_cut.mp4
./ubcNbili_data/video_dwpose/BV1mJ4m1Y7zy_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1Qc41167av_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1MH4y1y7uD_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1BG41117sV_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1c2421u7Ra_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1eN411B7LK_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1EK411u7e6_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1wp4y1d7nH_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Au4y1M7b7_segment_03_part_1_cut.mp4
./ubcNbili_data/video_dwpose/BV1Gu4y1e72H_segment_10.mp4
./ubcNbili_data/video_dwpose/BV1nZ4y1E7CD_segment_07.mp4
./ubcNbili_data/video_dwpose/BV1EK411u7e6_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1bz411e7JE_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1Po4y1Q7yB_segment_10.mp4
./ubcNbili_data/video_dwpose/BV1vF4m177jX_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1p84y1c7VC_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1Ze4y1D7Es_segment_10.mp4
./ubcNbili_data/video_dwpose/BV1J5411D7Wq_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1nZ4y1y77w_segment_01.mp4
./ubcNbili_data/video_dwpose/BV11x4y177hd_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1tj411j7Ff_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1iW4y1Y7HD_segment_10.mp4
./ubcNbili_data/video_dwpose/BV1iv411q7EL_segment_05.mp4
./ubcNbili_data/video_dwpose/BV13x421273d_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1aU4y1a7ns_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1s14y1u7GZ_segment_10.mp4
./ubcNbili_data/video_dwpose/BV1tY4y1E7hU_segment_07.mp4
./ubcNbili_data/video_dwpose/BV1jp4y1u7uj_segment_00_part_2_cut.mp4
./ubcNbili_data/video_dwpose/BV11e411i7hr_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1Dm4y1X7JT_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1Xm411f7mV_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1AN4y1s7h3_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1ED4y157AR_segment_02_part_1_cut.mp4
./ubcNbili_data/video_dwpose/BV1Lv4y1P7jy_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Nh4y1177f_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1pV4y1m7Am_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1rA4m137wf_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1Ew411g7MB_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1mx4y1f7eT_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1CY4y1M7Bj_segment_14.mp4
./ubcNbili_data/video_dwpose/BV1LK421y7kY_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1ot4y1Z715_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1gm4y1x7tg_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1cQ4y1F74b_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1aF4m157ZE_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1q6421u79H_segment_03.mp4
./ubcNbili_data/video_dwpose/BV16F4m1j7EV_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1c2421u7Ra_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1j24y1k7Pj_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Ze4y1D7Es_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1Gu4y1e72H_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1LK411Y7Nx_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1EY411g7Yd_segment_23.mp4
./ubcNbili_data/video_dwpose/BV1Dm4y1X7JT_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1YC4y1r7EH_segment_00.mp4
./ubcNbili_data/video_dwpose/BV125411e7Mq_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1MU4y1j7kN_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1Ht421t7u2_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1d34y1c7Vg_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1YF411v7GN_segment_10.mp4
./ubcNbili_data/video_dwpose/BV1aH4y177tp_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1zj411L7v9_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Ff4y1L7R7_segment_12.mp4
./ubcNbili_data/video_dwpose/BV1Dm4y1X7JT_segment_14.mp4
./ubcNbili_data/video_dwpose/BV1ny4y1F7qV_segment_02_part_1_cut.mp4
./ubcNbili_data/video_dwpose/BV1YF411v7GN_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1y5411o796_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1bj411V7zk_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1Dm4y1X7JT_segment_37.mp4
./ubcNbili_data/video_dwpose/BV1g94y1A7bg_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1UY4y1J7Up_segment_03_part_1_cut.mp4
./ubcNbili_data/video_dwpose/BV1Lf4y1G74q_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1jU4y187gq_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1fC4y1j7Hh_segment_00_part_2_cut.mp4
./ubcNbili_data/video_dwpose/BV1nm4y1Q7PU_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1LN411E7ff_segment_12.mp4
./ubcNbili_data/video_dwpose/BV1uZ421U7Eq_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1PS421T7Xe_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1cQ4y1p78p_segment_11.mp4
./ubcNbili_data/video_dwpose/BV1b841117TP_segment_04_part_1_cut.mp4
./ubcNbili_data/video_dwpose/BV1Ht421t7u2_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1qt4y187ZE_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1Dm4y1X7JT_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1BG41117sV_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1zQ4y1c77h_segment_01.mp4
./ubcNbili_data/video_dwpose/BV19A411J7YL_segment_13.mp4
./ubcNbili_data/video_dwpose/BV1kD4y1U7gB_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1EK411u7e6_segment_07.mp4
./ubcNbili_data/video_dwpose/BV1qt4y187ZE_segment_10.mp4
./ubcNbili_data/video_dwpose/BV1eH4y177H9_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1R3411A7g4_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1Q94y1V7ny_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1K84y1L7UU_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1i64y1w7SF_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1TZ4y1c7Te_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1cE411j7RL_segment_03.mp4
./ubcNbili_data/video_dwpose/BV12j411n768_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1Nr421s7mx_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1br4y1F7mr_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Pf4y177j7_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1T54111779_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1vF4m177jX_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1Ff4y1L7R7_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1ub4y1Q7kw_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1Ex4y1C7jD_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1qw411K7SF_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1EY411g7Yd_segment_16.mp4
./ubcNbili_data/video_dwpose/BV1sb4y1W748_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1U44y1678R_segment_04.mp4
./ubcNbili_data/video_dwpose/BV11x4y177hd_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1eJ4m1W7k4_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1b44y1F7BA_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1Wp4y1m7AY_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Co4y127MF_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1uV4y1x7RU_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1i3411J72N_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Th411w7Dn_segment_00.mp4
./ubcNbili_data/video_dwpose/BV12C411H776_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1Cs4y1X7Rg_segment_08.mp4
./ubcNbili_data/video_dwpose/BV19F411v7oz_segment_05.mp4
./ubcNbili_data/video_dwpose/BV11e411i7hr_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1va4y1m7LF_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1Nb411S7bR_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1ur4y1r7pU_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1cQ4y1F74b_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1Gj411y7vz_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1b44y1F7BA_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1Qu4m1c786_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1vQ4y177bt_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1V94y1i7NX_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1FD4y1w7Pi_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1PS421T7Xe_segment_00.mp4
./ubcNbili_data/video_dwpose/BV14V4y1Y78o_segment_02.mp4
./ubcNbili_data/video_dwpose/BV16F4m1j7EV_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1cQ4y1p78p_segment_10.mp4
./ubcNbili_data/video_dwpose/BV1LQ4y1W7x3_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1ZN411J7CN_segment_07.mp4
./ubcNbili_data/video_dwpose/BV19F411v7oz_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1Ff4y1L7R7_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1UC4y1M72W_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1s14y1u7GZ_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Ex4y1C7jD_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1kx421Z7ev_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1X8411B7v6_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1tF411M7i6_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1nM4y1p7YP_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1E24y1M7rt_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1Zi4y1B7UZ_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1mx4y1f7eT_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1Gj411y7vz_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1V34y1u7DX_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1sK411e7zk_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1L64y1n7MQ_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1fV411D7Es_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Gu4y1e72H_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1BZ421a7Zq_segment_00_part_2_cut.mp4
./ubcNbili_data/video_dwpose/BV1ti421d7sn_segment_02.mp4
./ubcNbili_data/video_dwpose/BV14B4y1G7nT_segment_11.mp4
./ubcNbili_data/video_dwpose/BV1YC4y1r7EH_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1eJ4m1W7k4_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1oP41127zM_segment_04.mp4
./ubcNbili_data/video_dwpose/BV14A41137wj_segment_09.mp4
./ubcNbili_data/video_dwpose/BV12d4y1Y7WB_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1YF411z7Mc_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1GK421k7jh_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1gy4y1U7Nr_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1mB4y1G7dX_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1pV4y1m7Am_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1HJ4m1471Q_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1YF411v7GN_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1nQ4y1X7L2_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1ir4y1K77w_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1M24y1d7Qc_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1ak4y1t7bv_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1QM411q79y_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1mw411Q76N_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1wZ421272g_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1bY4y1S7Fc_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1ot4y1Z715_segment_01.mp4
./ubcNbili_data/video_dwpose/BV11f4y1q7ns_segment_07.mp4
./ubcNbili_data/video_dwpose/BV1Lx4y127B1_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1h34y1e7h4_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1zo4y1x77S_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1XA4m1372X_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1br4y1F7mr_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1sp4y1L7rV_segment_06.mp4
./ubcNbili_data/video_dwpose/BV19A411J7YL_segment_18.mp4
./ubcNbili_data/video_dwpose/BV1eJ4m1W7k4_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1vu4m1P7yr_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1aU4y1a7ns_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1Tt411c7wy_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1yt4y1o7Jm_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1iW4y1Y7HD_segment_07.mp4
./ubcNbili_data/video_dwpose/BV1kD4y1U7gB_segment_04.mp4
./ubcNbili_data/video_dwpose/BV12H4y1L7Qf_segment_05.mp4
./ubcNbili_data/video_dwpose/BV19Q4y1z7kh_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1ac411S7id_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1LN411E7ff_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1Ct42157LT_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Z94y1M7WP_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1Lx4y127B1_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1E24y1M7rt_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1Vc411x7o3_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1e2421F7Vd_segment_02.mp4
./ubcNbili_data/video_dwpose/BV12H4y1L7Qf_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1LY4y1k7J7_segment_02.mp4
./ubcNbili_data/video_dwpose/BV18g4y1C7hc_segment_04.mp4
./ubcNbili_data/video_dwpose/BV14A41137wj_segment_05.mp4
./ubcNbili_data/video_dwpose/BV1ZN411J7CN_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1Gu4y1e72H_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1MM411Z7yZ_segment_02.mp4
./ubcNbili_data/video_dwpose/BV1sK411e7zk_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1Pa411K73e_segment_01.mp4
./ubcNbili_data/video_dwpose/BV14B4y1G7nT_segment_07.mp4
./ubcNbili_data/video_dwpose/BV1sW4y1m755_segment_01.mp4
./ubcNbili_data/video_dwpose/BV1kD4y1U7gB_segment_02.mp4
./ubcNbili_data/video_dwpose/BV19A411J7YL_segment_19.mp4
./ubcNbili_data/video_dwpose/BV1sp421o7rK_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1Pg4y1k77T_segment_08.mp4
./ubcNbili_data/video_dwpose/BV1G44y1V7KZ_segment_03.mp4
./ubcNbili_data/video_dwpose/BV19u4y1t7kv_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1mC4y1A7gt_segment_06.mp4
./ubcNbili_data/video_dwpose/BV1i3411J72N_segment_04.mp4
./ubcNbili_data/video_dwpose/BV1Ja4y1D7z6_segment_03.mp4
./ubcNbili_data/video_dwpose/BV1BN411u7YK_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1Zu4y137eF_segment_09.mp4
./ubcNbili_data/video_dwpose/BV1vr4y1m78d_segment_12.mp4
./ubcNbili_data/video_dwpose/BV19A411J7YL_segment_04.mp4
./ubcNbili_data/video_dwpose/BV11e411i7hr_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1Fp421o7Go_segment_00.mp4
./ubcNbili_data/video_dwpose/BV1ua411X7aW_segment_02.mp4
./ubcNbili_data/video_dwpose/91QFEra7jDS.mp4
./ubcNbili_data/video_dwpose/A15GWy1GbkS.mp4
./ubcNbili_data/video_dwpose/A18fOhsmJWS.mp4
./ubcNbili_data/video_dwpose/91oPpgf2K+S.mp4
./ubcNbili_data/video_dwpose/A11fCqkXWUS.mp4
./ubcNbili_data/video_dwpose/91M3UFSKn8S.mp4
./ubcNbili_data/video_dwpose/91i7wHVECAS.mp4
./ubcNbili_data/video_dwpose/A1W9PeerqUS.mp4
./ubcNbili_data/video_dwpose/91vXwMBQScS.mp4
./ubcNbili_data/video_dwpose/A1xGg3ZcFvS.mp4
./ubcNbili_data/video_dwpose/91GoF2Z9ryS.mp4
./ubcNbili_data/video_dwpose/A1WD56t39zS.mp4
./ubcNbili_data/video_dwpose/91X-fpTnQjS.mp4
./ubcNbili_data/video_dwpose/A1UJwaPJfqS.mp4
./ubcNbili_data/video_dwpose/A1va+qFspPS.mp4
./ubcNbili_data/video_dwpose/A1eAt1p54XS.mp4
./ubcNbili_data/video_dwpose/A196PE754WS.mp4
./ubcNbili_data/video_dwpose/A1mkjLY5A7S.mp4
./ubcNbili_data/video_dwpose/91AxNIzSlaS.mp4
./ubcNbili_data/video_dwpose/A1ZO5aMyXVS.mp4
./ubcNbili_data/video_dwpose/A1hlRP7BkZS.mp4
./ubcNbili_data/video_dwpose/A1ov09nlukS.mp4
./ubcNbili_data/video_dwpose/91-2Jb8DkfS.mp4
./ubcNbili_data/video_dwpose/A1VF4yftLZS.mp4
./ubcNbili_data/video_dwpose/A1XmzqScZLS.mp4
./ubcNbili_data/video_dwpose/91KyQhcAjmS.mp4
./ubcNbili_data/video_dwpose/A1PyMMv4i2S.mp4
./ubcNbili_data/video_dwpose/A1Gvc5tOblS.mp4
./ubcNbili_data/video_dwpose/910yQoWQpFS.mp4
./ubcNbili_data/video_dwpose/A1eiU3QtJZS.mp4
./ubcNbili_data/video_dwpose/A17Q4CoaY7S.mp4
./ubcNbili_data/video_dwpose/91uhnh+Z5rS.mp4
./ubcNbili_data/video_dwpose/91CHD5t6lkS.mp4
./ubcNbili_data/video_dwpose/91yKWACUNcS.mp4
./ubcNbili_data/video_dwpose/A1U3C2fmz4S.mp4
./ubcNbili_data/video_dwpose/917B4f44vhS.mp4
./ubcNbili_data/video_dwpose/A1QvutYxzXS.mp4
./ubcNbili_data/video_dwpose/A1lbfqR4BBS.mp4
./ubcNbili_data/video_dwpose/912b+e6TrKS.mp4
./ubcNbili_data/video_dwpose/A1kLuw+NNxS.mp4
./ubcNbili_data/video_dwpose/9194wrYnsuS.mp4
./ubcNbili_data/video_dwpose/91bxJwvqyzS.mp4
./ubcNbili_data/video_dwpose/A1JjifG3FwS.mp4
./ubcNbili_data/video_dwpose/A1AjDdqufJS.mp4
./ubcNbili_data/video_dwpose/A1ERataKU1S.mp4
./ubcNbili_data/video_dwpose/91t1iUyA-DS.mp4
./ubcNbili_data/video_dwpose/A11wqf-ILMS.mp4
./ubcNbili_data/video_dwpose/A1ZoOtffkDS.mp4
./ubcNbili_data/video_dwpose/917zWmTGVMS.mp4
./ubcNbili_data/video_dwpose/A14gz1dCU2S.mp4
./ubcNbili_data/video_dwpose/A1C4Gaa9q-S.mp4
./ubcNbili_data/video_dwpose/A15H3k0MRRS.mp4
./ubcNbili_data/video_dwpose/91iLyRqupFS.mp4
./ubcNbili_data/video_dwpose/91cC+1+C4SS.mp4
./ubcNbili_data/video_dwpose/A14qP36SAnS.mp4
./ubcNbili_data/video_dwpose/A1AaQORgLfS.mp4
./ubcNbili_data/video_dwpose/A1UjKO6+U6S.mp4
./ubcNbili_data/video_dwpose/A1lIw0RlrnS.mp4
./ubcNbili_data/video_dwpose/91DU41k+QfS.mp4
./ubcNbili_data/video_dwpose/91iEHRn6ZGS.mp4
./ubcNbili_data/video_dwpose/A1N1qLQou6S.mp4
./ubcNbili_data/video_dwpose/91sNkqpxajS.mp4
./ubcNbili_data/video_dwpose/91svRFuREKS.mp4
./ubcNbili_data/video_dwpose/91ZSSH2D-xS.mp4
./ubcNbili_data/video_dwpose/91FvbEl4WSS.mp4
./ubcNbili_data/video_dwpose/A17RfvegkdS.mp4
./ubcNbili_data/video_dwpose/A1FN3aNHu4S.mp4
./ubcNbili_data/video_dwpose/A1hkvS7COFS.mp4
./ubcNbili_data/video_dwpose/A1PY+v5m7oS.mp4
./ubcNbili_data/video_dwpose/91VUAXoMGxS.mp4
./ubcNbili_data/video_dwpose/91hVdrwKj8S.mp4
./ubcNbili_data/video_dwpose/91UN-4ypVfS.mp4
./ubcNbili_data/video_dwpose/91WvLcNpdzS.mp4
./ubcNbili_data/video_dwpose/91xHpayzM0S.mp4
./ubcNbili_data/video_dwpose/91iSAR-kQlS.mp4
./ubcNbili_data/video_dwpose/A1GJweFGAiS.mp4
./ubcNbili_data/video_dwpose/91+fUG+fyBS.mp4
./ubcNbili_data/video_dwpose/9129YDE+cmS.mp4
./ubcNbili_data/video_dwpose/91w13yjb8MS.mp4
./ubcNbili_data/video_dwpose/A1sinewFaDS.mp4
./ubcNbili_data/video_dwpose/A15g0ekJ1US.mp4
./ubcNbili_data/video_dwpose/A1ahIWv8SpS.mp4
./ubcNbili_data/video_dwpose/A1pDSUD-6kS.mp4
./ubcNbili_data/video_dwpose/A1W-zcVU1JS.mp4
./ubcNbili_data/video_dwpose/918uApIJYCS.mp4
./ubcNbili_data/video_dwpose/91Zc93fnXBS.mp4
./ubcNbili_data/video_dwpose/91L8FIoMnTS.mp4
./ubcNbili_data/video_dwpose/91JoGG4jzLS.mp4
./ubcNbili_data/video_dwpose/91j9x55ormS.mp4
./ubcNbili_data/video_dwpose/91+PxmDyrgS.mp4
./ubcNbili_data/video_dwpose/A1eEZvfJRUS.mp4
./ubcNbili_data/video_dwpose/A1o4gxGfr6S.mp4
./ubcNbili_data/video_dwpose/A1E-bnKT5iS.mp4
./ubcNbili_data/video_dwpose/91KbTM-Qs5S.mp4
./ubcNbili_data/video_dwpose/A14+p6E20BS.mp4
./ubcNbili_data/video_dwpose/A1xphfMiftS.mp4
./ubcNbili_data/video_dwpose/91sUrTdJbTS.mp4
./ubcNbili_data/video_dwpose/91yqh2WjK1S.mp4
./ubcNbili_data/video_dwpose/A1nR9+HvtBS.mp4
./ubcNbili_data/video_dwpose/A1nYlrP3+zS.mp4
./ubcNbili_data/video_dwpose/A1PirhB4wES.mp4
./ubcNbili_data/video_dwpose/A1sE2aFAZDS.mp4
./ubcNbili_data/video_dwpose/A17zkSVwKES.mp4
./ubcNbili_data/video_dwpose/91BeC-Eo4+S.mp4
./ubcNbili_data/video_dwpose/A1gHv2OcraS.mp4
./ubcNbili_data/video_dwpose/91zrX+VihKS.mp4
./ubcNbili_data/video_dwpose/A1lUNAejWgS.mp4
./ubcNbili_data/video_dwpose/91tvKOw3HqS.mp4
./ubcNbili_data/video_dwpose/A1-T14PwQPS.mp4
./ubcNbili_data/video_dwpose/A1vtrDlFZYS.mp4
./ubcNbili_data/video_dwpose/A1F1j+kNaDS.mp4
./ubcNbili_data/video_dwpose/A1ETn-UX+MS.mp4
./ubcNbili_data/video_dwpose/A1o+Q8NgFnS.mp4
./ubcNbili_data/video_dwpose/91ugqUYBFYS.mp4
./ubcNbili_data/video_dwpose/A1reZkUWSVS.mp4
./ubcNbili_data/video_dwpose/A1PldjC8veS.mp4
./ubcNbili_data/video_dwpose/A1E+sxSfPQS.mp4
./ubcNbili_data/video_dwpose/91mxkmTsn-S.mp4
./ubcNbili_data/video_dwpose/A1LI3q12smS.mp4
./ubcNbili_data/video_dwpose/A149qryhUwS.mp4
./ubcNbili_data/video_dwpose/91WIsDoNP2S.mp4
./ubcNbili_data/video_dwpose/A1SitGflSMS.mp4
./ubcNbili_data/video_dwpose/91PPEg-KXES.mp4
./ubcNbili_data/video_dwpose/A1fd4ZDyhwS.mp4
./ubcNbili_data/video_dwpose/A1EpEaUAOpS.mp4
./ubcNbili_data/video_dwpose/A1Yh-MNHbrS.mp4
./ubcNbili_data/video_dwpose/A1Z+OiecmLS.mp4
./ubcNbili_data/video_dwpose/A1hQq1tJyZS.mp4
./ubcNbili_data/video_dwpose/91Sc09ZXIWS.mp4
./ubcNbili_data/video_dwpose/A1EGxWmdmOS.mp4
./ubcNbili_data/video_dwpose/91iZ9x8NI0S.mp4
./ubcNbili_data/video_dwpose/91Wt9hr+93S.mp4
./ubcNbili_data/video_dwpose/A1nbMxob4VS.mp4
./ubcNbili_data/video_dwpose/911HPTHcpRS.mp4
./ubcNbili_data/video_dwpose/91S-OYN7chS.mp4
./ubcNbili_data/video_dwpose/91LVKVKed0S.mp4
./ubcNbili_data/video_dwpose/91UZGkoxFFS.mp4
./ubcNbili_data/video_dwpose/A15Tpiz-itS.mp4
./ubcNbili_data/video_dwpose/91dqdbuZPhS.mp4
./ubcNbili_data/video_dwpose/A1AhksKfjQS.mp4
./ubcNbili_data/video_dwpose/A1kOYTO9sRS.mp4
./ubcNbili_data/video_dwpose/91Rff5U3uHS.mp4
./ubcNbili_data/video_dwpose/91U6zCbi81S.mp4
./ubcNbili_data/video_dwpose/A1gga724uoS.mp4
./ubcNbili_data/video_dwpose/91FQY9XTyQS.mp4
./ubcNbili_data/video_dwpose/91d1Gs7pDqS.mp4
./ubcNbili_data/video_dwpose/91awragicLS.mp4
./ubcNbili_data/video_dwpose/A16dhGRUkuS.mp4
./ubcNbili_data/video_dwpose/91mOBF1cAJS.mp4
./ubcNbili_data/video_dwpose/91IYZzsL6oS.mp4
./ubcNbili_data/video_dwpose/A1A1OMtYuiS.mp4
./ubcNbili_data/video_dwpose/91C01Mb7OBS.mp4
./ubcNbili_data/video_dwpose/A1gVq1bGsKS.mp4
./ubcNbili_data/video_dwpose/91CO316EfOS.mp4
./ubcNbili_data/video_dwpose/A1TTCvb9eqS.mp4
./ubcNbili_data/video_dwpose/91Lvh4YdSUS.mp4
./ubcNbili_data/video_dwpose/A1s7vPmyheS.mp4
./ubcNbili_data/video_dwpose/A1nKGW2--gS.mp4
./ubcNbili_data/video_dwpose/91lrKy4o2MS.mp4
./ubcNbili_data/video_dwpose/A1Qf04-ORaS.mp4
./ubcNbili_data/video_dwpose/A1K2E+QDwJS.mp4
./ubcNbili_data/video_dwpose/91elFllk8oS.mp4
./ubcNbili_data/video_dwpose/91gIhZ2S4SS.mp4
./ubcNbili_data/video_dwpose/91qaGfTESXS.mp4
./ubcNbili_data/video_dwpose/91xuWi7D5kS.mp4
./ubcNbili_data/video_dwpose/A1fta5rGYwS.mp4
./ubcNbili_data/video_dwpose/A1l3RNefCZS.mp4
./ubcNbili_data/video_dwpose/A1+O5s8cUuS.mp4
./ubcNbili_data/video_dwpose/A1GOx7I3kfS.mp4
./ubcNbili_data/video_dwpose/914YKv4LeXS.mp4
./ubcNbili_data/video_dwpose/A1u5vhJKxtS.mp4
./ubcNbili_data/video_dwpose/A1PvgDCryLS.mp4
./ubcNbili_data/video_dwpose/91gayUSzkqS.mp4
./ubcNbili_data/video_dwpose/91+bCFG1jOS.mp4
./ubcNbili_data/video_dwpose/91xENK41GyS.mp4
./ubcNbili_data/video_dwpose/A1T-Ea-FlQS.mp4
./ubcNbili_data/video_dwpose/912Qm4qFZ5S.mp4
./ubcNbili_data/video_dwpose/A1P+0ZhMAiS.mp4
./ubcNbili_data/video_dwpose/A1ESU8dpYhS.mp4
./ubcNbili_data/video_dwpose/A1sqF9lNpqS.mp4
./ubcNbili_data/video_dwpose/91QfJhDwuxS.mp4
./ubcNbili_data/video_dwpose/A1TfaqtPn1S.mp4
./ubcNbili_data/video_dwpose/A1AcsqCBqyS.mp4
./ubcNbili_data/video_dwpose/A1-ZzOdQ9tS.mp4
./ubcNbili_data/video_dwpose/91XQ7iTrA3S.mp4
./ubcNbili_data/video_dwpose/A1MPiBwZuIS.mp4
./ubcNbili_data/video_dwpose/91lIYuUETHS.mp4
./ubcNbili_data/video_dwpose/A1tDo5PqA9S.mp4
./ubcNbili_data/video_dwpose/A18GIeJe6RS.mp4
./ubcNbili_data/video_dwpose/A1-cVlkGwjS.mp4
./ubcNbili_data/video_dwpose/91+PxYkdSaS.mp4
./ubcNbili_data/video_dwpose/A1w603Ci8KS.mp4
./ubcNbili_data/video_dwpose/A1ZKroqr8eS.mp4
./ubcNbili_data/video_dwpose/91P6l1xYVQS.mp4
./ubcNbili_data/video_dwpose/A1uwPz7+srS.mp4
./ubcNbili_data/video_dwpose/A1vvFYsp4PS.mp4
./ubcNbili_data/video_dwpose/A1Yin2If81S.mp4
./ubcNbili_data/video_dwpose/91yreZAG8CS.mp4
./ubcNbili_data/video_dwpose/91bTyHxr3KS.mp4
./ubcNbili_data/video_dwpose/91Gi3oNo0ZS.mp4
./ubcNbili_data/video_dwpose/91Ile3zLhMS.mp4
./ubcNbili_data/video_dwpose/A1klQ-odb4S.mp4
./ubcNbili_data/video_dwpose/A1aUOWf3EES.mp4
./ubcNbili_data/video_dwpose/A1cks+TzPgS.mp4
./ubcNbili_data/video_dwpose/91CUFOb58AS.mp4
./ubcNbili_data/video_dwpose/91rCwmuvuIS.mp4
./ubcNbili_data/video_dwpose/91T6U5fAejS.mp4
./ubcNbili_data/video_dwpose/A18SZEmAPmS.mp4
./ubcNbili_data/video_dwpose/A1bM5kktW7S.mp4
./ubcNbili_data/video_dwpose/917UaHxvTZS.mp4
./ubcNbili_data/video_dwpose/91iKyqnNwxS.mp4
./ubcNbili_data/video_dwpose/91NMtb9JuES.mp4
./ubcNbili_data/video_dwpose/91-kqBbzDIS.mp4
./ubcNbili_data/video_dwpose/91gY8y3phjS.mp4
./ubcNbili_data/video_dwpose/A15nWd4JYgS.mp4
./ubcNbili_data/video_dwpose/81FyMPk-WIS.mp4
./ubcNbili_data/video_dwpose/9167xfvs0RS.mp4
./ubcNbili_data/video_dwpose/A1fjrQMJMAS.mp4
./ubcNbili_data/video_dwpose/A1Bc1P0TdqS.mp4
./ubcNbili_data/video_dwpose/A12qCU8nimS.mp4
./ubcNbili_data/video_dwpose/A16G49pfsKS.mp4
./ubcNbili_data/video_dwpose/91fxYsir49S.mp4
./ubcNbili_data/video_dwpose/A1Y4RROg3yS.mp4
./ubcNbili_data/video_dwpose/91hECpvrRSS.mp4
./ubcNbili_data/video_dwpose/91pn9FUZ9NS.mp4
./ubcNbili_data/video_dwpose/91C+rpudhdS.mp4
./ubcNbili_data/video_dwpose/91ouZtTxM6S.mp4
./ubcNbili_data/video_dwpose/91USaiAdMdS.mp4
./ubcNbili_data/video_dwpose/91aHjkHkSQS.mp4
./ubcNbili_data/video_dwpose/91SOigLcQkS.mp4
./ubcNbili_data/video_dwpose/91dVyEFeFXS.mp4
./ubcNbili_data/video_dwpose/A17AGFxllwS.mp4
./ubcNbili_data/video_dwpose/91zpcutg68S.mp4
./ubcNbili_data/video_dwpose/91sHkbnXtmS.mp4
./ubcNbili_data/video_dwpose/91K7loBr8RS.mp4
./ubcNbili_data/video_dwpose/91NUBMaZ0MS.mp4
./ubcNbili_data/video_dwpose/91HzMhq7eOS.mp4
./ubcNbili_data/video_dwpose/91+lbQkcx5S.mp4
./ubcNbili_data/video_dwpose/91uY9usoa5S.mp4
./ubcNbili_data/video_dwpose/91CBf2PucRS.mp4
./ubcNbili_data/video_dwpose/912mf320DtS.mp4
./ubcNbili_data/video_dwpose/A1phWCkHzQS.mp4
./ubcNbili_data/video_dwpose/91Ucjr9DLYS.mp4
./ubcNbili_data/video_dwpose/91ODIc-gMDS.mp4
./ubcNbili_data/video_dwpose/A1BokHoyj1S.mp4
./ubcNbili_data/video_dwpose/A1Cjr6BQW1S.mp4
./ubcNbili_data/video_dwpose/A15VEzukScS.mp4
./ubcNbili_data/video_dwpose/A1qbiuHTZCS.mp4
./ubcNbili_data/video_dwpose/A135+VkA+9S.mp4
./ubcNbili_data/video_dwpose/A15F0+kY8rS.mp4
./ubcNbili_data/video_dwpose/A14e-5v80zS.mp4
./ubcNbili_data/video_dwpose/A1372XbmjQS.mp4
./ubcNbili_data/video_dwpose/A15KmK+PapS.mp4
./ubcNbili_data/video_dwpose/A190M-8Kh8S.mp4
./ubcNbili_data/video_dwpose/91inLIhjlkS.mp4
./ubcNbili_data/video_dwpose/A1uNC8clQGS.mp4
./ubcNbili_data/video_dwpose/A14LlIk0zZS.mp4
./ubcNbili_data/video_dwpose/A1dLq8J8cjS.mp4
./ubcNbili_data/video_dwpose/A1NPZwUqfaS.mp4
./ubcNbili_data/video_dwpose/91O0DKws+gS.mp4
./ubcNbili_data/video_dwpose/91bit+2uAGS.mp4
./ubcNbili_data/video_dwpose/A1rxI8K31eS.mp4
./ubcNbili_data/video_dwpose/91yJtf-8RBS.mp4
./ubcNbili_data/video_dwpose/A1A0BcAeG0S.mp4
./ubcNbili_data/video_dwpose/A1ubDo0PbQS.mp4
./ubcNbili_data/video_dwpose/91+xeI+ijRS.mp4
./ubcNbili_data/video_dwpose/91LR1LCcc1S.mp4
./ubcNbili_data/video_dwpose/91B1xPF+UGS.mp4
./ubcNbili_data/video_dwpose/A1fa0B8PGqS.mp4
./ubcNbili_data/video_dwpose/A1scLENhiYS.mp4
./ubcNbili_data/video_dwpose/A1o2X2SZJ8S.mp4
./ubcNbili_data/video_dwpose/A15HjAPmcrS.mp4
./ubcNbili_data/video_dwpose/91nLbWw4ezS.mp4
./ubcNbili_data/video_dwpose/A1Ir3wQYrRS.mp4
./ubcNbili_data/video_dwpose/91lkERFbNzS.mp4
./ubcNbili_data/video_dwpose/91ozYADVwYS.mp4
./ubcNbili_data/video_dwpose/A1kcvy7vUXS.mp4
./ubcNbili_data/video_dwpose/A191LVe3AzS.mp4
./ubcNbili_data/video_dwpose/A1u1X1B3bWS.mp4
./ubcNbili_data/video_dwpose/91NuxtjJ8sS.mp4
./ubcNbili_data/video_dwpose/A1wwPTTzVGS.mp4
./ubcNbili_data/video_dwpose/91I3ReFlwmS.mp4
./ubcNbili_data/video_dwpose/91rr7Fj3wmS.mp4
./ubcNbili_data/video_dwpose/91PbGctNl8S.mp4
./ubcNbili_data/video_dwpose/A1h5pleQRdS.mp4
./ubcNbili_data/video_dwpose/91+uwOT1POS.mp4
./ubcNbili_data/video_dwpose/91EWdk0xgDS.mp4
./ubcNbili_data/video_dwpose/A1qmerAHT-S.mp4
./ubcNbili_data/video_dwpose/91bMH7SeuoS.mp4
./ubcNbili_data/video_dwpose/A1HoF0l4NrS.mp4
./ubcNbili_data/video_dwpose/91cqgHAeFJS.mp4
./ubcNbili_data/video_dwpose/A1ca1mgsMUS.mp4
./ubcNbili_data/video_dwpose/A1Pd+czuQ5S.mp4
./ubcNbili_data/video_dwpose/91Ghs8ro+NS.mp4
./ubcNbili_data/video_dwpose/915AFYiy5HS.mp4
./ubcNbili_data/video_dwpose/A1S0xsQfaiS.mp4
./ubcNbili_data/video_dwpose/A1-Lv00GAzS.mp4
./ubcNbili_data/video_dwpose/A1WY8jED6NS.mp4
./ubcNbili_data/video_dwpose/A1FYkMYsPAS.mp4
./ubcNbili_data/video_dwpose/91mDd2Kmd6S.mp4
./ubcNbili_data/video_dwpose/A1e4FvDCO0S.mp4
./ubcNbili_data/video_dwpose/91mJs1Rx9ZS.mp4
./ubcNbili_data/video_dwpose/A1R2IbUSf5S.mp4
./ubcNbili_data/video_dwpose/91lel7JgImS.mp4
./ubcNbili_data/video_dwpose/91jKbe9-y4S.mp4
./ubcNbili_data/video_dwpose/A1NoOlWA5zS.mp4
./ubcNbili_data/video_dwpose/A1E5EPKP+GS.mp4
./ubcNbili_data/video_dwpose/91D23ZVV6NS.mp4
./ubcNbili_data/video_dwpose/91QI9OO73uS.mp4
./ubcNbili_data/video_dwpose/A1b0a5leqoS.mp4
./ubcNbili_data/video_dwpose/A1f94LlYyIS.mp4
./ubcNbili_data/video_dwpose/91wac4EX5NS.mp4
./ubcNbili_data/video_dwpose/91p9RBHG-qS.mp4
./ubcNbili_data/video_dwpose/91MTHBRuoeS.mp4
./ubcNbili_data/video_dwpose/A1cBV6jd+2S.mp4
./ubcNbili_data/video_dwpose/A1kBIdnD65S.mp4
./ubcNbili_data/video_dwpose/A14+vjk00kS.mp4
./ubcNbili_data/video_dwpose/A1mg7pn3zLS.mp4
./ubcNbili_data/video_dwpose/A16dzLO+diS.mp4
./ubcNbili_data/video_dwpose/91l9zhxnd-S.mp4
./ubcNbili_data/video_dwpose/A1TkcPfqMYS.mp4
./ubcNbili_data/video_dwpose/A1aF4I7s-hS.mp4
./ubcNbili_data/video_dwpose/91-NTYmAx2S.mp4
./ubcNbili_data/video_dwpose/91SkcqZbgHS.mp4
./ubcNbili_data/video_dwpose/910rxtT4lNS.mp4
./ubcNbili_data/video_dwpose/A10guZxkDfS.mp4
./ubcNbili_data/video_dwpose/91bxAN6BjAS.mp4
./ubcNbili_data/video_dwpose/91awBtSMraS.mp4
./ubcNbili_data/video_dwpose/91k62gM3CdS.mp4
./ubcNbili_data/video_dwpose/A129w3AF9lS.mp4
./ubcNbili_data/video_dwpose/A1q5q4JH2cS.mp4
./ubcNbili_data/video_dwpose/91SzoPJfbMS.mp4
./ubcNbili_data/video_dwpose/A1X+6J1KpJS.mp4
./ubcNbili_data/video_dwpose/A1w4zlJFPiS.mp4
./ubcNbili_data/video_dwpose/917v6EQuPJS.mp4
./ubcNbili_data/video_dwpose/91TinyLmtvS.mp4
./ubcNbili_data/video_dwpose/91Yw7J872gS.mp4
./ubcNbili_data/video_dwpose/91BjuE6irxS.mp4
./ubcNbili_data/video_dwpose/91jAkcMn9ES.mp4
./ubcNbili_data/video_dwpose/A1VAMY3TBhS.mp4
./ubcNbili_data/video_dwpose/A1as1eJFMZS.mp4
./ubcNbili_data/video_dwpose/A1zaFTtYy+S.mp4
./ubcNbili_data/video_dwpose/91VyWwPVZBS.mp4
./ubcNbili_data/video_dwpose/A1CGKqiQbKS.mp4
./ubcNbili_data/video_dwpose/91S0eS4UYDS.mp4
./ubcNbili_data/video_dwpose/91c+SL7Cg7S.mp4
./ubcNbili_data/video_dwpose/A16GrHceHTS.mp4
./ubcNbili_data/video_dwpose/A14QYvPah6S.mp4
./ubcNbili_data/video_dwpose/91hNaP-63aS.mp4
./ubcNbili_data/video_dwpose/A19FRhRmb-S.mp4
./ubcNbili_data/video_dwpose/A13qMfx+VdS.mp4
./ubcNbili_data/video_dwpose/91fdTDFn-0S.mp4
./ubcNbili_data/video_dwpose/91LxNkrQZVS.mp4
./ubcNbili_data/video_dwpose/A1LXiB9DErS.mp4
./ubcNbili_data/video_dwpose/A1IT+pEn0iS.mp4
./ubcNbili_data/video_dwpose/A15oY022hnS.mp4
./ubcNbili_data/video_dwpose/A1XHkvF+vvS.mp4
./ubcNbili_data/video_dwpose/91y4PIhedLS.mp4
./ubcNbili_data/video_dwpose/A1ZfjsfRsJS.mp4
./ubcNbili_data/video_dwpose/91rso8MLhNS.mp4
./ubcNbili_data/video_dwpose/91CYKqVIWKS.mp4
./ubcNbili_data/video_dwpose/91tr6rk41qS.mp4
./ubcNbili_data/video_dwpose/91Xg-11OuYS.mp4
./ubcNbili_data/video_dwpose/A12Bs0ylPrS.mp4
./ubcNbili_data/video_dwpose/91t9QPjy6IS.mp4
./ubcNbili_data/video_dwpose/91J6B8sbMuS.mp4
./ubcNbili_data/video_dwpose/91Hi+Nn+WQS.mp4
./ubcNbili_data/video_dwpose/A1Y8HsouVnS.mp4
./ubcNbili_data/video_dwpose/91ETNTpkRLS.mp4
./ubcNbili_data/video_dwpose/A1nolvbbFUS.mp4
./ubcNbili_data/video_dwpose/A1b1qjsklzS.mp4
./ubcNbili_data/video_dwpose/91LU1NHG2zS.mp4
./ubcNbili_data/video_dwpose/A1QjseidMkS.mp4
./ubcNbili_data/video_dwpose/91UzJdn3N+S.mp4
./ubcNbili_data/video_dwpose/A1MsFmxPg1S.mp4
./ubcNbili_data/video_dwpose/A1VKFP3456S.mp4
./ubcNbili_data/video_dwpose/91V2ZOx7L2S.mp4
./ubcNbili_data/video_dwpose/91GN31Z1rVS.mp4
./ubcNbili_data/video_dwpose/91hfgkIXQHS.mp4
./ubcNbili_data/video_dwpose/91-wIan2z6S.mp4
./ubcNbili_data/video_dwpose/91yr7XDTg6S.mp4
./ubcNbili_data/video_dwpose/A1-JigxCHVS.mp4
./ubcNbili_data/video_dwpose/91risc0HOhS.mp4
./ubcNbili_data/video_dwpose/A1f9DiUYpAS.mp4
./ubcNbili_data/video_dwpose/A1kmapmav2S.mp4
./ubcNbili_data/video_dwpose/A1AMAIOLUsS.mp4
./ubcNbili_data/video_dwpose/91SgMpTCl3S.mp4
./ubcNbili_data/video_dwpose/A1Cy4oQcwMS.mp4
./ubcNbili_data/video_dwpose/A1KcpmdyZcS.mp4
./ubcNbili_data/video_dwpose/91tZgBKfXRS.mp4
./ubcNbili_data/video_dwpose/A1H-rni1GwS.mp4
./ubcNbili_data/video_dwpose/91FQFmPfsPS.mp4
./ubcNbili_data/video_dwpose/A1jQBm9NGkS.mp4
./ubcNbili_data/video_dwpose/919t9Syv9mS.mp4
./ubcNbili_data/video_dwpose/91ZSnUjczES.mp4
./ubcNbili_data/video_dwpose/A1YsUDIAq3S.mp4
./ubcNbili_data/video_dwpose/91G0vO9z0KS.mp4
./ubcNbili_data/video_dwpose/91kau3zgnCS.mp4
./ubcNbili_data/video_dwpose/91jVgPE1s1S.mp4
./ubcNbili_data/video_dwpose/A1INOWKAOfS.mp4
./ubcNbili_data/video_dwpose/A1BjxrwYllS.mp4
./ubcNbili_data/video_dwpose/91Z2vahVOdS.mp4
./ubcNbili_data/video_dwpose/91Iegdp9HFS.mp4
./ubcNbili_data/video_dwpose/91w4r-YTBlS.mp4
./ubcNbili_data/video_dwpose/91Fl2rS9sAS.mp4
./ubcNbili_data/video_dwpose/91Om51OQRgS.mp4
./ubcNbili_data/video_dwpose/A16LhLE1PoS.mp4
./ubcNbili_data/video_dwpose/A1rkShWKy-S.mp4
./ubcNbili_data/video_dwpose/91VBdLPe6sS.mp4
./ubcNbili_data/video_dwpose/91a7ujDXN9S.mp4
./ubcNbili_data/video_dwpose/91yaNbzZb2S.mp4
./ubcNbili_data/video_dwpose/A134R3du4SS.mp4
./ubcNbili_data/video_dwpose/91v9t7spFnS.mp4
./ubcNbili_data/video_dwpose/911EyguDLjS.mp4
./ubcNbili_data/video_dwpose/916iz4hbIJS.mp4
./ubcNbili_data/video_dwpose/A1YwtL1+BqS.mp4
./ubcNbili_data/video_dwpose/A1gXF-c+0dS.mp4
./ubcNbili_data/video_dwpose/911dnmIdIYS.mp4
./ubcNbili_data/video_dwpose/91Jvr6TZf1S.mp4
./ubcNbili_data/video_dwpose/91Xs4WPjtTS.mp4
./ubcNbili_data/video_dwpose/91UveTFmA3S.mp4
./ubcNbili_data/video_dwpose/91xDLvKBQ8S.mp4
./ubcNbili_data/video_dwpose/A15HYlQRNUS.mp4
./ubcNbili_data/video_dwpose/A14lkhInQfS.mp4
./ubcNbili_data/video_dwpose/A15Ei5ve9BS.mp4
./ubcNbili_data/video_dwpose/91GeSv-KwSS.mp4
./ubcNbili_data/video_dwpose/91KzTEtLCcS.mp4
./ubcNbili_data/video_dwpose/A1bU-WYQgvS.mp4
./ubcNbili_data/video_dwpose/91jvQeBJovS.mp4
./ubcNbili_data/video_dwpose/A1ioBaIf3+S.mp4
./ubcNbili_data/video_dwpose/91zcRJwjygS.mp4
./ubcNbili_data/video_dwpose/A1P10LXQgGS.mp4
./ubcNbili_data/video_dwpose/A1B2qmziDvS.mp4
./ubcNbili_data/video_dwpose/A1s1Xh4xEtS.mp4
./ubcNbili_data/video_dwpose/91-3003CN5S.mp4
./ubcNbili_data/video_dwpose/A1trMn+DPCS.mp4
./ubcNbili_data/video_dwpose/A1-Y9IlsIAS.mp4
./ubcNbili_data/video_dwpose/A1-92I736ES.mp4
./ubcNbili_data/video_dwpose/91eqrZs3NOS.mp4
./ubcNbili_data/video_dwpose/91f66jHiIJS.mp4
./ubcNbili_data/video_dwpose/91wrpZZAuGS.mp4
./ubcNbili_data/video_dwpose/91fbGDJDEZS.mp4
./ubcNbili_data/video_dwpose/A1CyPGq5HcS.mp4
./ubcNbili_data/video_dwpose/A1F3EkKe2ES.mp4
./ubcNbili_data/video_dwpose/91smb5UEjsS.mp4
./ubcNbili_data/video_dwpose/A1+Ea25jPFS.mp4
./ubcNbili_data/video_dwpose/91kahoXLeSS.mp4
./ubcNbili_data/video_dwpose/91CL7aaNUQS.mp4
./ubcNbili_data/video_dwpose/A1W1FyxFN1S.mp4
./ubcNbili_data/video_dwpose/91syhDT3BfS.mp4
./ubcNbili_data/video_dwpose/91Z+ZFDkaLS.mp4
./ubcNbili_data/video_dwpose/A1TmEeuBxfS.mp4
./ubcNbili_data/video_dwpose/91dgMhmymOS.mp4
./ubcNbili_data/video_dwpose/91EfnBTEE2S.mp4
./ubcNbili_data/video_dwpose/91DccWQJovS.mp4
./ubcNbili_data/video_dwpose/91WwlYjebKS.mp4
./ubcNbili_data/video_dwpose/91id9oI6OXS.mp4
./ubcNbili_data/video_dwpose/91rduj3SEeS.mp4
./ubcNbili_data/video_dwpose/A1ObofRlFtS.mp4
./ubcNbili_data/video_dwpose/A1DKTgVSVaS.mp4
./ubcNbili_data/video_dwpose/A1K+AiMS9eS.mp4
./ubcNbili_data/video_dwpose/A1u0IlecjbS.mp4
./ubcNbili_data/video_dwpose/A10DsT3ew3S.mp4
./ubcNbili_data/video_dwpose/A1-6cHgMKxS.mp4
./ubcNbili_data/video_dwpose/A1p7F2+bYyS.mp4
./ubcNbili_data/video_dwpose/91wfTWcm+HS.mp4
./ubcNbili_data/video_dwpose/A14zSHJKopS.mp4
./ubcNbili_data/video_dwpose/A16PpDz4r2S.mp4
./ubcNbili_data/video_dwpose/91FxsyKa6cS.mp4
./ubcNbili_data/video_dwpose/A1L6XV+jeQS.mp4
./ubcNbili_data/video_dwpose/A1kmEeviTSS.mp4
./ubcNbili_data/video_dwpose/A1JAMLw3HHS.mp4
./ubcNbili_data/video_dwpose/A1maQZR5JNS.mp4
./ubcNbili_data/video_dwpose/91JGLZ4VuIS.mp4
./ubcNbili_data/video_dwpose/A1EjJwhDw-S.mp4
./ubcNbili_data/video_dwpose/916VcoojH6S.mp4
./ubcNbili_data/video_dwpose/91pUKQ9VE-S.mp4
./ubcNbili_data/video_dwpose/914LPAnGLuS.mp4
./ubcNbili_data/video_dwpose/91pO-0Xf8CS.mp4
./ubcNbili_data/video_dwpose/91e5Zx4EMDS.mp4
./ubcNbili_data/video_dwpose/A1fhGrcXjjS.mp4
./ubcNbili_data/video_dwpose/A1v-wPFDxBS.mp4
./ubcNbili_data/video_dwpose/91KN65tdl9S.mp4
./ubcNbili_data/video_dwpose/91HL0Dn7NzS.mp4
./ubcNbili_data/video_dwpose/A1YQG7hbxAS.mp4
./ubcNbili_data/video_dwpose/A1ZJTtuiBDS.mp4
./ubcNbili_data/video_dwpose/A1GHQG8KysS.mp4
./ubcNbili_data/video_dwpose/A14oLiUg7CS.mp4
./ubcNbili_data/video_dwpose/91He1e1jhwS.mp4
./ubcNbili_data/video_dwpose/91rbRfjMweS.mp4
./ubcNbili_data/video_dwpose/91z5omy4l9S.mp4
./ubcNbili_data/video_dwpose/A1wGG6QMSqS.mp4
./ubcNbili_data/video_dwpose/A1mKRBbECZS.mp4
./ubcNbili_data/video_dwpose/A1wrrhGRZmS.mp4
./ubcNbili_data/video_dwpose/911oR3cdb1S.mp4
./ubcNbili_data/video_dwpose/91nTMHxyitS.mp4
./ubcNbili_data/video_dwpose/A1dHQjHDEfS.mp4
./ubcNbili_data/video_dwpose/91ffZ+VFIKS.mp4
./ubcNbili_data/video_dwpose/A1vouAviXRS.mp4
./ubcNbili_data/video_dwpose/A1sJoS5NgBS.mp4
./ubcNbili_data/video_dwpose/91RAedLsgMS.mp4
./ubcNbili_data/video_dwpose/A1AMRLTiJGS.mp4
./ubcNbili_data/video_dwpose/91oFLFG8UNS.mp4
./ubcNbili_data/video_dwpose/91Un56FGzrS.mp4
./ubcNbili_data/video_dwpose/A1jBVJHhD5S.mp4
./ubcNbili_data/video_dwpose/A1VhNN47hUS.mp4
./ubcNbili_data/video_dwpose/A1rJWpLlTwS.mp4
./ubcNbili_data/video_dwpose/91UGeylvSYS.mp4
./ubcNbili_data/video_dwpose/A1s0+qM9T1S.mp4
./ubcNbili_data/video_dwpose/91sbGyAh4lS.mp4
./ubcNbili_data/video_dwpose/A127e+bWMvS.mp4
./ubcNbili_data/video_dwpose/91BxsiLn7uS.mp4
./ubcNbili_data/video_dwpose/A1MPQ2Fne+S.mp4
./ubcNbili_data/video_dwpose/A16SWkijSrS.mp4
./ubcNbili_data/video_dwpose/A11EAY9CR5S.mp4
./ubcNbili_data/video_dwpose/A1Oxt4PlX5S.mp4
./ubcNbili_data/video_dwpose/A13x1REw8VS.mp4
./ubcNbili_data/video_dwpose/91msz0VABdS.mp4
./ubcNbili_data/video_dwpose/A1+lct+7pqS.mp4
./ubcNbili_data/video_dwpose/91IiZsMxf9S.mp4
./ubcNbili_data/video_dwpose/91ierZSo5hS.mp4
./ubcNbili_data/video_dwpose/91xMHzpdyIS.mp4
./ubcNbili_data/video_dwpose/9191vM3gWUS.mp4
./ubcNbili_data/video_dwpose/A1YNmKj0sCS.mp4
./ubcNbili_data/video_dwpose/91uXUiMTuVS.mp4
./ubcNbili_data/video_dwpose/918l-RDCPAS.mp4
./ubcNbili_data/video_dwpose/A1p7n9-jHES.mp4
./ubcNbili_data/video_dwpose/A1b8hF5TgXS.mp4
./ubcNbili_data/video_dwpose/A1Wy60+1o4S.mp4
./ubcNbili_data/video_dwpose/91-BAOunZBS.mp4
./ubcNbili_data/video_dwpose/A158fYJR5gS.mp4
./ubcNbili_data/video_dwpose/919k+nrbR6S.mp4
./ubcNbili_data/video_dwpose/A1l1po6YF2S.mp4
./ubcNbili_data/video_dwpose/91EPHko9EMS.mp4
./ubcNbili_data/video_dwpose/A11unT+W2IS.mp4
./ubcNbili_data/video_dwpose/91k+vG5ThUS.mp4
./ubcNbili_data/video_dwpose/A1-zDL7VfyS.mp4
./ubcNbili_data/video_dwpose/A15mtgshxeS.mp4
./ubcNbili_data/video_dwpose/A1UhbV6ZA4S.mp4
./ubcNbili_data/video_dwpose/A1je3steNHS.mp4
./ubcNbili_data/video_dwpose/A1IPcs--FPS.mp4
./ubcNbili_data/video_dwpose/91+20mY7UJS.mp4
./ubcNbili_data/video_dwpose/91zX7m12qrS.mp4
./ubcNbili_data/video_dwpose/9193ZptS2NS.mp4
./ubcNbili_data/video_dwpose/A1AVN6EopvS.mp4
./ubcNbili_data/video_dwpose/A1uLhwCC4mS.mp4
./ubcNbili_data/video_dwpose/A1cG9GzVTHS.mp4
./ubcNbili_data/video_dwpose/A12XuDE2T3S.mp4
./ubcNbili_data/video_dwpose/91JfCp4ysES.mp4
./ubcNbili_data/video_dwpose/A1y4ALrThUS.mp4
./ubcNbili_data/video_dwpose/A1LCBvndW3S.mp4
./ubcNbili_data/video_dwpose/91mPG16px8S.mp4
./ubcNbili_data/video_dwpose/91+z7oAY-IS.mp4
./ubcNbili_data/video_dwpose/9174TjkrG2S.mp4
./ubcNbili_data/video_dwpose/A190oHyN7XS.mp4
./ubcNbili_data/video_dwpose/91tO0imIhfS.mp4
./ubcNbili_data/video_dwpose/A1PBDuSfdiS.mp4
./ubcNbili_data/video_dwpose/919vwGkoBZS.mp4
./ubcNbili_data/video_dwpose/91VR+4O2H6S.mp4
./ubcNbili_data/video_dwpose/919quRNFxHS.mp4
./ubcNbili_data/video_dwpose/91ENRDaKyIS.mp4
./ubcNbili_data/video_dwpose/91d3Lx1uCYS.mp4
./ubcNbili_data/video_dwpose/A1X3koQIOoS.mp4
./ubcNbili_data/video_dwpose/91-9wEBqAlS.mp4
./ubcNbili_data/video_dwpose/A1AKpUs29AS.mp4
./ubcNbili_data/video_dwpose/A1YTUoD6VLS.mp4
./ubcNbili_data/video_dwpose/A1JverX4ADS.mp4
./ubcNbili_data/video_dwpose/91WwxuTSaYS.mp4
./ubcNbili_data/video_dwpose/A1iFJaDDBKS.mp4
./ubcNbili_data/video_dwpose/A1jBHy9ecoS.mp4
./ubcNbili_data/video_dwpose/A11yVBcrftS.mp4
./ubcNbili_data/video_dwpose/A15Khtac8xS.mp4
./ubcNbili_data/video_dwpose/A1cofEJXx3S.mp4
./ubcNbili_data/video_dwpose/91A3-6ehAhS.mp4
./ubcNbili_data/video_dwpose/91t-lS-8VkS.mp4
./ubcNbili_data/video_dwpose/A1jx8Z9eIHS.mp4
./ubcNbili_data/video_dwpose/91TDp5Z08AS.mp4
./ubcNbili_data/video_dwpose/A11UTfKe+tS.mp4
./ubcNbili_data/video_dwpose/919iQ+Yy6qS.mp4
./ubcNbili_data/video_dwpose/91gJasnQEyS.mp4
./ubcNbili_data/video_dwpose/A15Qq5inIHS.mp4
./ubcNbili_data/video_dwpose/A1weUj8AndS.mp4
./ubcNbili_data/video_dwpose/91ORyUPpoDS.mp4
./ubcNbili_data/video_dwpose/A1GqSV9JCNS.mp4
./ubcNbili_data/video_dwpose/A1Er7goewxS.mp4
./ubcNbili_data/video_dwpose/917sawDuD5S.mp4
./ubcNbili_data/video_dwpose/A1662PT6CWS.mp4
./ubcNbili_data/video_dwpose/A1jqlFsctyS.mp4
./ubcNbili_data/video_dwpose/91IVZ9g5lGS.mp4
./ubcNbili_data/video_dwpose/91lOX26QQ+S.mp4
./ubcNbili_data/video_dwpose/A1IWb5AMMlS.mp4
./ubcNbili_data/video_dwpose/91lUoCMPlGS.mp4
./ubcNbili_data/video_dwpose/A1jFMj0n1JS.mp4
./ubcNbili_data/video_dwpose/A11y7ZJTf6S.mp4
./ubcNbili_data/video_dwpose/91khUDfmcnS.mp4
./ubcNbili_data/video_dwpose/A1lLkaME1RS.mp4
./ubcNbili_data/video_dwpose/91itUP78RnS.mp4
./ubcNbili_data/video_dwpose/91NlXJ+dBRS.mp4
./ubcNbili_data/meta_data/
./ubcNbili_data/meta_data/ubcNbili_meta.json
该数据集由三部分组成,分别为元数据、原始视频以及对应动作视频,其中元数据记录对应原始视频和动作视频的路径,动作视频提取方式参考自MooreThreads/Moore-AnimateAnyone,训练数据文件结构如下:
├── ubcNbili_data # 训练数据根目录
├── meta_data # 元数据文件夹
├── ubcNbili_meta.json
├── video # 原始视频文件夹
├── 00001.mp4
├── 00002.mp4
├── ...
├── 03049.mp4
├── video_dwpose # 动作视频文件夹
├── 00001.mp4
├── 00002.mp4
├── ...
├── 03049.mp4
4.2 第一阶段训练
第一阶段由于训练参数规模较大无法在单卡 NVIDIA V100 32G GPU 或 NVIDIA A100 40G GPU 上运行,可在单机多卡下开启显存优化分组切片技术 --sharding
进行训练,训练命令如下:
In [10]
!python -u -m paddle.distributed.launch --gpus "0" scripts/trainer_stage1.py \
--do_train \
--output_dir ./exp_output/stage1 \
--save_strategy 'steps' \
--save_total_limit 2 \
--save_steps 2000 \
--per_device_train_batch_size 1 \
--gradient_accumulation_steps 1 \
--learning_rate 1.0e-5 \
--weight_decay 1.0e-2 \
--max_steps 30000 \
--lr_scheduler_type "constant" \
--warmup_steps 1 \
--seed 42 \
--report_to all \
--sharding "stage1" \
--fp16 True \
--fp16_opt_level O2
LAUNCH INFO 2024-04-20 16:52:52,361 ----------- Configuration ----------------------
LAUNCH INFO 2024-04-20 16:52:52,361 auto_parallel_config: None
LAUNCH INFO 2024-04-20 16:52:52,361 auto_tuner_json: None
LAUNCH INFO 2024-04-20 16:52:52,361 devices: 0
LAUNCH INFO 2024-04-20 16:52:52,361 elastic_level: -1
LAUNCH INFO 2024-04-20 16:52:52,361 elastic_timeout: 30
LAUNCH INFO 2024-04-20 16:52:52,361 enable_gpu_log: True
LAUNCH INFO 2024-04-20 16:52:52,361 gloo_port: 6767
LAUNCH INFO 2024-04-20 16:52:52,361 host: None
LAUNCH INFO 2024-04-20 16:52:52,361 ips: None
LAUNCH INFO 2024-04-20 16:52:52,361 job_id: default
LAUNCH INFO 2024-04-20 16:52:52,361 legacy: False
LAUNCH INFO 2024-04-20 16:52:52,361 log_dir: log
LAUNCH INFO 2024-04-20 16:52:52,361 log_level: INFO
LAUNCH INFO 2024-04-20 16:52:52,361 log_overwrite: False
LAUNCH INFO 2024-04-20 16:52:52,361 master: None
LAUNCH INFO 2024-04-20 16:52:52,361 max_restart: 3
LAUNCH INFO 2024-04-20 16:52:52,361 nnodes: 1
LAUNCH INFO 2024-04-20 16:52:52,361 nproc_per_node: None
LAUNCH INFO 2024-04-20 16:52:52,361 rank: -1
LAUNCH INFO 2024-04-20 16:52:52,362 run_mode: collective
LAUNCH INFO 2024-04-20 16:52:52,362 server_num: None
LAUNCH INFO 2024-04-20 16:52:52,362 servers:
LAUNCH INFO 2024-04-20 16:52:52,362 sort_ip: False
LAUNCH INFO 2024-04-20 16:52:52,362 start_port: 6070
LAUNCH INFO 2024-04-20 16:52:52,362 trainer_num: None
LAUNCH INFO 2024-04-20 16:52:52,362 trainers:
LAUNCH INFO 2024-04-20 16:52:52,362 training_script: scripts/trainer_stage1.py
LAUNCH INFO 2024-04-20 16:52:52,362 training_script_args: ['--do_train', '--output_dir', './exp_output/stage1', '--save_strategy', 'steps', '--save_total_limit', '2', '--save_steps', '2000', '--per_device_train_batch_size', '1', '--gradient_accumulation_steps', '1', '--learning_rate', '1.0e-5', '--weight_decay', '1.0e-2', '--max_steps', '30000', '--lr_scheduler_type', 'constant', '--warmup_steps', '1', '--seed', '42', '--report_to', 'all', '--sharding', 'stage1', '--fp16', 'True', '--fp16_opt_level', 'O2']
LAUNCH INFO 2024-04-20 16:52:52,362 with_gloo: 1
LAUNCH INFO 2024-04-20 16:52:52,362 --------------------------------------------------
LAUNCH INFO 2024-04-20 16:52:52,362 Job: default, mode collective, replicas 1[1:1], elastic False
LAUNCH INFO 2024-04-20 16:52:52,366 Run Pod: owdogg, replicas 1, status ready
LAUNCH INFO 2024-04-20 16:52:52,402 Watching Pod: owdogg, replicas 1, status running
/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
W0420 16:52:56.421471 232881 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 12.0, Runtime API Version: 11.8
W0420 16:52:56.422761 232881 gpu_resources.cc:164] device: 0, cuDNN Version: 8.9.
[2024-04-20 16:52:58,970] [ DEBUG] - ============================================================
[2024-04-20 16:52:58,971] [ DEBUG] - Model Configuration Arguments
[2024-04-20 16:52:58,971] [ DEBUG] - paddle commit id : fbf852dd832bc0e63ae31cd4aa37defd829e4c03
[2024-04-20 16:52:58,971] [ DEBUG] - paddlenlp commit id : b39e701e21d11ff66ac3abfc81d384b6af8f8240
[2024-04-20 16:52:58,971] [ DEBUG] - base_model_path : lambdalabs/sd-image-variations-diffusers
[2024-04-20 16:52:58,971] [ DEBUG] - benchmark : False
[2024-04-20 16:52:58,971] [ DEBUG] - beta_end : 0.012
[2024-04-20 16:52:58,971] [ DEBUG] - beta_schedule : scaled_linear
[2024-04-20 16:52:58,971] [ DEBUG] - beta_start : 0.00085
[2024-04-20 16:52:58,971] [ DEBUG] - clip_sample : False
[2024-04-20 16:52:58,971] [ DEBUG] - controlnet_openpose_path : ./pretrained_weights/tsaiyue/AnimateAnyone_PD/control_v11p_sd15_openpose.pdparams
[2024-04-20 16:52:58,971] [ DEBUG] - denoising_unet_base_model_path: ./pretrained_weights/tsaiyue/AnimateAnyone_PD/denoising_unet_initial4stage1.pdparams
[2024-04-20 16:52:58,972] [ DEBUG] - denoising_unet_config_path : ./pretrained_weights/tsaiyue/AnimateAnyone_PD/config.json
[2024-04-20 16:52:58,972] [ DEBUG] - image_encoder_path : lambdalabs/sd-image-variations-diffusers
[2024-04-20 16:52:58,972] [ DEBUG] - noise_offset : 0.05
[2024-04-20 16:52:58,972] [ DEBUG] - num_train_timesteps : 1000
[2024-04-20 16:52:58,972] [ DEBUG] - pose_guider_pretrain : True
[2024-04-20 16:52:58,972] [ DEBUG] - prediction_type : v_prediction
[2024-04-20 16:52:58,972] [ DEBUG] - profiler_options : None
[2024-04-20 16:52:58,972] [ DEBUG] - rescale_betas_zero_snr : True
[2024-04-20 16:52:58,972] [ DEBUG] - snr_gamma : 5.0
[2024-04-20 16:52:58,972] [ DEBUG] - steps_offset : 1
[2024-04-20 16:52:58,972] [ DEBUG] - timestep_spacing : trailing
[2024-04-20 16:52:58,972] [ DEBUG] - uncond_ratio : 0.1
[2024-04-20 16:52:58,972] [ DEBUG] - vae_model_path : stabilityai/sd-vae-ft-mse
[2024-04-20 16:52:58,972] [ DEBUG] -
[2024-04-20 16:52:58,972] [ DEBUG] - ============================================================
[2024-04-20 16:52:58,972] [ DEBUG] - Data Configuration Arguments
[2024-04-20 16:52:58,972] [ DEBUG] - paddle commit id : fbf852dd832bc0e63ae31cd4aa37defd829e4c03
[2024-04-20 16:52:58,973] [ DEBUG] - paddlenlp commit id : b39e701e21d11ff66ac3abfc81d384b6af8f8240
[2024-04-20 16:52:58,973] [ DEBUG] - meta_paths : ./ubcNbili_data/meta_data/ubcNbili_meta.json
[2024-04-20 16:52:58,973] [ DEBUG] - sample_margin : 30
[2024-04-20 16:52:58,973] [ DEBUG] - train_height : 768
[2024-04-20 16:52:58,973] [ DEBUG] - train_width : 768
[2024-04-20 16:52:58,973] [ DEBUG] -
The config attributes {'resnet_pre_temb_non_linearity': False} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file.
Some weights of the model checkpoint at lambdalabs/sd-image-variations-diffusers were not used when initializing UNet2DConditionModel: ['conv_norm_out.bias', 'conv_norm_out.weight', 'conv_out.bias', 'conv_out.weight']
- This IS expected if you are initializing UNet2DConditionModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing UNet2DConditionModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
### missing keys: 0;
### unexpected keys: 0;
[2024-04-20 16:53:53,316] [ INFO] - Found /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/config.json
[2024-04-20 16:53:53,317] [ INFO] - Loading configuration file /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/config.json
[2024-04-20 16:53:53,318] [ INFO] - Model config CLIPVisionConfig {
"_name_or_path": "/home/jpinkney/.cache/huggingface/diffusers/models--lambdalabs--sd-image-variations-diffusers/snapshots/ca6f97f838ae1b5bf764f31363a21f388f4d8f3e/image_encoder",
"architectures": [
"CLIPVisionModelWithProjection"
],
"attention_dropout": 0.0,
"dropout": 0.0,
"hidden_act": "quick_gelu",
"hidden_size": 1024,
"image_size": 224,
"initializer_factor": 1.0,
"initializer_range": 0.02,
"intermediate_size": 4096,
"layer_norm_eps": 1e-05,
"model_type": "clip_vision_model",
"num_attention_heads": 16,
"num_channels": 3,
"num_hidden_layers": 24,
"paddlenlp_version": null,
"patch_size": 14,
"projection_dim": 768,
"return_dict": true,
"transformers_version": "4.25.1"
}
[2024-04-20 16:53:53,433] [ INFO] - Already cached /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/model_state.pdparams
[2024-04-20 16:53:53,433] [ INFO] - Loading weights file model_state.pdparams from cache at /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/model_state.pdparams
[2024-04-20 16:53:54,408] [ INFO] - Loaded weights file from disk, setting weights to model.
[2024-04-20 16:53:55,614] [ INFO] - All model checkpoint weights were used when initializing CLIPVisionModelWithProjection.
[2024-04-20 16:53:55,614] [ INFO] - All the weights of CLIPVisionModelWithProjection were initialized from the model checkpoint at lambdalabs/sd-image-variations-diffusers/image_encoder.
If your task is similar to the task the model of the checkpoint was trained on, you can already use CLIPVisionModelWithProjection for predictions without further training.
/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/nn/layer/layers.py:2084: UserWarning: Skip loading for conv_out.weight. conv_out.weight is not found in the provided dict.
warnings.warn(f"Skip loading for {key}. " + str(err))
/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/nn/layer/layers.py:2084: UserWarning: Skip loading for conv_out.bias. conv_out.bias is not found in the provided dict.
warnings.warn(f"Skip loading for {key}. " + str(err))
[2024-04-20 16:54:05,689] [ INFO] - Missing key for pose guider: 2
[2024-04-20 16:54:11,124] [ INFO] - The global seed is set to 42, local seed is set to 43 and random seed is set to 42.
[2024-04-20 16:54:11,214] [ INFO] - max_steps is given, it will override any value given in num_train_epochs
[2024-04-20 16:54:11,214] [ INFO] - Using half precision
[2024-04-20 16:54:11,632] [ DEBUG] - ============================================================
[2024-04-20 16:54:11,633] [ DEBUG] - Training Configuration Arguments
[2024-04-20 16:54:11,633] [ DEBUG] - paddle commit id : fbf852dd832bc0e63ae31cd4aa37defd829e4c03
[2024-04-20 16:54:11,633] [ DEBUG] - paddlenlp commit id : b39e701e21d11ff66ac3abfc81d384b6af8f8240
[2024-04-20 16:54:11,633] [ DEBUG] - _no_sync_in_gradient_accumulation: True
[2024-04-20 16:54:11,633] [ DEBUG] - adam_beta1 : 0.9
[2024-04-20 16:54:11,633] [ DEBUG] - adam_beta2 : 0.999
[2024-04-20 16:54:11,633] [ DEBUG] - adam_epsilon : 1e-08
[2024-04-20 16:54:11,633] [ DEBUG] - amp_custom_black_list : None
[2024-04-20 16:54:11,633] [ DEBUG] - amp_custom_white_list : None
[2024-04-20 16:54:11,633] [ DEBUG] - amp_master_grad : False
[2024-04-20 16:54:11,633] [ DEBUG] - bf16 : False
[2024-04-20 16:54:11,634] [ DEBUG] - bf16_full_eval : False
[2024-04-20 16:54:11,634] [ DEBUG] - current_device : gpu:0
[2024-04-20 16:54:11,634] [ DEBUG] - data_parallel_rank : 0
[2024-04-20 16:54:11,634] [ DEBUG] - dataloader_drop_last : False
[2024-04-20 16:54:11,634] [ DEBUG] - dataloader_num_workers : 0
[2024-04-20 16:54:11,634] [ DEBUG] - dataset_rank : 0
[2024-04-20 16:54:11,634] [ DEBUG] - dataset_world_size : 1
[2024-04-20 16:54:11,634] [ DEBUG] - device : gpu
[2024-04-20 16:54:11,634] [ DEBUG] - disable_tqdm : False
[2024-04-20 16:54:11,634] [ DEBUG] - distributed_dataloader : False
[2024-04-20 16:54:11,634] [ DEBUG] - do_eval : False
[2024-04-20 16:54:11,634] [ DEBUG] - do_export : False
[2024-04-20 16:54:11,634] [ DEBUG] - do_predict : False
[2024-04-20 16:54:11,634] [ DEBUG] - do_train : True
[2024-04-20 16:54:11,635] [ DEBUG] - eval_accumulation_steps : None
[2024-04-20 16:54:11,635] [ DEBUG] - eval_batch_size : 8
[2024-04-20 16:54:11,635] [ DEBUG] - eval_steps : None
[2024-04-20 16:54:11,635] [ DEBUG] - evaluation_strategy : IntervalStrategy.NO
[2024-04-20 16:54:11,635] [ DEBUG] - flatten_param_grads : False
[2024-04-20 16:54:11,635] [ DEBUG] - force_reshard_pp : False
[2024-04-20 16:54:11,635] [ DEBUG] - fp16 : True
[2024-04-20 16:54:11,635] [ DEBUG] - fp16_full_eval : False
[2024-04-20 16:54:11,635] [ DEBUG] - fp16_opt_level : O2
[2024-04-20 16:54:11,635] [ DEBUG] - gradient_accumulation_steps : 1
[2024-04-20 16:54:11,635] [ DEBUG] - greater_is_better : None
[2024-04-20 16:54:11,635] [ DEBUG] - hybrid_parallel_topo_order : None
[2024-04-20 16:54:11,635] [ DEBUG] - ignore_data_skip : False
[2024-04-20 16:54:11,635] [ DEBUG] - ignore_load_lr_and_optim : False
[2024-04-20 16:54:11,636] [ DEBUG] - label_names : None
[2024-04-20 16:54:11,636] [ DEBUG] - lazy_data_processing : True
[2024-04-20 16:54:11,636] [ DEBUG] - learning_rate : 1e-05
[2024-04-20 16:54:11,636] [ DEBUG] - load_best_model_at_end : False
[2024-04-20 16:54:11,636] [ DEBUG] - load_sharded_model : False
[2024-04-20 16:54:11,636] [ DEBUG] - local_process_index : 0
[2024-04-20 16:54:11,636] [ DEBUG] - local_rank : -1
[2024-04-20 16:54:11,636] [ DEBUG] - log_level : -1
[2024-04-20 16:54:11,636] [ DEBUG] - log_level_replica : -1
[2024-04-20 16:54:11,636] [ DEBUG] - log_on_each_node : True
[2024-04-20 16:54:11,636] [ DEBUG] - logging_dir : ./exp_output/stage1/runs/Apr20_16-52-58_jupyter-530807-7490749
[2024-04-20 16:54:11,636] [ DEBUG] - logging_first_step : False
[2024-04-20 16:54:11,636] [ DEBUG] - logging_steps : 500
[2024-04-20 16:54:11,636] [ DEBUG] - logging_strategy : IntervalStrategy.STEPS
[2024-04-20 16:54:11,636] [ DEBUG] - logical_process_index : 0
[2024-04-20 16:54:11,636] [ DEBUG] - lr_end : 1e-07
[2024-04-20 16:54:11,637] [ DEBUG] - lr_scheduler_type : SchedulerType.CONSTANT
[2024-04-20 16:54:11,637] [ DEBUG] - max_evaluate_steps : -1
[2024-04-20 16:54:11,637] [ DEBUG] - max_grad_norm : 1.0
[2024-04-20 16:54:11,637] [ DEBUG] - max_steps : 30000
[2024-04-20 16:54:11,637] [ DEBUG] - metric_for_best_model : None
[2024-04-20 16:54:11,637] [ DEBUG] - minimum_eval_times : None
[2024-04-20 16:54:11,637] [ DEBUG] - no_cuda : False
[2024-04-20 16:54:11,637] [ DEBUG] - num_cycles : 0.5
[2024-04-20 16:54:11,637] [ DEBUG] - num_train_epochs : 3.0
[2024-04-20 16:54:11,637] [ DEBUG] - optim : OptimizerNames.ADAMW
[2024-04-20 16:54:11,637] [ DEBUG] - optimizer_name_suffix : None
[2024-04-20 16:54:11,637] [ DEBUG] - output_dir : ./exp_output/stage1
[2024-04-20 16:54:11,637] [ DEBUG] - overwrite_output_dir : False
[2024-04-20 16:54:11,637] [ DEBUG] - past_index : -1
[2024-04-20 16:54:11,637] [ DEBUG] - per_device_eval_batch_size : 8
[2024-04-20 16:54:11,637] [ DEBUG] - per_device_train_batch_size : 1
[2024-04-20 16:54:11,638] [ DEBUG] - pipeline_parallel_config :
[2024-04-20 16:54:11,638] [ DEBUG] - pipeline_parallel_degree : -1
[2024-04-20 16:54:11,638] [ DEBUG] - pipeline_parallel_rank : 0
[2024-04-20 16:54:11,638] [ DEBUG] - power : 1.0
[2024-04-20 16:54:11,638] [ DEBUG] - prediction_loss_only : False
[2024-04-20 16:54:11,638] [ DEBUG] - process_index : 0
[2024-04-20 16:54:11,638] [ DEBUG] - recompute : False
[2024-04-20 16:54:11,638] [ DEBUG] - remove_unused_columns : True
[2024-04-20 16:54:11,638] [ DEBUG] - report_to : ['custom_visualdl']
[2024-04-20 16:54:11,638] [ DEBUG] - resume_from_checkpoint : None
[2024-04-20 16:54:11,638] [ DEBUG] - run_name : ./exp_output/stage1
[2024-04-20 16:54:11,638] [ DEBUG] - save_on_each_node : False
[2024-04-20 16:54:11,638] [ DEBUG] - save_sharded_model : False
[2024-04-20 16:54:11,638] [ DEBUG] - save_steps : 2000
[2024-04-20 16:54:11,638] [ DEBUG] - save_strategy : IntervalStrategy.STEPS
[2024-04-20 16:54:11,638] [ DEBUG] - save_total_limit : 2
[2024-04-20 16:54:11,639] [ DEBUG] - scale_loss : 32768
[2024-04-20 16:54:11,639] [ DEBUG] - seed : 42
[2024-04-20 16:54:11,639] [ DEBUG] - sep_parallel_degree : -1
[2024-04-20 16:54:11,639] [ DEBUG] - sharding : []
[2024-04-20 16:54:11,639] [ DEBUG] - sharding_degree : -1
[2024-04-20 16:54:11,639] [ DEBUG] - sharding_parallel_config :
[2024-04-20 16:54:11,639] [ DEBUG] - sharding_parallel_degree : -1
[2024-04-20 16:54:11,639] [ DEBUG] - sharding_parallel_rank : 0
[2024-04-20 16:54:11,639] [ DEBUG] - should_load_dataset : True
[2024-04-20 16:54:11,639] [ DEBUG] - should_load_sharding_stage1_model: False
[2024-04-20 16:54:11,639] [ DEBUG] - should_log : True
[2024-04-20 16:54:11,639] [ DEBUG] - should_save : True
[2024-04-20 16:54:11,639] [ DEBUG] - should_save_model_state : True
[2024-04-20 16:54:11,639] [ DEBUG] - should_save_sharding_stage1_model: False
[2024-04-20 16:54:11,639] [ DEBUG] - skip_memory_metrics : True
[2024-04-20 16:54:11,640] [ DEBUG] - skip_profile_timer : True
[2024-04-20 16:54:11,640] [ DEBUG] - tensor_parallel_config :
[2024-04-20 16:54:11,640] [ DEBUG] - tensor_parallel_degree : -1
[2024-04-20 16:54:11,640] [ DEBUG] - tensor_parallel_rank : 0
[2024-04-20 16:54:11,640] [ DEBUG] - to_static : False
[2024-04-20 16:54:11,640] [ DEBUG] - train_batch_size : 1
[2024-04-20 16:54:11,640] [ DEBUG] - unified_checkpoint : False
[2024-04-20 16:54:11,640] [ DEBUG] - unified_checkpoint_config :
[2024-04-20 16:54:11,640] [ DEBUG] - use_auto_parallel : False
[2024-04-20 16:54:11,640] [ DEBUG] - use_hybrid_parallel : False
[2024-04-20 16:54:11,640] [ DEBUG] - warmup_ratio : 0.0
[2024-04-20 16:54:11,640] [ DEBUG] - warmup_steps : 1
[2024-04-20 16:54:11,640] [ DEBUG] - weight_decay : 0.01
[2024-04-20 16:54:11,640] [ DEBUG] - weight_name_suffix : None
[2024-04-20 16:54:11,640] [ DEBUG] - world_size : 1
[2024-04-20 16:54:11,640] [ DEBUG] -
[2024-04-20 16:54:11,645] [ INFO] - Starting training from resume_from_checkpoint : None
/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/distributed/parallel.py:410: UserWarning: The program will return to single-card operation. Please check 1, whether you use spawn or fleetrun to start the program. 2, Whether it is a multi-card program. 3, Is the current environment multi-card.
warnings.warn(
[2024-04-20 16:54:11,655] [ INFO] - [timelog] checkpoint loading time: 0.00s (2024-04-20 16:54:11)
[2024-04-20 16:54:11,655] [ INFO] - ***** Running training *****
[2024-04-20 16:54:11,655] [ INFO] - Num examples = 3,049
[2024-04-20 16:54:11,655] [ INFO] - Num Epochs = 10
[2024-04-20 16:54:11,656] [ INFO] - Instantaneous batch size per device = 1
[2024-04-20 16:54:11,656] [ INFO] - Total train batch size (w. parallel, distributed & accumulation) = 1
[2024-04-20 16:54:11,656] [ INFO] - Gradient Accumulation steps = 1
[2024-04-20 16:54:11,656] [ INFO] - Total optimization steps = 30,000
[2024-04-20 16:54:11,656] [ INFO] - Total num train samples = 30,000
[2024-04-20 16:54:11,665] [ DEBUG] - Number of trainable parameters = 860,595,280 (per device)
TrainProcess: 0%| | 0/30000 [00:00<?, ?it/s]
TrainProcess: 0%| | 1/30000 [00:11<92:15:24, 11.07s/it][2024-04-20 16:54:22,745] [ INFO] - loss: 1.91078484, learning_rate: 1e-05, global_step: 1, interval_runtime: 11.0801, interval_samples_per_second: 0.09025176825989731, interval_steps_per_second: 0.09025176825989731, progress_or_epoch: 0.0003
TrainProcess: 0%| | 2/30000 [00:13<48:05:11, 5.77s/it][2024-04-20 16:54:24,801] [ INFO] - loss: 0.1175241, learning_rate: 1e-05, global_step: 2, interval_runtime: 2.0553, interval_samples_per_second: 0.48655843440231544, interval_steps_per_second: 0.48655843440231544, progress_or_epoch: 0.0007
TrainProcess: 0%| | 3/30000 [00:17<42:56:09, 5.15s/it][2024-04-20 16:54:29,224] [ INFO] - loss: 0.47150442, learning_rate: 1e-05, global_step: 3, interval_runtime: 4.4235, interval_samples_per_second: 0.2260655978214763, interval_steps_per_second: 0.2260655978214763, progress_or_epoch: 0.001
TrainProcess: 0%| | 4/30000 [00:22<41:09:31, 4.94s/it][2024-04-20 16:54:33,831] [ INFO] - loss: 0.03123931, learning_rate: 1e-05, global_step: 4, interval_runtime: 4.6068, interval_samples_per_second: 0.21706876249290527, interval_steps_per_second: 0.21706876249290527, progress_or_epoch: 0.0013
TrainProcess: 0%| | 5/30000 [00:26<39:23:45, 4.73s/it][2024-04-20 16:54:38,190] [ INFO] - loss: 0.75247884, learning_rate: 1e-05, global_step: 5, interval_runtime: 4.359, interval_samples_per_second: 0.22940956369452584, interval_steps_per_second: 0.22940956369452584, progress_or_epoch: 0.0016
TrainProcess: 0%| | 6/30000 [00:30<38:21:42, 4.60s/it][2024-04-20 16:54:42,550] [ INFO] - loss: 0.54467314, learning_rate: 1e-05, global_step: 6, interval_runtime: 4.3595, interval_samples_per_second: 0.22938243883674017, interval_steps_per_second: 0.22938243883674017, progress_or_epoch: 0.002
TrainProcess: 0%| | 7/30000 [00:32<30:31:01, 3.66s/it][2024-04-20 16:54:44,273] [ INFO] - loss: 0.22554155, learning_rate: 1e-05, global_step: 7, interval_runtime: 1.7233, interval_samples_per_second: 0.5802851547557036, interval_steps_per_second: 0.5802851547557036, progress_or_epoch: 0.0023
TrainProcess: 0%| | 8/30000 [00:36<31:57:59, 3.84s/it][2024-04-20 16:54:48,483] [ INFO] - loss: 0.236911, learning_rate: 1e-05, global_step: 8, interval_runtime: 4.2098, interval_samples_per_second: 0.23754199796625777, interval_steps_per_second: 0.23754199796625777, progress_or_epoch: 0.0026
^C
LAUNCH INFO 2024-04-20 16:54:49,276 Terminating with signal 2
训练脚本基于 paddlenlp.trainer 实现,支持单卡、多卡训练,可通过 --gpus
指定训练使用的GPU卡号,在多卡环境上支持分组切片技术以降低显存占用。训练过程中的阶段性权重以及可视化训练监控文件将存储于 exp_output/stage1
目录下。训练流程相关参数详见 paddlenlp.trainer,模型与数据相关参数详见 src/trainer/args_stage1.py
。
4.3 第二阶段训练
第二阶段训练支持单卡 NVIDIA V100 32G GPU 的硬件环境,训练命令如下:
In [9]
!python -u -m paddle.distributed.launch --gpus "0" scripts/trainer_stage2.py \
--do_train \
--output_dir ./exp_output/stage2 \
--save_strategy 'steps' \
--save_total_limit 2 \
--save_steps 2000 \
--per_device_train_batch_size 1 \
--gradient_accumulation_steps 1 \
--learning_rate 1.0e-5 \
--weight_decay 1.0e-2 \
--max_steps 30000 \
--lr_scheduler_type "constant" \
--warmup_steps 1 \
--seed 42 \
--report_to all \
--fp16 True \
--fp16_opt_level O2 \
--train_width 256 \
--train_height 512
LAUNCH INFO 2024-04-20 16:50:05,751 ----------- Configuration ----------------------
LAUNCH INFO 2024-04-20 16:50:05,751 auto_parallel_config: None
LAUNCH INFO 2024-04-20 16:50:05,751 auto_tuner_json: None
LAUNCH INFO 2024-04-20 16:50:05,751 devices: 0
LAUNCH INFO 2024-04-20 16:50:05,751 elastic_level: -1
LAUNCH INFO 2024-04-20 16:50:05,751 elastic_timeout: 30
LAUNCH INFO 2024-04-20 16:50:05,751 enable_gpu_log: True
LAUNCH INFO 2024-04-20 16:50:05,751 gloo_port: 6767
LAUNCH INFO 2024-04-20 16:50:05,751 host: None
LAUNCH INFO 2024-04-20 16:50:05,751 ips: None
LAUNCH INFO 2024-04-20 16:50:05,751 job_id: default
LAUNCH INFO 2024-04-20 16:50:05,751 legacy: False
LAUNCH INFO 2024-04-20 16:50:05,751 log_dir: log
LAUNCH INFO 2024-04-20 16:50:05,751 log_level: INFO
LAUNCH INFO 2024-04-20 16:50:05,751 log_overwrite: False
LAUNCH INFO 2024-04-20 16:50:05,751 master: None
LAUNCH INFO 2024-04-20 16:50:05,751 max_restart: 3
LAUNCH INFO 2024-04-20 16:50:05,752 nnodes: 1
LAUNCH INFO 2024-04-20 16:50:05,752 nproc_per_node: None
LAUNCH INFO 2024-04-20 16:50:05,752 rank: -1
LAUNCH INFO 2024-04-20 16:50:05,752 run_mode: collective
LAUNCH INFO 2024-04-20 16:50:05,752 server_num: None
LAUNCH INFO 2024-04-20 16:50:05,752 servers:
LAUNCH INFO 2024-04-20 16:50:05,752 sort_ip: False
LAUNCH INFO 2024-04-20 16:50:05,752 start_port: 6070
LAUNCH INFO 2024-04-20 16:50:05,752 trainer_num: None
LAUNCH INFO 2024-04-20 16:50:05,752 trainers:
LAUNCH INFO 2024-04-20 16:50:05,752 training_script: scripts/trainer_stage2.py
LAUNCH INFO 2024-04-20 16:50:05,752 training_script_args: ['--do_train', '--output_dir', './exp_output/stage2', '--save_strategy', 'steps', '--save_total_limit', '2', '--save_steps', '2000', '--per_device_train_batch_size', '1', '--gradient_accumulation_steps', '1', '--learning_rate', '1.0e-5', '--weight_decay', '1.0e-2', '--max_steps', '30000', '--lr_scheduler_type', 'constant', '--warmup_steps', '1', '--seed', '42', '--report_to', 'all', '--fp16', 'True', '--fp16_opt_level', 'O2', '--train_width', '256', '--train_height', '512']
LAUNCH INFO 2024-04-20 16:50:05,752 with_gloo: 1
LAUNCH INFO 2024-04-20 16:50:05,752 --------------------------------------------------
LAUNCH INFO 2024-04-20 16:50:05,752 Job: default, mode collective, replicas 1[1:1], elastic False
LAUNCH INFO 2024-04-20 16:50:05,755 Run Pod: dtnavn, replicas 1, status ready
LAUNCH INFO 2024-04-20 16:50:05,784 Watching Pod: dtnavn, replicas 1, status running
/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
W0420 16:50:09.388610 227925 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 12.0, Runtime API Version: 11.8
W0420 16:50:09.390048 227925 gpu_resources.cc:164] device: 0, cuDNN Version: 8.9.
[2024-04-20 16:50:11,944] [ DEBUG] - ============================================================
[2024-04-20 16:50:11,945] [ DEBUG] - Model Configuration Arguments
[2024-04-20 16:50:11,945] [ DEBUG] - paddle commit id : fbf852dd832bc0e63ae31cd4aa37defd829e4c03
[2024-04-20 16:50:11,945] [ DEBUG] - paddlenlp commit id : b39e701e21d11ff66ac3abfc81d384b6af8f8240
[2024-04-20 16:50:11,945] [ DEBUG] - base_model_path : runwayml/stable-diffusion-v1-5
[2024-04-20 16:50:11,945] [ DEBUG] - benchmark : False
[2024-04-20 16:50:11,945] [ DEBUG] - beta_end : 0.012
[2024-04-20 16:50:11,945] [ DEBUG] - beta_schedule : scaled_linear
[2024-04-20 16:50:11,945] [ DEBUG] - beta_start : 0.00085
[2024-04-20 16:50:11,945] [ DEBUG] - clip_sample : False
[2024-04-20 16:50:11,945] [ DEBUG] - denoising_unet_base_model_path: ./pretrained_weights/tsaiyue/AnimateAnyone_PD/denoising_unet.pdparams
[2024-04-20 16:50:11,945] [ DEBUG] - denoising_unet_config_path : ./pretrained_weights/tsaiyue/AnimateAnyone_PD/config.json
[2024-04-20 16:50:11,945] [ DEBUG] - image_encoder_path : lambdalabs/sd-image-variations-diffusers
[2024-04-20 16:50:11,945] [ DEBUG] - inference_config_path : ./configs/inference/inference_v2.yaml
[2024-04-20 16:50:11,945] [ DEBUG] - motion_module_path : ./pretrained_weights/tsaiyue/AnimateAnyone_PD/animatediff_mm_sd_v15_v2.pdparams
[2024-04-20 16:50:11,945] [ DEBUG] - noise_offset : 0.05
[2024-04-20 16:50:11,946] [ DEBUG] - num_train_timesteps : 1000
[2024-04-20 16:50:11,946] [ DEBUG] - pose_guider_path : ./pretrained_weights/tsaiyue/AnimateAnyone_PD/pose_guider.pdparams
[2024-04-20 16:50:11,946] [ DEBUG] - pose_guider_pretrain : True
[2024-04-20 16:50:11,946] [ DEBUG] - prediction_type : v_prediction
[2024-04-20 16:50:11,946] [ DEBUG] - profiler_options : None
[2024-04-20 16:50:11,946] [ DEBUG] - reference_unet_path : ./pretrained_weights/tsaiyue/AnimateAnyone_PD/reference_unet.pdparams
[2024-04-20 16:50:11,946] [ DEBUG] - rescale_betas_zero_snr : True
[2024-04-20 16:50:11,946] [ DEBUG] - snr_gamma : 5.0
[2024-04-20 16:50:11,946] [ DEBUG] - steps_offset : 1
[2024-04-20 16:50:11,946] [ DEBUG] - timestep_spacing : trailing
[2024-04-20 16:50:11,946] [ DEBUG] - uncond_ratio : 0.1
[2024-04-20 16:50:11,946] [ DEBUG] - vae_model_path : stabilityai/sd-vae-ft-mse
[2024-04-20 16:50:11,946] [ DEBUG] -
[2024-04-20 16:50:11,946] [ DEBUG] - ============================================================
[2024-04-20 16:50:11,946] [ DEBUG] - Data Configuration Arguments
[2024-04-20 16:50:11,946] [ DEBUG] - paddle commit id : fbf852dd832bc0e63ae31cd4aa37defd829e4c03
[2024-04-20 16:50:11,946] [ DEBUG] - paddlenlp commit id : b39e701e21d11ff66ac3abfc81d384b6af8f8240
[2024-04-20 16:50:11,947] [ DEBUG] - meta_paths : ./ubcNbili_data/meta_data/ubcNbili_meta.json
[2024-04-20 16:50:11,947] [ DEBUG] - n_sample_frames : 16
[2024-04-20 16:50:11,947] [ DEBUG] - sample_rate : 4
[2024-04-20 16:50:11,947] [ DEBUG] - train_height : 512
[2024-04-20 16:50:11,947] [ DEBUG] - train_width : 256
[2024-04-20 16:50:11,947] [ DEBUG] -
[2024-04-20 16:50:12,905] [ INFO] - Found /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/config.json
[2024-04-20 16:50:12,906] [ INFO] - Loading configuration file /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/config.json
[2024-04-20 16:50:12,907] [ INFO] - Model config CLIPVisionConfig {
"_name_or_path": "/home/jpinkney/.cache/huggingface/diffusers/models--lambdalabs--sd-image-variations-diffusers/snapshots/ca6f97f838ae1b5bf764f31363a21f388f4d8f3e/image_encoder",
"architectures": [
"CLIPVisionModelWithProjection"
],
"attention_dropout": 0.0,
"dropout": 0.0,
"hidden_act": "quick_gelu",
"hidden_size": 1024,
"image_size": 224,
"initializer_factor": 1.0,
"initializer_range": 0.02,
"intermediate_size": 4096,
"layer_norm_eps": 1e-05,
"model_type": "clip_vision_model",
"num_attention_heads": 16,
"num_channels": 3,
"num_hidden_layers": 24,
"paddlenlp_version": null,
"patch_size": 14,
"projection_dim": 768,
"return_dict": true,
"transformers_version": "4.25.1"
}
[2024-04-20 16:50:13,019] [ INFO] - Already cached /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/model_state.pdparams
[2024-04-20 16:50:13,020] [ INFO] - Loading weights file model_state.pdparams from cache at /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/model_state.pdparams
[2024-04-20 16:50:14,322] [ INFO] - Loaded weights file from disk, setting weights to model.
[2024-04-20 16:50:15,544] [ INFO] - All model checkpoint weights were used when initializing CLIPVisionModelWithProjection.
[2024-04-20 16:50:15,544] [ INFO] - All the weights of CLIPVisionModelWithProjection were initialized from the model checkpoint at lambdalabs/sd-image-variations-diffusers/image_encoder.
If your task is similar to the task the model of the checkpoint was trained on, you can already use CLIPVisionModelWithProjection for predictions without further training.
Some weights of the model checkpoint at runwayml/stable-diffusion-v1-5 were not used when initializing UNet2DConditionModel: ['conv_norm_out.bias', 'conv_norm_out.weight', 'conv_out.bias', 'conv_out.weight']
- This IS expected if you are initializing UNet2DConditionModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing UNet2DConditionModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
### missing keys: 0;
### unexpected keys: 0;
[2024-04-20 16:51:57,524] [ INFO] - The global seed is set to 42, local seed is set to 43 and random seed is set to 42.
[2024-04-20 16:51:57,624] [ INFO] - max_steps is given, it will override any value given in num_train_epochs
[2024-04-20 16:51:57,625] [ INFO] - Using half precision
[2024-04-20 16:51:58,204] [ DEBUG] - ============================================================
[2024-04-20 16:51:58,205] [ DEBUG] - Training Configuration Arguments
[2024-04-20 16:51:58,206] [ DEBUG] - paddle commit id : fbf852dd832bc0e63ae31cd4aa37defd829e4c03
[2024-04-20 16:51:58,206] [ DEBUG] - paddlenlp commit id : b39e701e21d11ff66ac3abfc81d384b6af8f8240
[2024-04-20 16:51:58,206] [ DEBUG] - _no_sync_in_gradient_accumulation: True
[2024-04-20 16:51:58,206] [ DEBUG] - adam_beta1 : 0.9
[2024-04-20 16:51:58,206] [ DEBUG] - adam_beta2 : 0.999
[2024-04-20 16:51:58,206] [ DEBUG] - adam_epsilon : 1e-08
[2024-04-20 16:51:58,206] [ DEBUG] - amp_custom_black_list : None
[2024-04-20 16:51:58,206] [ DEBUG] - amp_custom_white_list : None
[2024-04-20 16:51:58,206] [ DEBUG] - amp_master_grad : False
[2024-04-20 16:51:58,206] [ DEBUG] - bf16 : False
[2024-04-20 16:51:58,207] [ DEBUG] - bf16_full_eval : False
[2024-04-20 16:51:58,207] [ DEBUG] - current_device : gpu:0
[2024-04-20 16:51:58,207] [ DEBUG] - data_parallel_rank : 0
[2024-04-20 16:51:58,207] [ DEBUG] - dataloader_drop_last : False
[2024-04-20 16:51:58,207] [ DEBUG] - dataloader_num_workers : 0
[2024-04-20 16:51:58,207] [ DEBUG] - dataset_rank : 0
[2024-04-20 16:51:58,207] [ DEBUG] - dataset_world_size : 1
[2024-04-20 16:51:58,207] [ DEBUG] - device : gpu
[2024-04-20 16:51:58,207] [ DEBUG] - disable_tqdm : False
[2024-04-20 16:51:58,207] [ DEBUG] - distributed_dataloader : False
[2024-04-20 16:51:58,207] [ DEBUG] - do_eval : False
[2024-04-20 16:51:58,207] [ DEBUG] - do_export : False
[2024-04-20 16:51:58,207] [ DEBUG] - do_predict : False
[2024-04-20 16:51:58,208] [ DEBUG] - do_train : True
[2024-04-20 16:51:58,208] [ DEBUG] - eval_accumulation_steps : None
[2024-04-20 16:51:58,208] [ DEBUG] - eval_batch_size : 8
[2024-04-20 16:51:58,208] [ DEBUG] - eval_steps : None
[2024-04-20 16:51:58,208] [ DEBUG] - evaluation_strategy : IntervalStrategy.NO
[2024-04-20 16:51:58,208] [ DEBUG] - flatten_param_grads : False
[2024-04-20 16:51:58,208] [ DEBUG] - force_reshard_pp : False
[2024-04-20 16:51:58,208] [ DEBUG] - fp16 : True
[2024-04-20 16:51:58,208] [ DEBUG] - fp16_full_eval : False
[2024-04-20 16:51:58,208] [ DEBUG] - fp16_opt_level : O2
[2024-04-20 16:51:58,208] [ DEBUG] - gradient_accumulation_steps : 1
[2024-04-20 16:51:58,208] [ DEBUG] - greater_is_better : None
[2024-04-20 16:51:58,208] [ DEBUG] - hybrid_parallel_topo_order : None
[2024-04-20 16:51:58,208] [ DEBUG] - ignore_data_skip : False
[2024-04-20 16:51:58,209] [ DEBUG] - ignore_load_lr_and_optim : False
[2024-04-20 16:51:58,209] [ DEBUG] - label_names : None
[2024-04-20 16:51:58,209] [ DEBUG] - lazy_data_processing : True
[2024-04-20 16:51:58,209] [ DEBUG] - learning_rate : 1e-05
[2024-04-20 16:51:58,209] [ DEBUG] - load_best_model_at_end : False
[2024-04-20 16:51:58,209] [ DEBUG] - load_sharded_model : False
[2024-04-20 16:51:58,209] [ DEBUG] - local_process_index : 0
[2024-04-20 16:51:58,209] [ DEBUG] - local_rank : -1
[2024-04-20 16:51:58,209] [ DEBUG] - log_level : -1
[2024-04-20 16:51:58,209] [ DEBUG] - log_level_replica : -1
[2024-04-20 16:51:58,209] [ DEBUG] - log_on_each_node : True
[2024-04-20 16:51:58,209] [ DEBUG] - logging_dir : ./exp_output/stage2/runs/Apr20_16-50-11_jupyter-530807-7490749
[2024-04-20 16:51:58,209] [ DEBUG] - logging_first_step : False
[2024-04-20 16:51:58,209] [ DEBUG] - logging_steps : 500
[2024-04-20 16:51:58,209] [ DEBUG] - logging_strategy : IntervalStrategy.STEPS
[2024-04-20 16:51:58,210] [ DEBUG] - logical_process_index : 0
[2024-04-20 16:51:58,210] [ DEBUG] - lr_end : 1e-07
[2024-04-20 16:51:58,210] [ DEBUG] - lr_scheduler_type : SchedulerType.CONSTANT
[2024-04-20 16:51:58,210] [ DEBUG] - max_evaluate_steps : -1
[2024-04-20 16:51:58,210] [ DEBUG] - max_grad_norm : 1.0
[2024-04-20 16:51:58,210] [ DEBUG] - max_steps : 30000
[2024-04-20 16:51:58,210] [ DEBUG] - metric_for_best_model : None
[2024-04-20 16:51:58,210] [ DEBUG] - minimum_eval_times : None
[2024-04-20 16:51:58,210] [ DEBUG] - no_cuda : False
[2024-04-20 16:51:58,210] [ DEBUG] - num_cycles : 0.5
[2024-04-20 16:51:58,210] [ DEBUG] - num_train_epochs : 3.0
[2024-04-20 16:51:58,210] [ DEBUG] - optim : OptimizerNames.ADAMW
[2024-04-20 16:51:58,210] [ DEBUG] - optimizer_name_suffix : None
[2024-04-20 16:51:58,210] [ DEBUG] - output_dir : ./exp_output/stage2
[2024-04-20 16:51:58,210] [ DEBUG] - overwrite_output_dir : False
[2024-04-20 16:51:58,211] [ DEBUG] - past_index : -1
[2024-04-20 16:51:58,211] [ DEBUG] - per_device_eval_batch_size : 8
[2024-04-20 16:51:58,211] [ DEBUG] - per_device_train_batch_size : 1
[2024-04-20 16:51:58,211] [ DEBUG] - pipeline_parallel_config :
[2024-04-20 16:51:58,211] [ DEBUG] - pipeline_parallel_degree : -1
[2024-04-20 16:51:58,211] [ DEBUG] - pipeline_parallel_rank : 0
[2024-04-20 16:51:58,211] [ DEBUG] - power : 1.0
[2024-04-20 16:51:58,211] [ DEBUG] - prediction_loss_only : False
[2024-04-20 16:51:58,211] [ DEBUG] - process_index : 0
[2024-04-20 16:51:58,211] [ DEBUG] - recompute : False
[2024-04-20 16:51:58,211] [ DEBUG] - remove_unused_columns : True
[2024-04-20 16:51:58,211] [ DEBUG] - report_to : ['custom_visualdl']
[2024-04-20 16:51:58,211] [ DEBUG] - resume_from_checkpoint : None
[2024-04-20 16:51:58,211] [ DEBUG] - run_name : ./exp_output/stage2
[2024-04-20 16:51:58,212] [ DEBUG] - save_on_each_node : False
[2024-04-20 16:51:58,212] [ DEBUG] - save_sharded_model : False
[2024-04-20 16:51:58,212] [ DEBUG] - save_steps : 2000
[2024-04-20 16:51:58,212] [ DEBUG] - save_strategy : IntervalStrategy.STEPS
[2024-04-20 16:51:58,212] [ DEBUG] - save_total_limit : 2
[2024-04-20 16:51:58,212] [ DEBUG] - scale_loss : 32768
[2024-04-20 16:51:58,212] [ DEBUG] - seed : 42
[2024-04-20 16:51:58,212] [ DEBUG] - sep_parallel_degree : -1
[2024-04-20 16:51:58,212] [ DEBUG] - sharding : []
[2024-04-20 16:51:58,212] [ DEBUG] - sharding_degree : -1
[2024-04-20 16:51:58,212] [ DEBUG] - sharding_parallel_config :
[2024-04-20 16:51:58,212] [ DEBUG] - sharding_parallel_degree : -1
[2024-04-20 16:51:58,212] [ DEBUG] - sharding_parallel_rank : 0
[2024-04-20 16:51:58,212] [ DEBUG] - should_load_dataset : True
[2024-04-20 16:51:58,212] [ DEBUG] - should_load_sharding_stage1_model: False
[2024-04-20 16:51:58,213] [ DEBUG] - should_log : True
[2024-04-20 16:51:58,213] [ DEBUG] - should_save : True
[2024-04-20 16:51:58,213] [ DEBUG] - should_save_model_state : True
[2024-04-20 16:51:58,213] [ DEBUG] - should_save_sharding_stage1_model: False
[2024-04-20 16:51:58,213] [ DEBUG] - skip_memory_metrics : True
[2024-04-20 16:51:58,213] [ DEBUG] - skip_profile_timer : True
[2024-04-20 16:51:58,213] [ DEBUG] - tensor_parallel_config :
[2024-04-20 16:51:58,213] [ DEBUG] - tensor_parallel_degree : -1
[2024-04-20 16:51:58,213] [ DEBUG] - tensor_parallel_rank : 0
[2024-04-20 16:51:58,213] [ DEBUG] - to_static : False
[2024-04-20 16:51:58,213] [ DEBUG] - train_batch_size : 1
[2024-04-20 16:51:58,214] [ DEBUG] - unified_checkpoint : False
[2024-04-20 16:51:58,214] [ DEBUG] - unified_checkpoint_config :
[2024-04-20 16:51:58,214] [ DEBUG] - use_auto_parallel : False
[2024-04-20 16:51:58,214] [ DEBUG] - use_hybrid_parallel : False
[2024-04-20 16:51:58,214] [ DEBUG] - warmup_ratio : 0.0
[2024-04-20 16:51:58,214] [ DEBUG] - warmup_steps : 1
[2024-04-20 16:51:58,214] [ DEBUG] - weight_decay : 0.01
[2024-04-20 16:51:58,214] [ DEBUG] - weight_name_suffix : None
[2024-04-20 16:51:58,214] [ DEBUG] - world_size : 1
[2024-04-20 16:51:58,214] [ DEBUG] -
[2024-04-20 16:51:58,220] [ INFO] - Starting training from resume_from_checkpoint : None
/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/paddle/distributed/parallel.py:410: UserWarning: The program will return to single-card operation. Please check 1, whether you use spawn or fleetrun to start the program. 2, Whether it is a multi-card program. 3, Is the current environment multi-card.
warnings.warn(
[2024-04-20 16:51:58,236] [ INFO] - [timelog] checkpoint loading time: 0.00s (2024-04-20 16:51:58)
[2024-04-20 16:51:58,236] [ INFO] - ***** Running training *****
[2024-04-20 16:51:58,236] [ INFO] - Num examples = 3,049
[2024-04-20 16:51:58,236] [ INFO] - Num Epochs = 10
[2024-04-20 16:51:58,236] [ INFO] - Instantaneous batch size per device = 1
[2024-04-20 16:51:58,236] [ INFO] - Total train batch size (w. parallel, distributed & accumulation) = 1
[2024-04-20 16:51:58,236] [ INFO] - Gradient Accumulation steps = 1
[2024-04-20 16:51:58,236] [ INFO] - Total optimization steps = 30,000
[2024-04-20 16:51:58,236] [ INFO] - Total num train samples = 30,000
[2024-04-20 16:51:58,248] [ DEBUG] - Number of trainable parameters = 453,209,280 (per device)
TrainProcess: 0%| | 0/30000 [00:00<?, ?it/s]
TrainProcess: 0%| | 1/30000 [00:10<86:40:21, 10.40s/it][2024-04-20 16:52:08,653] [ INFO] - loss: 0.31033456, learning_rate: 1e-05, global_step: 1, interval_runtime: 10.4051, interval_samples_per_second: 0.09610696034800571, interval_steps_per_second: 0.09610696034800571, progress_or_epoch: 0.0003
TrainProcess: 0%| | 2/30000 [00:12<44:13:50, 5.31s/it][2024-04-20 16:52:10,396] [ INFO] - loss: 0.07570316, learning_rate: 1e-05, global_step: 2, interval_runtime: 1.7429, interval_samples_per_second: 0.5737676414948122, interval_steps_per_second: 0.5737676414948122, progress_or_epoch: 0.0007
TrainProcess: 0%| | 3/30000 [00:14<31:12:57, 3.75s/it][2024-04-20 16:52:12,284] [ INFO] - loss: 0.17628008, learning_rate: 1e-05, global_step: 3, interval_runtime: 1.8879, interval_samples_per_second: 0.5296838228933366, interval_steps_per_second: 0.5296838228933366, progress_or_epoch: 0.001
TrainProcess: 0%| | 4/30000 [00:16<25:44:42, 3.09s/it][2024-04-20 16:52:14,368] [ INFO] - loss: 0.02141302, learning_rate: 1e-05, global_step: 4, interval_runtime: 2.0838, interval_samples_per_second: 0.47989160773432177, interval_steps_per_second: 0.47989160773432177, progress_or_epoch: 0.0013
TrainProcess: 0%| | 5/30000 [00:17<22:01:19, 2.64s/it][2024-04-20 16:52:16,218] [ INFO] - loss: 0.06001347, learning_rate: 1e-05, global_step: 5, interval_runtime: 1.8505, interval_samples_per_second: 0.5403961277767699, interval_steps_per_second: 0.5403961277767699, progress_or_epoch: 0.0016
TrainProcess: 0%| | 6/30000 [00:20<21:15:16, 2.55s/it][2024-04-20 16:52:18,591] [ INFO] - loss: 0.04047307, learning_rate: 1e-05, global_step: 6, interval_runtime: 2.3727, interval_samples_per_second: 0.4214657847141199, interval_steps_per_second: 0.4214657847141199, progress_or_epoch: 0.002
TrainProcess: 0%| | 7/30000 [00:22<19:49:24, 2.38s/it][2024-04-20 16:52:20,617] [ INFO] - loss: 0.11743552, learning_rate: 1e-05, global_step: 7, interval_runtime: 2.0259, interval_samples_per_second: 0.4936056574359056, interval_steps_per_second: 0.4936056574359056, progress_or_epoch: 0.0023
TrainProcess: 0%| | 8/30000 [00:25<20:58:17, 2.52s/it][2024-04-20 16:52:23,429] [ INFO] - loss: 0.07337131, learning_rate: 1e-05, global_step: 8, interval_runtime: 2.8122, interval_samples_per_second: 0.35558750769366415, interval_steps_per_second: 0.35558750769366415, progress_or_epoch: 0.0026
^C
LAUNCH INFO 2024-04-20 16:52:24,566 Terminating with signal 2
该训练脚本同样基于paddlenlp.trainer实现,支持单卡、多卡训练,可通过 --gpus
指定训练使用的GPU卡号。训练过程中的阶段性权重以及可视化训练监控文件将存储于 exp_output/stage2
目录下。训练流程相关参数详见 paddlenlp.trainer,模型与数据相关参数详见 src/trainer/args_stage2.py
。
Note: 可根据具体算力情况适当调整生成视频分辨率相关参数 --train_width
和 --train_width
,以获得更好的训练效果。
4.4 第二阶段微调前后对比
在第二阶段训练中,利用 animatediff初始化权重对模型组网中的motion_modules进行微调,微调前后生成效果对比如下:
5. 模型推理
模型可在NVIDIA V100 32G GPU下进行推理。运行以下推理命令,生成指定宽高和帧数的动画,将存储在 ./output
下。
In [11]
!python -m scripts.pose2vid --config ./configs/inference/animation.yaml -W 600 -H 784 -L 120
/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
W0420 16:56:02.659374 238642 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 12.0, Runtime API Version: 11.8
W0420 16:56:02.660616 238642 gpu_resources.cc:164] device: 0, cuDNN Version: 8.9.
Some weights of the model checkpoint at runwayml/stable-diffusion-v1-5 were not used when initializing UNet2DConditionModel: ['conv_norm_out.bias', 'conv_norm_out.weight', 'conv_out.bias', 'conv_out.weight']
- This IS expected if you are initializing UNet2DConditionModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing UNet2DConditionModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
### missing keys: 0;
### unexpected keys: 0;
[2024-04-20 16:56:25,594] [ INFO] - Found /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/config.json
[2024-04-20 16:56:25,595] [ INFO] - Loading configuration file /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/config.json
[2024-04-20 16:56:25,596] [ INFO] - Model config CLIPVisionConfig {
"_name_or_path": "/home/jpinkney/.cache/huggingface/diffusers/models--lambdalabs--sd-image-variations-diffusers/snapshots/ca6f97f838ae1b5bf764f31363a21f388f4d8f3e/image_encoder",
"architectures": [
"CLIPVisionModelWithProjection"
],
"attention_dropout": 0.0,
"dropout": 0.0,
"hidden_act": "quick_gelu",
"hidden_size": 1024,
"image_size": 224,
"initializer_factor": 1.0,
"initializer_range": 0.02,
"intermediate_size": 4096,
"layer_norm_eps": 1e-05,
"model_type": "clip_vision_model",
"num_attention_heads": 16,
"num_channels": 3,
"num_hidden_layers": 24,
"paddlenlp_version": null,
"patch_size": 14,
"projection_dim": 768,
"return_dict": true,
"transformers_version": "4.25.1"
}
[2024-04-20 16:56:25,701] [ INFO] - Already cached /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/model_state.pdparams
[2024-04-20 16:56:25,702] [ INFO] - Loading weights file model_state.pdparams from cache at /home/aistudio/.cache/paddlenlp/ppdiffusers/lambdalabs/sd-image-variations-diffusers/image_encoder/model_state.pdparams
[2024-04-20 16:56:26,653] [ INFO] - Loaded weights file from disk, setting weights to model.
[2024-04-20 16:56:27,848] [ INFO] - All model checkpoint weights were used when initializing CLIPVisionModelWithProjection.
[2024-04-20 16:56:27,849] [ INFO] - All the weights of CLIPVisionModelWithProjection were initialized from the model checkpoint at lambdalabs/sd-image-variations-diffusers/image_encoder.
If your task is similar to the task the model of the checkpoint was trained on, you can already use CLIPVisionModelWithProjection for predictions without further training.
pose video has 390 frames, with 30 fps
0%| | 0/1 [00:00<?, ?it/s]W0420 16:57:11.692540 238642 multiply_fwd_func.cc:64] got different data type, run type protmotion automatically, this may cause data type been changed.
100%|█████████████████████████████████████████████| 1/1 [00:29<00:00, 29.16s/it]
100%|█████████████████████████████████████████| 120/120 [00:11<00:00, 10.30it/s]
6. 定制化角色动作
您可以依据自己喜欢的角色和动作,参考 animation.yaml 的格式添加自己的角色参考图像 ref_images
或动作视频 pose_videos
。要将原始视频转换为动作视频(关键点序列),可以运行以下命令:
In [12]
!python scripts/vid2pose.py --video_path ./configs/inference/raw_videos/tiktok1.mp4
2024-04-20 16:57:32.822096498 [W:onnxruntime:, session_state.cc:1162 VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2024-04-20 16:57:32.822160337 [W:onnxruntime:, session_state.cc:1164 VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
---VID2POSE DONE!