c++ cudnn模仿resnet18训练cifar10（70分）

使用自己的实线残差，yolo v3改造的虚线残差，模仿resnet18，上了70分，有点小成绩了！

架构如下：

layers.emplace_back(std::make_shared<Conv2D>(cudnn, batch, 5, 32, 32, 32, 3, 1, 1));

//第一残差开始

layers.emplace_back(std::make_shared<residualExt22>(cudnn, batch, 32, 32, 32));

//第二残差开始

layers.emplace_back(std::make_shared<Conv2D>(cudnn, batch, 32, 64, 32, 32, 3, 1, 1));

layers.emplace_back(std::make_shared<residualExt22>(cudnn, batch, 64, 32, 32));

//第三残差开始，虚线 //xuxianResidualyolo

layers.emplace_back(std::make_shared<xuxianResidual2>(cudnn, batch, 64, 32, 32));//out->batch, 128, 16, 16

//第四残差开始，虚线

layers.emplace_back(std::make_shared<Conv2D>(cudnn, batch, 128, 128, 16, 16, 3, 1, 1));

layers.emplace_back(std::make_shared<xuxianResidual2>(cudnn, batch, 128, 16, 16));//out->batch, 256, 8, 8

////第五残差开始，虚线

layers.emplace_back(std::make_shared<Conv2D>(cudnn, batch, 256, 256, 8, 8, 3, 1, 1));

layers.emplace_back(std::make_shared<xuxianResidual2>(cudnn, batch, 256, 8, 8));//out->batch, 512, 4, 4

//第六残差开始

layers.emplace_back(std::make_shared<residualExt22>(cudnn, batch, 512, 4, 4));

layers.emplace_back(std::make_shared<averPool2D>(cudnn, batch, 512, 4,4, 2, 2, 0, 2));

layers.emplace_back(std::make_shared<Linear>(cublas, batch, 512 * 4, 384));

layers.emplace_back(std::make_shared<BN>(cudnn, batch, 384, 1, 1));

layers.emplace_back(std::make_shared<LeakyRL>(cudnn, batch, 384, 1, 1));

layers.emplace_back(std::make_shared<Dropout>(cudnn, batch, 384, 1, 1));//=.5

layers.emplace_back(std::make_shared<Linear>(cublas, batch, 384, 10));

三个实线配合三个虚线残差，还不错！今天跨过70分了！

唯一缺点，就是训练的次数有点多！

轮次：29

learn rate：0.0001

时间: 29732.675781 ms

train Classification result: 69.61% ok (used 49984 images)

时间: 2174.239990 ms

Test Classification result: 10.00% ok (used 9984 images)

轮次：30

learn rate：0.0001

时间: 29830.599609 ms

train Classification result: 70.16% ok (used 49984 images)

时间: 2154.638916 ms

Test Classification result: 9.99% ok (used 9984 images)

轮次：31

learn rate：1e-05

时间: 29786.416016 ms

train Classification result: 72.20% ok (used 49984 images)

时间: 2146.649902 ms

Test Classification result: 69.66% ok (used 9984 images)

轮次：32

learn rate：1e-05

时间: 29754.357422 ms

train Classification result: 73.03% ok (used 49984 images)

时间: 2196.406006 ms

Test Classification result: 68.80% ok (used 9984 images)

轮次：33

learn rate：1e-05

时间: 29806.121094 ms

train Classification result: 73.36% ok (used 49984 images)

时间: 2188.530029 ms

Test Classification result: 68.99% ok (used 9984 images)

轮次：34

learn rate：1e-06

时间: 29792.988281 ms

train Classification result: 73.93% ok (used 49984 images)

时间: 2174.410889 ms

Test Classification result: 70.85% ok (used 9984 images)

轮次：35

learn rate：1e-06

时间: 29842.667969 ms

train Classification result: 73.96% ok (used 49984 images)

时间: 2187.137939 ms

Test Classification result: 70.86% ok (used 9984 images)

轮次：36

learn rate：1e-06

时间: 29772.611328 ms

train Classification result: 73.91% ok (used 49984 images)

时间: 2171.908936 ms

Test Classification result: 70.78% ok (used 9984 images)

学习率用0.001，训练大于40分，lr=0.0001，训练大于70，lr=0.00001；

分析这个结果，learn rate：1e-05跑了三次，少了，应该多跑，

到了learn rate：1e-06，并不增长！

先做个记录！