在街景房屋号码数据集(SVHN)(Netzer et al., 2011)上,当标记数据稀缺时,我们使用DCGAN的鉴别器的特征进行监督目的。按照CIFAR-10实验中的类似数据集准备规则,我们从非额外集合中分离出一个包含10,000个样本的验证集,并用它进行所有超参数和模型选择。随机选择1000个均匀分布的类别训练样本,并在CIFAR-10上使用的同样的特征提取流程上训练一个正则化的线性L2-SVM分类器。这达到了22.48%的测试误差,改进了另一种旨在利用未标记数据的CNNs的修改方法(Zhao et al., 2015)。此外,我们验证了在DCGAN中使用的CNN架构不是模型性能的主要贡献因素,通过在同样的数据上训练一个纯粹的监督CNN,并使用相同的架构,通过对64个超参数试验进行随机搜索优化这个模型(Bergstra & Bengio, 2012)。它达到了更高的28.87%的验证误差。
这些演示表明,可以使用我们的模型学习到的Z表示来开发有趣的应用。已经有之前的证明,条件生成模型可以学习说服力地模拟对象属性,如尺度、旋转和位置 (Dosovitskiy et al., 2014)。据我们所知,这是首次在纯无监督模型中出现的演示。进一步探索和开发上述向量算术可能会大大减少条件生成建模复杂图像分布所需的数据量。
Bergstra, James & Bengio, Yoshua. (2012). Random search for hyper-parameter optimization. JMLR.
Coates, Adam & Ng, Andrew. (2011). Selecting receptive fields in deep networks. NIPS.
Coates, Adam & Ng, Andrew Y. (2012). Learning feature representations with k-means. In Neural Networks: Tricks of the Trade (pp. 561--580). Springer.
Deng, Jia, Dong, Wei, Socher, Richard, Li, Li-Jia, Li, Kai, & Fei-Fei, Li. (2009). Imagenet: A large-scale hierarchical image database. In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on (pp. 248--255). IEEE.
Denton, Emily, Chintala, Soumith, Szlam, Arthur, & Fergus, Rob. (2015). Deep generative image models using a laplacian pyramid of adversarial networks. arXiv preprint arXiv:1506.05751.
Dosovitskiy, Alexey, Springenberg, Jost Tobias, & Brox, Thomas. (2014). Learning to generate chairs with convolutional neural networks. arXiv preprint arXiv:1411.5928.
Dosovitskiy, Alexey et al. (2015). Discriminative unsupervised feature learning with exemplar convolutional neural networks. Pattern Analysis and Machine Intelligence, IEEE Transactions on, volume 99. IEEE.
Efros, Alexei et al. (1999). Texture synthesis by non-parametric sampling. In Computer Vision, The Proceedings of the Seventh IEEE International Conference on, volume 2, pp. 1033--1038. IEEE.
Freeman, William T. et al. (2002). Example-based super-resolution. Computer Graphics and Applications, IEEE, 22(2):56--65.
Goodfellow, Ian J. et al. (2013). Maxout networks. arXiv preprint arXiv:1302.4389.
Goodfellow, Ian J. et al. (2014). Generative adversarial nets. NIPS.
Gregor, Karol et al. (2015). Draw: A recurrent neural network for image generation. arXiv preprint arXiv:1502.04623.
Hardt, Moritz et al. (2015). Train faster, generalize better: Stability of stochastic gradient descent. arXiv preprint arXiv:1509.01240.
Hauberg, Sren et al. (2015). Dreaming more data: Class-dependent distributions over diffeomorphisms for learned data augmentation. arXiv preprint arXiv:1510.02795.
Hays, James & Efros, Alexei A. (2007). Scene completion using millions of photographs. ACM Transactions on Graphics (TOG), 26(3):4.
Ioffe, Sergey & Szegedy, Christian. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167.
Kingma, Diederik P. & Ba, Jimmy Lei. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Lee, Honglak et al. (2009). Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th Annual International Conference on Machine Learning, pp. 609--616. ACM.
Loosli, Gaëlle et al. (2007). Training invariant support vector machines using selective sampling. In Large Scale Kernel Machines, pp. 301--320. MIT Press, Cambridge, MA.
Maas, Andrew L. et al. (2013). Rectifier nonlinearities improve neural network acoustic models. Proc. ICML, volume 30.
Mikolov, Tomas et al. (2013). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pp. 3111--3119.
Mordvintsev, Alexander et al. Inceptionism: Going deeper into neural networks. Google Research Blog. [Online]. Accessed: 2015-06-17.
Nair, Vinod & Hinton, Geoffrey E. (2010). Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on Machine Learning (ICML-10), pp. 807--814.
Netzer, Yuval et al. (2011). Reading digits in natural images with unsupervised feature learning. In NIPS workshop on deep learning and unsupervised feature learning, volume 2011, pp. 5. Granada, Spain.
Oquab, M. et al. (2014). Learning and transferring mid-level image representations using convolutional neural networks. In CVPR.
Portilla, Javier & Simoncelli, Eero P. (2000). A parametric texture model based on joint statistics of complex wavelet coefficients. International Journal of Computer Vision, 40(1):49--70.
Rasmus, Antti et al. (2015). Semi-supervised learning with ladder network. arXiv preprint arXiv:1507.02672.
Sohl-Dickstein, Jascha et al. (2015). Deep unsupervised learning using nonequilibrium thermodynamics. arXiv preprint arXiv:1503.03585.
Springenberg, Jost Tobias et al. (2014). Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806.
Theis, L. et al. (2015). A note on the evaluation of generative models. arXiv:1511.01844.
Vincent, Pascal et al. (2010). Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. The Journal of Machine Learning Research, 11:3371--3408.
Xu, Bing et al. (2015). Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.00853.
Yu, Fisher et al. (2015). Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365.
Zeiler, Matthew D & Fergus, Rob. (2014). Visualizing and understanding convolutional networks. In Computer Vision--ECCV 2014, pp. 818--833. Springer.