论文标题
通过边际可能性最大化指导神经网络初始化
Guiding Neural Network Initialization via Marginal Likelihood Maximization
论文作者
论文摘要
我们提出了一种简单的数据驱动方法，以帮助指导神经网络初始化的超参数选择。我们利用具有相应激活和协方差函数的神经网络和高斯过程模型之间的关系来推断模型初始化所需的超参数值。我们的实验表明，在实验约束下，边际可能性最大化提供了建议在MNIST分类任务上产生几乎最佳的预测性能。此外，我们的经验结果表明了所提出的技术的一致性，表明该过程的计算成本可以大大降低，而较小的训练集可以大大降低。
We propose a simple, data-driven approach to help guide hyperparameter selection for neural network initialization. We leverage the relationship between neural network and Gaussian process models having corresponding activation and covariance functions to infer the hyperparameter values desirable for model initialization. Our experiment shows that marginal likelihood maximization provides recommendations that yield near-optimal prediction performance on MNIST classification task under experiment constraints. Furthermore, our empirical results indicate consistency in the proposed technique, suggesting that computation cost for the procedure could be significantly reduced with smaller training sets.
