论文标题
块期张量神经网络
Block-term Tensor Neural Networks
论文作者
论文摘要
深层神经网络（DNNS）在广泛的应用中取得了出色的性能，例如，图像分类，自然语言处理等。尽管性能良好，DNNS的大量参数带来了有效培训DNN的挑战，并在低端设备中的部署具有有限的计算资源。在本文中，我们探讨了重量矩阵中的相关性，并与低级数量量张量近似重量矩阵。我们将新的相应结构命名为扩展张量层（BT层），可以很容易地适应神经网络模型，例如CNNS和RNN。特别是，BT层中的输入和输出被重塑为具有相似或改进的表示功率的低维高量张量。足够的实验表明，CNN和RNN中的BT层可以在参数数量上达到非常大的压缩比，同时保留或提高原始DNN的表示能力。
Deep neural networks (DNNs) have achieved outstanding performance in a wide range of applications, e.g., image classification, natural language processing, etc. Despite the good performance, the huge number of parameters in DNNs brings challenges to efficient training of DNNs and also their deployment in low-end devices with limited computing resources. In this paper, we explore the correlations in the weight matrices, and approximate the weight matrices with the low-rank block-term tensors. We name the new corresponding structure as block-term tensor layers (BT-layers), which can be easily adapted to neural network models, such as CNNs and RNNs. In particular, the inputs and the outputs in BT-layers are reshaped into low-dimensional high-order tensors with a similar or improved representation power. Sufficient experiments have demonstrated that BT-layers in CNNs and RNNs can achieve a very large compression ratio on the number of parameters while preserving or improving the representation power of the original DNNs.
