论文标题
基于Ernie预训练模型的双频道新闻标题分类的研究
Research on Dual Channel News Headline Classification Based on ERNIE Pre-training Model
论文作者
论文摘要
新闻头条的分类是NLP领域的重要方向,其数据具有紧凑性,独特性和各种形式的特征。针对一个问题,即传统的神经网络模型无法充分捕获数据的基本特征信息,并且无法共同提取关键的全球特征和深层本地特征,提出了基于Ernie预培训模型的双通道网络模型DC-EBAD。使用Ernie在文本的底部提取词汇,语义和上下文特征信息,生成与上下文融合的动态单词矢量表示,然后使用Bilstm-AT网络渠道来辅助数据的全局功能,并使用注意力机制来使关键部分更高,以更高的dpcnn渠道的重量来克服较长的文本依赖性问题。局部和全局特征向量被剪接,最后传递到完全连接的层,最终的分类结果通过SoftMax输出。实验结果表明,与传统的神经网络模型和在相同条件下的单渠道模型相比,提出的模型提高了新闻标题分类的准确性,精度和F1得分。可以看出,它可以在大型数据量的新闻标题文本的多分类应用中表现良好。
The classification of news headlines is an important direction in the field of NLP, and its data has the characteristics of compactness, uniqueness and various forms. Aiming at the problem that the traditional neural network model cannot adequately capture the underlying feature information of the data and cannot jointly extract key global features and deep local features, a dual-channel network model DC-EBAD based on the ERNIE pre-training model is proposed. Use ERNIE to extract the lexical, semantic and contextual feature information at the bottom of the text, generate dynamic word vector representations fused with context, and then use the BiLSTM-AT network channel to secondary extract the global features of the data and use the attention mechanism to give key parts higher The weight of the DPCNN channel is used to overcome the long-distance text dependence problem and obtain deep local features. The local and global feature vectors are spliced, and finally passed to the fully connected layer, and the final classification result is output through Softmax. The experimental results show that the proposed model improves the accuracy, precision and F1-score of news headline classification compared with the traditional neural network model and the single-channel model under the same conditions. It can be seen that it can perform well in the multi-classification application of news headline text under large data volume.