论文标题
对比的自我监督学习中的文本转换：评论
Text Transformations in Contrastive Self-Supervised Learning: A Review
论文作者
论文摘要
对比自我监督的学习已成为表示学习的重要技术。这些方法的主要步骤是对比鲜明的语义相似和不同的样品对。但是，在自然语言处理（NLP）的领域中，用于创建对比度学习（CL）假设的类似对的增强方法具有挑战性。这是因为，即使在输入中简单地修改单词也可能会改变句子的语义含义，因此也会违反分布假设。在本综述的论文中，我们正式化了对比度学习框架，强调需要在数据转换步骤中解决的考虑，并回顾NLP中对比度表示学习的最新方法和评估。最后，我们描述了使用对比方法学习更好的文本表示的一些挑战和潜在方向。
Contrastive self-supervised learning has become a prominent technique in representation learning. The main step in these methods is to contrast semantically similar and dissimilar pairs of samples. However, in the domain of Natural Language Processing (NLP), the augmentation methods used in creating similar pairs with regard to contrastive learning (CL) assumptions are challenging. This is because, even simply modifying a word in the input might change the semantic meaning of the sentence, and hence, would violate the distributional hypothesis. In this review paper, we formalize the contrastive learning framework, emphasize the considerations that need to be addressed in the data transformation step, and review the state-of-the-art methods and evaluations for contrastive representation learning in NLP. Finally, we describe some challenges and potential directions for learning better text representations using contrastive methods.
