论文标题
预训练的语言模型可以是完全零摄的学习者
Pre-trained Language Models Can be Fully Zero-Shot Learners
论文作者
论文摘要
我们如何将预训练的模型扩展到许多语言理解任务，而无需标记或其他未标记数据？预训练的语言模型（PLM）对广泛的NLP任务有效。但是，现有的方法要么需要在标记的数据集上进行微调或手动构建适当的提示。在本文中，我们提出了非参数提示PLM（NPPROMPT），以完全零击语言理解。与以前的方法不同，NPPROMPT仅使用预训练的语言模型，并且不需要任何标记的数据或其他RAW语料库进行进一步的微调，也不依赖于人类来构建一组全面的提示单词。我们对以前的主要几杆和零射门学习方法评估了NPPROMP，这些方法在不同的NLP任务上进行了评估：包括文本分类，文本需要，类似的文本检索和释义。实验结果表明，我们的NPPROMPT优于先前最佳的全零射击方法，而文本分类的准确性的绝对优势为12.8％，在胶水基准上获得了18.9％。
How can we extend a pre-trained model to many language understanding tasks, without labeled or additional unlabeled data? Pre-trained language models (PLMs) have been effective for a wide range of NLP tasks. However, existing approaches either require fine-tuning on downstream labeled datasets or manually constructing proper prompts. In this paper, we propose nonparametric prompting PLM (NPPrompt) for fully zero-shot language understanding. Unlike previous methods, NPPrompt uses only pre-trained language models and does not require any labeled data or additional raw corpus for further fine-tuning, nor does it rely on humans to construct a comprehensive set of prompt label words. We evaluate NPPrompt against previous major few-shot and zero-shot learning methods on diverse NLP tasks: including text classification, text entailment, similar text retrieval, and paraphrasing. Experimental results demonstrate that our NPPrompt outperforms the previous best fully zero-shot method by big margins, with absolute gains of 12.8% in accuracy on text classification and 18.9% on the GLUE benchmark.
