论文标题
NLU测试套件的快速调试
Fast Few-shot Debugging for NLU Test Suites
论文作者
论文摘要
我们使用最近普及的测试套件来研究基于变压器的自然语言理解模型的调试，不仅可以诊断，而且可以纠正问题。考虑到一些某种现象的调试示例，以及相同现象的持有测试集，我们旨在以原始测试集的准确性最低成本来最大程度地提高现象的准确性。我们检查了几种比完整时期再训练更快的方法。我们介绍了一种新的快速方法，该方法从原始训练集中采样了一些室内示例。与使用参数距离约束或kullback-leibler差异相比，我们实现了卓越的原始精度以实现可比的调试精度。
We study few-shot debugging of transformer based natural language understanding models, using recently popularized test suites to not just diagnose but correct a problem. Given a few debugging examples of a certain phenomenon, and a held-out test set of the same phenomenon, we aim to maximize accuracy on the phenomenon at a minimal cost of accuracy on the original test set. We examine several methods that are faster than full epoch retraining. We introduce a new fast method, which samples a few in-danger examples from the original training set. Compared to fast methods using parameter distance constraints or Kullback-Leibler divergence, we achieve superior original accuracy for comparable debugging accuracy.
