论文标题
熟练程度是语法误差校正中的质量估计
Proficiency Matters Quality Estimation in Grammatical Error Correction
论文作者
论文摘要
这项研究调查了语法误差校正（GEC）的监督质量估计（QE）模型如何受到学习者对数据的熟练程度的影响。在先前工作中进行GEC评估的QE模型与手动评估有很高的相关性。但是，当在现实世界中工作时，用于报告结果的数据存在局限性，因为先前的工作偏向于具有较高能力水平的学习者。为了解决此问题，我们创建了一个QE数据集，其中包含多个熟练程度，并探讨了对GEC量化量化量化质量标准进行能力评估的必要性。我们的实验表明，评估数据集的差异能力会影响量化宽松模型的性能，而熟练程度的评估有助于创建更健壮的模型。
This study investigates how supervised quality estimation (QE) models of grammatical error correction (GEC) are affected by the learners' proficiency with the data. QE models for GEC evaluations in prior work have obtained a high correlation with manual evaluations. However, when functioning in a real-world context, the data used for the reported results have limitations because prior works were biased toward data by learners with relatively high proficiency levels. To address this issue, we created a QE dataset that includes multiple proficiency levels and explored the necessity of performing proficiency-wise evaluation for QE of GEC. Our experiments demonstrated that differences in evaluation dataset proficiency affect the performance of QE models, and proficiency-wise evaluation helps create more robust models.
