论文标题
联合转向和对话级别的用户满意度估计多域对话
Joint Turn and Dialogue level User Satisfaction Estimation on Multi-Domain Conversations
论文作者
论文摘要
对话水平质量估计对于优化数据驱动的对话管理至关重要。当前的自动化方法估算转向和对话级别的用户满意度采用手工制作的功能并依赖复杂的注释方案，从而降低了训练有素的模型的普遍性。我们提出了一种新颖的用户满意度估计方法，该方法将自适应的多任务损失函数最小化，以便共同预测专家提供的转向级响应质量标签和最终用户提供的显式对话级别评级。拟议的基于Bilstm的深神经网模型会自动权衡每个转弯对估计的对话级评级的贡献，隐含地编码时间依赖性，并消除对手工艺特征的需求。 在从28个Alexa域，两个对话系统和三个用户组中采样的对话中，联合对话级满意度估计模型的实现最高可达到27％（0.43-> 0.70）和7％（0.63-> 0.70），而基线深度net和Benchmark Markmmark Boostient Boostient BoostInient BoostInient BoostInient Boosting Recormention Comprientation Adressional contression contression contression contression corts Intor。
Dialogue level quality estimation is vital for optimizing data driven dialogue management. Current automated methods to estimate turn and dialogue level user satisfaction employ hand-crafted features and rely on complex annotation schemes, which reduce the generalizability of the trained models. We propose a novel user satisfaction estimation approach which minimizes an adaptive multi-task loss function in order to jointly predict turn-level Response Quality labels provided by experts and explicit dialogue-level ratings provided by end users. The proposed BiLSTM based deep neural net model automatically weighs each turn's contribution towards the estimated dialogue-level rating, implicitly encodes temporal dependencies, and removes the need to hand-craft features. On dialogues sampled from 28 Alexa domains, two dialogue systems and three user groups, the joint dialogue-level satisfaction estimation model achieved up to an absolute 27% (0.43->0.70) and 7% (0.63->0.70) improvement in linear correlation performance over baseline deep neural net and benchmark Gradient boosting regression models, respectively.
