论文标题
带有临床笔记的死亡率预测模型在单词和句子水平上使用稀疏的注意力
Mortality Prediction Models with Clinical Notes Using Sparse Attention at the Word and Sentence Levels
论文作者
论文摘要
重症监护院内死亡率预测具有各种临床应用。神经预测模型，尤其是在利用临床笔记时，已经提出了对当前现有模型的改进。但是，可以接受这些模型应该是性能和透明的。这项工作研究了临床神经预测模型的不同关注机制，以歧视和校准。具体而言，我们研究了稀疏的注意力作为临床注释内院内死亡率预测的任务，替代了密集的注意力重量。我们基于以下方式评估注意力机制：i）句子中对单词的本地自我注意，ii）跨句子的变压器体系结构的全球自我注意。我们证明，稀疏的机制方法在预测性能和公开数据集方面，稀疏的机制方法优于当地自我注意的密集，并更加关注预先指定的相关指令词。然而，句子级别的表现会随着句子的恶化而恶化，包括有影响力的指令词，往往会丢弃。
Intensive Care in-hospital mortality prediction has various clinical applications. Neural prediction models, especially when capitalising on clinical notes, have been put forward as improvement on currently existing models. However, to be acceptable these models should be performant and transparent. This work studies different attention mechanisms for clinical neural prediction models in terms of their discrimination and calibration. Specifically, we investigate sparse attention as an alternative to dense attention weights in the task of in-hospital mortality prediction from clinical notes. We evaluate the attention mechanisms based on: i) local self-attention over words in a sentence, and ii) global self-attention with a transformer architecture across sentences. We demonstrate that the sparse mechanism approach outperforms the dense one for the local self-attention in terms of predictive performance with a publicly available dataset, and puts higher attention to prespecified relevant directive words. The performance at the sentence level, however, deteriorates as sentences including the influential directive words tend to be dropped all together.
