论文标题
通过应用于Covid-19分层的潜在偏移的多模式解释性
Multimodal Explainability via Latent Shift applied to COVID-19 stratification
论文作者
论文摘要
我们目睹了医疗保健中人工智能的广泛采用。但是，该领域深度学习的大多数进步仅考虑单峰数据，忽略了其他模式。他们的多模式解释对于支持诊断，预后和治疗决策所需的多模式解释。在这项工作中，我们提出了一个深度的体系结构，该体系结构共同学习了使用表格和成像数据的模态重建和样本分类。采取的决定的解释是通过应用潜在偏移来计算的，该偏移模拟了反事实预测，该预测揭示了每种方式的特征，这些特征对决策贡献了最大的作用，并指示了表明方式重要性的定量分数。我们使用Aiforcovid数据集在Covid-19大流行的背景下验证了我们的方法，该数据集包含多模式数据，用于早期鉴定有严重预后风险的患者。结果表明，所提出的方法提供了有意义的解释，而不会降低分类性能。
We are witnessing a widespread adoption of artificial intelligence in healthcare. However, most of the advancements in deep learning in this area consider only unimodal data, neglecting other modalities. Their multimodal interpretation necessary for supporting diagnosis, prognosis and treatment decisions. In this work we present a deep architecture, which jointly learns modality reconstructions and sample classifications using tabular and imaging data. The explanation of the decision taken is computed by applying a latent shift that, simulates a counterfactual prediction revealing the features of each modality that contribute the most to the decision and a quantitative score indicating the modality importance. We validate our approach in the context of COVID-19 pandemic using the AIforCOVID dataset, which contains multimodal data for the early identification of patients at risk of severe outcome. The results show that the proposed method provides meaningful explanations without degrading the classification performance.
