论文标题
ROC对数据不平衡的局限性：LVAD死亡率风险评分评估
Limitations of ROC on Imbalanced Data: Evaluation of LVAD Mortality Risk Scores
论文作者
论文摘要
目的：这项研究说明了ROC在评估90天LVAD死亡率的两个分类器时的歧义。本文还引入了Precision召回曲线（PRC）作为补充指标，在预测少数群体时更代表LVAD分类器的表现。 背景：在LVAD域中，接收器操作特性（ROC）是分类器性能的通常指标。但是，ROC可以提供分类器的扭曲视图，可以预测由于多大比例生存的患者（即数据不平衡），因此可以预测短期死亡率。 方法：这项研究比较了800例LVAD死亡率的ROC和PRC在2006年至2016年之间获得连续流LVAD的800例患者（测试组）的两种分类器的结果（测试组）（平均年龄为59岁； 146名女性; 146个女性vs. 654男性）仅在90-At Patagation（Impalative fate）中（Immpalative for Date）（Immpalative）（Immpaland Palpate）。这两个分类器是心友风险评分（HMR）和一个随机森林（RF）。 结果：ROC表明RF和HRMS分类器的性能分别为0.77和0.63。这与他们的PRC相反，RF和HRM的AUC分别为0.43和0.16。 HRMS的中国表现出的精度迅速下降至仅10％，灵敏度略有提高。 结论：将ROC应用于不平衡数据时，可以描绘分类器或风险评分的过度表现。中国通过关注少数族裔阶级，可以更好地了解分类器的性能。
Objective: This study illustrates the ambiguity of ROC in evaluating two classifiers of 90-day LVAD mortality. This paper also introduces the precision recall curve (PRC) as a supplemental metric that is more representative of LVAD classifiers performance in predicting the minority class. Background: In the LVAD domain, the receiver operating characteristic (ROC) is a commonly applied metric of performance of classifiers. However, ROC can provide a distorted view of classifiers ability to predict short-term mortality due to the overwhelmingly greater proportion of patients who survive, i.e. imbalanced data. Methods: This study compared the ROC and PRC for the outcome of two classifiers for 90-day LVAD mortality for 800 patients (test group) recorded in INTERMACS who received a continuous-flow LVAD between 2006 and 2016 (mean age of 59 years; 146 females vs. 654 males) in which mortality rate is only %8 at 90-day (imbalanced data). The two classifiers were HeartMate Risk Score (HMRS) and a Random Forest (RF). Results: The ROC indicates fairly good performance of RF and HRMS classifiers with Area Under Curves (AUC) of 0.77 vs. 0.63, respectively. This is in contrast with their PRC with AUC of 0.43 vs. 0.16 for RF and HRMS, respectively. The PRC for HRMS showed the precision rapidly dropped to only 10% with slightly increasing sensitivity. Conclusion: The ROC can portray an overly-optimistic performance of a classifier or risk score when applied to imbalanced data. The PRC provides better insight about the performance of a classifier by focusing on the minority class.
