论文标题

在反馈协变量转移下的保形预测的生物分子设计

Conformal Prediction Under Feedback Covariate Shift for Biomolecular Design

论文作者

Fannjiang, Clara, Bates, Stephen, Angelopoulos, Anastasios N., Listgarten, Jennifer, Jordan, Michael I.

论文摘要

机器学习方法的许多应用都涉及一种迭代协议,其中收集数据,训练模型,然后使用该模型的输出来选择接下来要考虑的数据。例如,一种设计蛋白质的数据驱动方法是训练回归模型以预测蛋白质序列的适应性,然后使用它提出比训练数据中观察到的新序列的适应性。由于在湿实验室中验证设计的序列通常成本高昂,因此量化模型预测中的不确定性很重要。这是具有挑战性的,因为在设计设置中训练和测试数据之间的分布变化类型 - 训练和测试数据在统计上依赖于一种分布,因为后者是根据前者选择的。因此,模型在测试数据上的误差(即设计序列)与训练数据上的误差具有未知且可能复杂的关系。我们引入了一种量化此类设置中预测不确定性的方法。我们这样做是通过为预测构建置信集的置信集,以说明培训数据和测试数据之间的依赖性。我们构建的置信度集具有有限样本的保证,即使训练有素的模型选择测试时间输入分布,也可以保证任何预测算法。作为一种激励的用例,我们使用几个实际数据集证明了我们的方法如何量化设计蛋白质的预测适应性的不确定性,因此可以用于选择设计算法,这些设计算法在高预测的健身和低预测性不确定性之间实现可接受的权衡。

Many applications of machine learning methods involve an iterative protocol in which data are collected, a model is trained, and then outputs of that model are used to choose what data to consider next. For example, one data-driven approach for designing proteins is to train a regression model to predict the fitness of protein sequences, then use it to propose new sequences believed to exhibit greater fitness than observed in the training data. Since validating designed sequences in the wet lab is typically costly, it is important to quantify the uncertainty in the model's predictions. This is challenging because of a characteristic type of distribution shift between the training and test data in the design setting -- one in which the training and test data are statistically dependent, as the latter is chosen based on the former. Consequently, the model's error on the test data -- that is, the designed sequences -- has an unknown and possibly complex relationship with its error on the training data. We introduce a method to quantify predictive uncertainty in such settings. We do so by constructing confidence sets for predictions that account for the dependence between the training and test data. The confidence sets we construct have finite-sample guarantees that hold for any prediction algorithm, even when a trained model chooses the test-time input distribution. As a motivating use case, we demonstrate with several real data sets how our method quantifies uncertainty for the predicted fitness of designed proteins, and can therefore be used to select design algorithms that achieve acceptable trade-offs between high predicted fitness and low predictive uncertainty.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源