论文标题
终身测试时间适应的概率框架
A Probabilistic Framework for Lifelong Test-Time Adaptation
论文作者
论文摘要
测试时间适应（TTA）是从不同目标域在推理时间内更新预训练的源模型的问题。大多数现有的TTA方法都假定目标域是静止的设置，即所有测试输入均来自单个目标域。但是，在许多实际设置中，测试输入分布可能会随着时间的流逝而表现出终身/持续的变化。此外，现有的TTA方法还缺乏提供可靠的不确定性估计值的能力，当源和目标域之间的分布变化发生时，这至关重要。为了解决这些问题，我们提出了花瓣（概率的终身测试时间适应以自我培训的先验），它使用概率方法求解了终生TTA，并且自然会导致（1）（1）一个教师模型，其中教师模型是学生模型的指数移动平均值，以及（2）使用定期型号的模型将模型定于定期模型。为了防止在终生/连续TTA设置中的模型漂移，我们还提出了一种数据驱动的参数恢复技术，该技术有助于通过仅恢复无关参数来减少误差积累并保持最新域的知识。 In terms of predictive error rate as well as uncertainty based metrics such as Brier score and negative log-likelihood, our method achieves better results than the current state-of-the-art for online lifelong test-time adaptation across various benchmarks, such as CIFAR-10C, CIFAR-100C, ImageNetC, and ImageNet3DCC datasets.我们方法的源代码可在https://github.com/dhanajitb/petal上访问。
Test-time adaptation (TTA) is the problem of updating a pre-trained source model at inference time given test input(s) from a different target domain. Most existing TTA approaches assume the setting in which the target domain is stationary, i.e., all the test inputs come from a single target domain. However, in many practical settings, the test input distribution might exhibit a lifelong/continual shift over time. Moreover, existing TTA approaches also lack the ability to provide reliable uncertainty estimates, which is crucial when distribution shifts occur between the source and target domain. To address these issues, we present PETAL (Probabilistic lifElong Test-time Adaptation with seLf-training prior), which solves lifelong TTA using a probabilistic approach, and naturally results in (1) a student-teacher framework, where the teacher model is an exponential moving average of the student model, and (2) regularizing the model updates at inference time using the source model as a regularizer. To prevent model drift in the lifelong/continual TTA setting, we also propose a data-driven parameter restoration technique which contributes to reducing the error accumulation and maintaining the knowledge of recent domains by restoring only the irrelevant parameters. In terms of predictive error rate as well as uncertainty based metrics such as Brier score and negative log-likelihood, our method achieves better results than the current state-of-the-art for online lifelong test-time adaptation across various benchmarks, such as CIFAR-10C, CIFAR-100C, ImageNetC, and ImageNet3DCC datasets. The source code for our approach is accessible at https://github.com/dhanajitb/petal.
