论文标题
基于反事实的基于监督的信息瓶颈，用于分布概括
Counterfactual Supervision-based Information Bottleneck for Out-of-Distribution Generalization
论文作者
论文摘要
最近的学习不变（因果）特征（OOD）概括最近引起了广泛的关注，在建议中，不变风险最小化（IRM）是一个显着的解决方案。尽管其对线性回归的理论希望，但在线性分类问题中使用IRM的挑战仍然存在。通过将信息瓶颈（IB）原理引入IRM的学习，IB-IRS方法证明了其解决这些挑战的能力。在本文中，我们从两个方面进一步改善了IB-IRM。首先，我们表明，IB-IRM中使用的不变特征的支持重叠的关键假设是为了保证OOD泛化，并且在没有此假设的情况下仍然可以实现最佳解决方案。其次，我们说明了两种故障模式，IB-IRM（和IRM）可能无法学习不变功能，并且为了解决此类失败，我们提出了一个\ textIt {基于反事实监督的信息瓶颈（CSIB）}学习算法，这些算法可以证明可以恢复不变的功能。通过需要反事实推断，CSIB即使从单个环境访问数据时也起作用。几个数据集的经验实验验证了我们的理论结果。
Learning invariant (causal) features for out-of-distribution (OOD) generalization has attracted extensive attention recently, and among the proposals invariant risk minimization (IRM) is a notable solution. In spite of its theoretical promise for linear regression, the challenges of using IRM in linear classification problems remain. By introducing the information bottleneck (IB) principle into the learning of IRM, IB-IRM approach has demonstrated its power to solve these challenges. In this paper, we further improve IB-IRM from two aspects. First, we show that the key assumption of support overlap of invariant features used in IB-IRM is strong for the guarantee of OOD generalization and it is still possible to achieve the optimal solution without this assumption. Second, we illustrate two failure modes that IB-IRM (and IRM) could fail for learning the invariant features, and to address such failures, we propose a \textit{Counterfactual Supervision-based Information Bottleneck (CSIB)} learning algorithm that provably recovers the invariant features. By requiring counterfactual inference, CSIB works even when accessing data from a single environment. Empirical experiments on several datasets verify our theoretical results.
