论文标题
社会偏见符合数据偏见：标签和测量错误对公平标准的影响
Social Bias Meets Data Bias: The Impacts of Labeling and Measurement Errors on Fairness Criteria
论文作者
论文摘要
尽管已经提出了许多公平的标准，以确保机器学习算法不会表现出或扩大我们现有的社会偏见，但这些算法是在统计上偏见的数据集中培训的。在本文中，我们研究了算法对偏见的数据进行培训时，许多现有（人口）公平标准的鲁棒性。我们考虑了两种形式的数据集偏见：在标签过程中先前的决策者遇到的错误，以及在弱势群体的特征测量中的错误。我们在分析上表明，在面对某些统计偏见时，某些约束（例如人口统计学奇偶校验）可以保持健壮，而如果接受有偏见的数据培训，则会显着违反其他约束（例如均衡的赔率）。我们还分析了这些标准的敏感性以及决策者对偏见的效用。我们根据三个现实世界数据集（FICO，成人和德国信用评分数据集）提供数值实验，以支持我们的分析结果。我们的发现提出了在现有公平标准之间选择的其他指南，或者在可用数据集时提出新标准。
Although many fairness criteria have been proposed to ensure that machine learning algorithms do not exhibit or amplify our existing social biases, these algorithms are trained on datasets that can themselves be statistically biased. In this paper, we investigate the robustness of a number of existing (demographic) fairness criteria when the algorithm is trained on biased data. We consider two forms of dataset bias: errors by prior decision makers in the labeling process, and errors in measurement of the features of disadvantaged individuals. We analytically show that some constraints (such as Demographic Parity) can remain robust when facing certain statistical biases, while others (such as Equalized Odds) are significantly violated if trained on biased data. We also analyze the sensitivity of these criteria and the decision maker's utility to biases. We provide numerical experiments based on three real-world datasets (the FICO, Adult, and German credit score datasets) supporting our analytical findings. Our findings present an additional guideline for choosing among existing fairness criteria, or for proposing new criteria, when available datasets may be biased.
