论文标题
在线受控实验中的统计挑战：A/B测试方法的综述
Statistical Challenges in Online Controlled Experiments: A Review of A/B Testing Methodology
论文作者
论文摘要
1990年代后期，基于互联网的服务和产品的兴起为在线企业提供了一个前所未有的机会，可以从事大规模数据驱动的决策。在过去的二十年中，诸如Airbnb，Alibaba，Amazon，Baidu，预订，Alphabet的Google，LinkedIn，Lyft，Lyft，Meta的Facebook，Microsoft，Netflix，Twitter，Uber和Yandex等组织在线对在线对照实验（OCES（OCES）进行了巨大资源，以评估Innoving Innoving of Innoving他们的客户和商务公司。大规模运行OCE已提出了许多挑战，这些挑战需要许多领域的解决方案。在本文中，我们审查了需要新的统计方法来解决这些方法的挑战。特别是，我们讨论了在线实验的实践和文化及其统计文献，将当前的方法置于其相关统计谱系中，并提供了OCE应用的说明示例。我们的目标是提高学术统计学家对这些新的研究机会的认识，以增加学术界与在线行业之间的合作。
The rise of internet-based services and products in the late 1990's brought about an unprecedented opportunity for online businesses to engage in large scale data-driven decision making. Over the past two decades, organizations such as Airbnb, Alibaba, Amazon, Baidu, Booking, Alphabet's Google, LinkedIn, Lyft, Meta's Facebook, Microsoft, Netflix, Twitter, Uber, and Yandex have invested tremendous resources in online controlled experiments (OCEs) to assess the impact of innovation on their customers and businesses. Running OCEs at scale has presented a host of challenges requiring solutions from many domains. In this paper we review challenges that require new statistical methodologies to address them. In particular, we discuss the practice and culture of online experimentation, as well as its statistics literature, placing the current methodologies within their relevant statistical lineages and providing illustrative examples of OCE applications. Our goal is to raise academic statisticians' awareness of these new research opportunities to increase collaboration between academia and the online industry.
