论文标题
网络实验
Network experimentation at scale
论文作者
论文摘要
我们描述了在Facebook部署的框架，这些框架通过群集随机实验来解释实验单元之间的干扰。我们记录了该系统，包括设计和估计程序以及我们从大规模使用该系统的许多实验中获得的详细见解。我们引入了基于群集的回归调整，该调整可大大提高估计全球治疗效果的精度，并在我们的估计程序中测试干扰。通过这种回归调整，我们发现不平衡的簇可以比平衡簇更好地解释干扰，而无需牺牲准确性。此外，我们还展示了如何使用日志记录接触治疗，以减少额外的方差。干扰是在线现场实验中广泛认可的问题，但是实际上证明了在线设置中干扰的现实世界实验的证据较少。我们通过描述两个捕获重大网络效应并突出该实验框架的价值的案例研究来填补这一空白。
We describe our framework, deployed at Facebook, that accounts for interference between experimental units through cluster-randomized experiments. We document this system, including the design and estimation procedures, and detail insights we have gained from the many experiments that have used this system at scale. We introduce a cluster-based regression adjustment that substantially improves precision for estimating global treatment effects as well as testing for interference as part of our estimation procedure. With this regression adjustment, we find that imbalanced clusters can better account for interference than balanced clusters without sacrificing accuracy. In addition, we show how logging exposure to a treatment can be used for additional variance reduction. Interference is a widely acknowledged issue with online field experiments, yet there is less evidence from real-world experiments demonstrating interference in online settings. We fill this gap by describing two case studies that capture significant network effects and highlight the value of this experimentation framework.
