论文标题
机器学习系统质量保证的理论和实践
Theory and Practice of Quality Assurance for Machine Learning Systems An Experiment Driven Approach
论文作者
论文摘要
基于机器学习的制作（ML）系统需要在整个生命周期中进行统计控制。仔细量化业务需求和影响业务需求的关键因素的识别会降低项目故障的风险。业务需求的量化导致定义随机变量，代表系统关键性能指标，需要通过统计实验来分析。此外，用于培训和实验结果的可用数据会影响系统的设计。一旦开发系统，就会对其进行测试并不断监控，以确保其满足其业务需求。这是通过继续应用统计实验来分析和控制关键绩效指标的。本书教授制作和开发基于ML的系统的艺术。它主张一种“实验第一”方法，强调从项目生命周期开始定义统计实验的必要性。它还详细讨论了如何在其整个生命周期中对基于ML的系统应用统计控制。
The crafting of machine learning (ML) based systems requires statistical control throughout its life cycle. Careful quantification of business requirements and identification of key factors that impact the business requirements reduces the risk of a project failure. The quantification of business requirements results in the definition of random variables representing the system key performance indicators that need to be analyzed through statistical experiments. In addition, available data for training and experiment results impact the design of the system. Once the system is developed, it is tested and continually monitored to ensure it meets its business requirements. This is done through the continued application of statistical experiments to analyze and control the key performance indicators. This book teaches the art of crafting and developing ML based systems. It advocates an "experiment first" approach stressing the need to define statistical experiments from the beginning of the project life cycle. It also discusses in detail how to apply statistical control on the ML based system throughout its lifecycle.
