论文标题
合着者：设计人类协作写作数据集用于探索语言模型功能
CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities
论文作者
论文摘要
大型语言模型（LMS）提供了前所未有的语言发电能力和激动人心的互动设计机会。但是，它们高度依赖上下文的功能很难掌握，并且经常被主观解释。在本文中，我们认为，通过策划和分析大型交互数据集，HCI社区可以促进对LMS生成能力的更敏锐的检查。为了说明这种方法，我们介绍了合着者，该数据集旨在揭示GPT-3在协助创造性和争论性写作方面的功能。合着者在1445年的写作会议上捕捉了63位作家和四个GPT-3实例之间的丰富互动。我们证明合着者可以解决有关GPT-3的语言，构想和协作功能的问题，并在良好协作的各种定义下揭示其作为写作“合作者”的贡献。最后，我们讨论这项工作如何促进有关LMS与互动设计有关的诺言和陷阱的更有原则的讨论。用于重播写作会话的数据集和接口可在https://coauthor.stanford.edu上公开获得。
Large language models (LMs) offer unprecedented language generation capabilities and exciting opportunities for interaction design. However, their highly context-dependent capabilities are difficult to grasp and are often subjectively interpreted. In this paper, we argue that by curating and analyzing large interaction datasets, the HCI community can foster more incisive examinations of LMs' generative capabilities. Exemplifying this approach, we present CoAuthor, a dataset designed for revealing GPT-3's capabilities in assisting creative and argumentative writing. CoAuthor captures rich interactions between 63 writers and four instances of GPT-3 across 1445 writing sessions. We demonstrate that CoAuthor can address questions about GPT-3's language, ideation, and collaboration capabilities, and reveal its contribution as a writing "collaborator" under various definitions of good collaboration. Finally, we discuss how this work may facilitate a more principled discussion around LMs' promises and pitfalls in relation to interaction design. The dataset and an interface for replaying the writing sessions are publicly available at https://coauthor.stanford.edu.
