论文标题
人类反应产生的相互作用变压器
Interaction Transformer for Human Reaction Generation
论文作者
论文摘要
我们解决了人类反应产生的具有挑战性的任务，该任务旨在基于输入动作产生相应的反应。大多数现有作品都不专注于产生和预测反应，并且在仅作为输入给出动作时就无法产生运动。为了解决这一限制，我们提出了一种新型的相互作用变压器（Interformer），该变压器由具有时间和空间注意力的变压器网络组成。具体而言，时间关注捕获了字符及其相互作用运动的时间依赖性，而空间注意力则了解每个字符的不同身体部位与相互作用的一部分之间的依赖关系。此外，我们建议使用图形通过相互作用距离模块提高空间注意力的性能，从而有助于关注两个字符的附近关节。关于SBU相互作用，K3HI和Duetdance数据集的广泛实验证明了Interformer的有效性。我们的方法是一般的，可用于产生更复杂和长期的相互作用。我们还通过https://github.com/cristal-3dsam/interform提供了生成的反应和代码的视频和代码的视频
We address the challenging task of human reaction generation, which aims to generate a corresponding reaction based on an input action. Most of the existing works do not focus on generating and predicting the reaction and cannot generate the motion when only the action is given as input. To address this limitation, we propose a novel interaction Transformer (InterFormer) consisting of a Transformer network with both temporal and spatial attention. Specifically, temporal attention captures the temporal dependencies of the motion of both characters and of their interaction, while spatial attention learns the dependencies between the different body parts of each character and those which are part of the interaction. Moreover, we propose using graphs to increase the performance of spatial attention via an interaction distance module that helps focus on nearby joints from both characters. Extensive experiments on the SBU interaction, K3HI, and DuetDance datasets demonstrate the effectiveness of InterFormer. Our method is general and can be used to generate more complex and long-term interactions. We also provide videos of generated reactions and the code with pre-trained models at https://github.com/CRISTAL-3DSAM/InterFormer
