论文标题
使用无约束的启发来了解增强现实中的操纵任务的手势和言语多模式相互作用
Understanding Gesture and Speech Multimodal Interactions for Manipulation Tasks in Augmented Reality Using Unconstrained Elicitation
论文作者
论文摘要
这项研究可以更好地理解语音互动中的语法选择，以及如何使用增强现实中的无约束对象操纵环境中的用户产生语音，手势和多模式的手势和语音交互。这项工作提出了一项由24名参与者进行的多模式启发研究。用于翻译，旋转和比例尺的规范引用者与一些抽象的指南一起使用（创建，销毁和选择）。在这项研究的时间窗口中，使用手势和语音的开始和停止时间以及stoke的手势时代开发了用于手势和语音多模式相互作用的窗口。虽然手势通常是在81毫秒之前先于语音之前的，但我们发现手势的中风通常在演讲开始的10毫秒内。表明手势的信息内容及其同时发生的语音相互对齐。最后，研究了每种方式最常见的建议的趋势。表明提议之间的分歧通常是由手姿势或语法的变化引起的。允许我们提出混乱建议，以增加未来多模式交互系统捕获的用户自然相互作用的百分比。
This research establishes a better understanding of the syntax choices in speech interactions and of how speech, gesture, and multimodal gesture and speech interactions are produced by users in unconstrained object manipulation environments using augmented reality. The work presents a multimodal elicitation study conducted with 24 participants. The canonical referents for translation, rotation, and scale were used along with some abstract referents (create, destroy, and select). In this study time windows for gesture and speech multimodal interactions are developed using the start and stop times of gestures and speech as well as the stoke times for gestures. While gestures commonly precede speech by 81 ms we find that the stroke of the gesture is commonly within 10 ms of the start of speech. Indicating that the information content of a gesture and its co-occurring speech are well aligned to each other. Lastly, the trends across the most common proposals for each modality are examined. Showing that the disagreement between proposals is often caused by a variation of hand posture or syntax. Allowing us to present aliasing recommendations to increase the percentage of users' natural interactions captured by future multimodal interactive systems.
