论文标题
UI层组检测器：通过文本融合和盒子注意分组UI层
UI Layers Group Detector: Grouping UI Layers via Text Fusion and Box Attention
论文作者
论文摘要
图形用户界面（GUI）在移动应用程序的普及和繁荣方面面临着巨大的需求。 UI设计草案中的自动UI代码生成极大地简化了开发过程。但是，设计草案中的嵌套结构会影响生成的代码的质量和可用性。现有的GUI自动化技术很少检测并将嵌套层分组以提高生成的代码的可访问性。在本文中，我们提出了UI层群检测器作为一种基于视觉的方法，该方法自动检测图像（即基本形状和视觉元素）和具有相同语义含义的文本层。我们提出了两个插件组件，即文本融合和盒子注意，它们利用设计草案中的文本信息作为组本地化的先验信息。我们构建了一个大规模的UI数据集用于培训和测试，并提供了一种数据增强方法来提高检测性能。该实验表明，所提出的方法在分组分组方面达到了不错的精度。
Graphic User Interface (GUI) is facing great demand with the popularization and prosperity of mobile apps. Automatic UI code generation from UI design draft dramatically simplifies the development process. However, the nesting layer structure in the design draft affects the quality and usability of the generated code. Few existing GUI automated techniques detect and group the nested layers to improve the accessibility of generated code. In this paper, we proposed our UI Layers Group Detector as a vision-based method that automatically detects images (i.e., basic shapes and visual elements) and text layers that present the same semantic meanings. We propose two plug-in components, text fusion and box attention, that utilize text information from design drafts as a priori information for group localization. We construct a large-scale UI dataset for training and testing, and present a data augmentation approach to boost the detection performance. The experiment shows that the proposed method achieves a decent accuracy regarding layers grouping.
