论文标题
视频流中复杂事件检测的视觉语义多媒体事件模型
Visual Semantic Multimedia Event Model for Complex Event Detection in Video Streams
论文作者
论文摘要
多媒体数据具有很高的表现力，传统上对于机器来说很难解释。中间件系统，例如数据流中的复杂事件处理（CEP）矿场模式，并及时向用户发送通知。目前，CEP系统由于其数据复杂性和缺乏基础结构化数据模型而具有处理多媒体流的固有局限性。在这项工作中，我们提出了一种视觉事件规范方法，可以通过创建从低级媒体流派生的语义知识表示来启用复杂的多媒体事件处理。该方法可以使用模式检测功能集合从媒体流中检测高级语义概念。语义模型与多媒体CEP引擎深度学习模型对齐，以使最终用户使用时空事件计算构建规则。这增强了CEP的能力，可以从媒体流中检测模式，并在高度表达知识的用户查询与多媒体数据的低级特征之间弥合语义差距。我们建立了一个小型的交通事件本体原型来验证方法和性能。本文的贡献是三倍：i）我们提出了多媒体流的知识图表示，ii）一个分层事件网络，用于从媒体流中检测视觉模式和iii）使用事件计算为复杂多媒体事件定义复杂的模式规则
Multimedia data is highly expressive and has traditionally been very difficult for a machine to interpret. Middleware systems such as complex event processing (CEP) mine patterns from data streams and send notifications to users in a timely fashion. Presently, CEP systems have inherent limitations to process multimedia streams due to its data complexity and the lack of an underlying structured data model. In this work, we present a visual event specification method to enable complex multimedia event processing by creating a semantic knowledge representation derived from low-level media streams. The method enables the detection of high-level semantic concepts from the media streams using an ensemble of pattern detection capabilities. The semantic model is aligned with a multimedia CEP engine deep learning models to give flexibility to end-users to build rules using spatiotemporal event calculus. This enhances CEP capability to detect patterns from media streams and bridge the semantic gap between highly expressive knowledge-centric user queries to the low-level features of the multi-media data. We have built a small traffic event ontology prototype to validate the approach and performance. The paper contribution is threefold: i) we present a knowledge graph representation for multimedia streams, ii) a hierarchical event network to detect visual patterns from media streams and iii) define complex pattern rules for complex multimedia event reasoning using event calculus
