论文标题
通过选择性可视化内部激活节点来解释深度时间表示
Interpretation of Deep Temporal Representations by Selective Visualization of Internally Activated Nodes
论文作者
论文摘要
最近，深度神经网络在许多时间或顺序数据的分类和回归任务中表现出了竞争性能。但是，仍然很难理解时间深神经网络的分类机制。在本文中，我们提出了两个新框架，以可视化从深神网络中学到的时间表示。给定的输入数据和输出，我们的算法通过提取高度激活的周期来解释时间神经网络的决定，并可视化输入数据的子序列，这有助于激活单位。此外，我们通过聚类来表征这种子序列，并计算建议类型和实际数据的不确定性。我们还建议从单元的输出而不是最终输出的层次相关性，而向后的蒙特卡洛辍学，以显示每个输入点的相关性得分，以激活单元，并提供有关这种影响的不确定性的可视化表示。
Recently deep neural networks demonstrate competitive performances in classification and regression tasks for many temporal or sequential data. However, it is still hard to understand the classification mechanisms of temporal deep neural networks. In this paper, we propose two new frameworks to visualize temporal representations learned from deep neural networks. Given input data and output, our algorithm interprets the decision of temporal neural network by extracting highly activated periods and visualizes a sub-sequence of input data which contributes to activate the units. Furthermore, we characterize such sub-sequences with clustering and calculate the uncertainty of the suggested type and actual data. We also suggest Layer-wise Relevance from the output of a unit, not from the final output, with backward Monte-Carlo dropout to show the relevance scores of each input point to activate units with providing a visual representation of the uncertainty about this impact.
