论文标题
基于注意的随机森林和污染模型
Attention-based Random Forest and Contamination Model
论文作者
论文摘要
提出了一种称为ABRF的新方法（基于注意的随机森林）及其将注意机制应用于随机森林（RF）进行回归和分类的修改。拟议的ABRF模型背后的主要思想是以特定方式为决策树分配带有可训练参数的注意力。权重取决于一个实例之间的距离，该实例落入了树的相应叶子和实例，该实例落在同一叶子中。这个想法源于nadaraya-watson内核回归的形式。提出了一般方法的三个修改。第一个是基于应用Huber的污染模型以及通过解决二次或线性优化问题来计算注意力权重的。第二和第三修改使用基于梯度的算法来计算可训练的参数。具有各种回归和分类数据集的数值实验说明了提出的方法。
A new approach called ABRF (the attention-based random forest) and its modifications for applying the attention mechanism to the random forest (RF) for regression and classification are proposed. The main idea behind the proposed ABRF models is to assign attention weights with trainable parameters to decision trees in a specific way. The weights depend on the distance between an instance, which falls into a corresponding leaf of a tree, and instances, which fall in the same leaf. This idea stems from representation of the Nadaraya-Watson kernel regression in the form of a RF. Three modifications of the general approach are proposed. The first one is based on applying the Huber's contamination model and on computing the attention weights by solving quadratic or linear optimization problems. The second and the third modifications use the gradient-based algorithms for computing trainable parameters. Numerical experiments with various regression and classification datasets illustrate the proposed method.
