论文标题
随机线性匪徒的残留自举勘探
Residual Bootstrap Exploration for Stochastic Linear Bandit
论文作者
论文摘要
我们为随机线性匪徒问题提出了一种新的基于自举的在线算法。关键的想法是采用残留的自举勘探，在这种探索中，代理通过重新采样平均奖励估算的残差来估算下一步奖励。我们的算法，随机线性匪徒（\ texttt {linreboot}）的残留bootstrap探索，从其重新采样分布中估算了线性奖励，并以最高的奖励估计拉动了手臂。特别是，我们为理论框架做出了一个理论框架，以使基于自举的探索机制在随机线性匪徒问题中脱颖而出。关键的见解是，Bootstrap探索的强度基于在线学习模型和残差的重新采样分布之间的乐观情绪。这样的观察使我们能够证明所提出的\ texttt {linreboot}确保了高概率$ \ tilde {o}（d \ sqrt {n}）$ sub-linear sublinear在温和条件下的遗憾。我们的实验支持\ texttt {Reboot}原理在线性匪徒问题的各种公式中的简单概括性，并显示了\ texttt {linreboot}的显着计算效率。
We propose a new bootstrap-based online algorithm for stochastic linear bandit problems. The key idea is to adopt residual bootstrap exploration, in which the agent estimates the next step reward by re-sampling the residuals of mean reward estimate. Our algorithm, residual bootstrap exploration for stochastic linear bandit (\texttt{LinReBoot}), estimates the linear reward from its re-sampling distribution and pulls the arm with the highest reward estimate. In particular, we contribute a theoretical framework to demystify residual bootstrap-based exploration mechanisms in stochastic linear bandit problems. The key insight is that the strength of bootstrap exploration is based on collaborated optimism between the online-learned model and the re-sampling distribution of residuals. Such observation enables us to show that the proposed \texttt{LinReBoot} secure a high-probability $\tilde{O}(d \sqrt{n})$ sub-linear regret under mild conditions. Our experiments support the easy generalizability of the \texttt{ReBoot} principle in the various formulations of linear bandit problems and show the significant computational efficiency of \texttt{LinReBoot}.
