论文标题
向前靠背快速探索随机最佳控制的随机树
Forward-Backward Rapidly-Exploring Random Trees for Stochastic Optimal Control
论文作者
论文摘要
我们提出了一种数值方法，用于计算前向后的随机微分方程（FBSDE），该方程（FBSDE）出现在随机最佳控制问题中的值函数的Feynman-KAC表示中。通过使用Girsanov的变化概率度量，只要在向后集成通行证中适当补偿了迅速探索的随机树（RRT）方法，即可用于向前集成通行证。随后，通过沿构造的RRT的边缘向后求后向后求解一系列函数近似问题，提出了值函数的数值近似。此外，开发了局部熵加权最小二乘蒙特卡洛（LSMC）方法，以浓缩功能近似精度，最有可能通过最佳控制的轨迹访问。提出的方法的结果是在非二次运行成本的线性和非线性随机最佳控制问题上证明的，这揭示了对以前基于FBSDE的数值解决方案方法的显着收敛改善。
We propose a numerical method for the computation of the forward-backward stochastic differential equations (FBSDE) appearing in the Feynman-Kac representation of the value function in stochastic optimal control problems. By the use of the Girsanov change of probability measures, it is demonstrated how a rapidly-exploring random tree (RRT) method can be utilized for the forward integration pass, as long as the controlled drift terms are appropriately compensated in the backward integration pass. Subsequently, a numerical approximation of the value function is proposed by solving a series of function approximation problems backwards in time along the edges of the constructed RRT. Moreover, a local entropy-weighted least squares Monte Carlo (LSMC) method is developed to concentrate function approximation accuracy in regions most likely to be visited by optimally controlled trajectories. The results of the proposed methodology are demonstrated on linear and nonlinear stochastic optimal control problems with non-quadratic running costs, which reveal significant convergence improvements over previous FBSDE-based numerical solution methods.
