论文标题
通过数据辅助混合控制加速了连续的时间近似动态编程
Accelerated Continuous-Time Approximate Dynamic Programming via Data-Assisted Hybrid Control
论文作者
论文摘要
我们引入了一种新的闭环体系结构，以在线解决方案在连续时间系统的背景下近似最佳控制问题。具体而言，我们介绍了第一种算法，该算法结合了参与者批评结构中的动态动量，以控制输入中具有仿射结构的连续时间动态植物。通过将动态动量纳入我们的算法，我们能够加速闭环系统的收敛性能，与传统的基于梯度的技术相比，实现了优越的短暂性能。此外，通过利用过去记录的数据具有足够丰富的信息属性，我们将传统上强加于评论家和演员的回归者的激发条件的持续存在。鉴于我们连续的基于时间动量的动力学还结合了定期的离散时间重置，以模拟机器学习文献中使用的重新启动技术，因此我们利用混合动力学系统理论的工具来为闭环系统建立渐近稳定性属性。我们用数字示例来说明结果。
We introduce a new closed-loop architecture for the online solution of approximate optimal control problems in the context of continuous-time systems. Specifically, we introduce the first algorithm that incorporates dynamic momentum in actor-critic structures to control continuous-time dynamic plants with an affine structure in the input. By incorporating dynamic momentum in our algorithm, we are able to accelerate the convergence properties of the closed-loop system, achieving superior transient performance compared to traditional gradient-descent based techniques. In addition, by leveraging the existence of past recorded data with sufficiently rich information properties, we dispense with the persistence of excitation condition traditionally imposed on the regressors of the critic and the actor. Given that our continuous-time momentum-based dynamics also incorporate periodic discrete-time resets that emulate restarting techniques used in the machine learning literature, we leverage tools from hybrid dynamical systems theory to establish asymptotic stability properties for the closed-loop system. We illustrate our results with a numerical example.
