1. L. Cui, Z.P. Jiang, E.D. Sontag, and R.D. Braatz. Perturbed gradient descent algorithms are small-disturbance input-to-state stable. Automatica, 2025. Note: Submitted. Also arXiv:2507.02131. [PDF] [doi:https://doi.org/10.48550/arXiv.2507.02131] Keyword(s): Input-to-state stability (ISS), gradient systems, policy optimization, linear quadratic regulator (LQR). Abstract:

   This article investigates the robustness of gradient descent algorithms under perturbations. The concept of small-disturbance input-to-state stability (ISS) for discrete-time nonlinear dynamical systems is introduced, along with its Lyapunov characterization. The conventional linear \emph{Polyak-\L{}ojasiewicz} (PL) condition is then extended to a nonlinear version, and it is shown that the gradient descent algorithm is small-disturbance ISS provided the objective function satisfies the generalized nonlinear PL condition. This small-disturbance ISS property guarantees that the gradient descent algorithm converges to a small neighborhood of the optimum under sufficiently small perturbations. As a direct application of the developed framework, we demonstrate that the LQR cost satisfies the generalized nonlinear PL condition, thereby establishing that the policy gradient algorithm for LQR is small-disturbance ISS. Additionally, other popular policy gradient algorithms, including natural policy gradient and Gauss-Newton method, are also proven to be small-disturbance ISS.

This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders.

This document was translated from BibT_EX by bibtex2html