We propose a framework for certifying the robustness of Q-learning algorithms, which is notably the first that provides the robustness certification w.r.t. the cumulative reward; We propose two robustness certification criteria for Q-learning algorithms, together with corre- sponding certification algorithms based on global and local smoothing strategies; We theoretically prove the certification radius for input state and lower bound of perturbed cumula- tive reward under bounded adversarial state perturbations; We conduct extensive experiments to provide certification for nine empirically robust RL algorithms on multiple RL environments. We provide several interesting observations which would further inspire the development of robust RL algorithms.

We present a learning-based approach with pose perceptual loss for automatic music video generation.