Abstract

Reinforcement learning (RL) can be used to tune data-driven (economic)nonlinear model predictive controllers ((e)NMPCs) for optimal performance in aspecific control task by optimizing the dynamic model or parameters in thepolicy's objective function or constraints, such as state bounds. However, thesample efficiency of RL is crucial, and to improve it, we combine a model-basedRL algorithm with our published method that turns Koopman (e)NMPCs intoautomatically differentiable policies. We apply our approach to an eNMPC casestudy of a continuous stirred-tank reactor (CSTR) model from the literature.The approach outperforms benchmark methods, i.e., data-driven eNMPCs usingmodels based on system identification without further RL tuning of theresulting policy, and neural network controllers trained with model-based RL,by achieving superior control performance and higher sample efficiency.Furthermore, utilizing partial prior knowledge about the system dynamics viaphysics-informed learning further increases sample efficiency.