You're invited to attend
Marcus A. Pereira
(Advisor: Dr. Evangelos A. Theodorou)
"Scalable and Safe Deep Learning Architectures and Algorithms for Stochastic Optimal Control using Forward-Backward Stochastic Differential Equations"
Monday, November 1
10:00 am – 12:00 pm EST
Teams Meeting Link
Montgomery Knight Building, MK 317
Stochastic Optimal Control in continuous-time requires solving the Hamilton-Jacobi-Bellman equation which suffers from the well-known curse-of-dimensionality. Instead of directly attempting to solve the HJB, one can obtain probabilistic representations of the solution via the Nonlinear Feynman-Kac lemma that relates the unique solution of the HJB to that of a system of Foward-Backward Stochastic Differential Equations or FBSDEs. This work develops novel algorithms that leverage the function approximation capabilities of deep neural networks to solve FBSDEs and empirically demonstrates that the framework is immune to compounding errors in comparison to past FBSDE works and is therefore, scalable. The resulting deep FBSDE framework is memory efficient, provides temporally smoother controls and can be employed for long time-horizons owing to the underlying Long-Short Term Memory network architecture. Starting from a basic Stochastic Optimal Control problem, the framework is extended to problem formulations such as systems with control-multiplicative noise, systems with dynamics non-affine with respect to the control and for non-quadratic control cost functions, for safety critical tasks which employs stochastic control barrier functions and for L1 stochastic optimal control in minimum-fuel aerospace applications. Each problem formulation is accompanied with necessary structural modifications to the deep learning architecture to enable end-to-end learning. Two directions of future research are then proposed which, (i.) improve scalability and (ii.) improve applicability of the framework. The first employs the Alternating Direction Method of Multipliers to offer a scalable solution for large-scale safe multi-agent stochastic optimal control and the second direction aims to extend the framework to systems with unknown dynamics using recent techniques borrowed from model-based reinforcement learning.
- Dr. Evangelos A. Theodorou (Advisor) – Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology
- Dr. Enlu Zhou – H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology
- Dr. Samuel Coogan – School of Electrical and Computer Engineering, Georgia Institute of Technology
- Dr. Yongxin Chen – Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology
- Dr. Kyriakos G. Vamvoudakis – Daniel Guggenheim School of Aerospace Engineering, Georgia Institute of Technology
- Dr. Ioannis Exarchos - Microsoft