Peter Carr Brooklyn Quant Experience (BQE) Seminar Series: Amine Mohamed Aboussalah

Title

Exploiting Structure in Reinforcement Learning to Mitigate Risk in Real-World Financial Control Problems

Abstract

Researchers have been applying a variety of machine learning tools to a variety of financial control problems. Reinforcement learning would seem to be an appropriate tool for portfolio management and related problems which involve interaction between an agent and an environment but the applications of this tool have been limited. We studied the challenges for the application of reinforcement learning to real-world problems and developed solutions for these challenges. Along the way, we developed a theoretical framework for understanding the performance of reinforcement learning for time series data which incorporates concepts from operations research. We found that incorporating prior knowledge as structure in the reinforcement learning algorithm improves performance. Our research identified four different ways of incorporating structure from the real world into control algorithms. One of the most important challenges for financial control problems is optimizing the trade-off between risk and return. We formulated risk as a structure in our reinforcement learning framework to encourage the learning algorithms to meet a risk goal or to balance risk and return. Adding financial structure helped improve learning leading to more reliable and sample efficient reinforcement learning algorithms with reduced variance, and better performance in terms of total return.

Bio

Amine Mohamed Aboussalah is an Industry Assistant Professor in the Department of Finance and Risk Engineering at the NYU Tandon School of Engineering. He earned his Ph.D. in Artificial Intelligence and Operations Research at the University of Toronto. His research interests lie broadly in artificial intelligence and dynamical systems. He enjoys applying theoretical mathematical concepts such as information geometry to develop new machine learning algorithms for a variety of practical real-world dynamical systems applications. He uses the financial application domain as a challenging real-world dynamical systems environment in which to advance reinforcement learning. Professor Aboussalah's primary research interest is improving reinforcement learning algorithms for solving and controlling dynamical systems by exploiting topological properties of time-series data and partial differential equations. As a teacher, he likes to mix theory and practice by sharing both his research and his industry experiences.