FRE Special Seminar: Anaïs Després & Zhenjie Ren

4:00 p.m.

Anaïs Després

Title

Specific Entropic Martingale Optimal Transport: From Duality to Mirror Flow

Abstract

Classical entropy regularization is not well adapted to continuous-time martingale transport because relative entropy between diffusion laws constrains volatility dynamics. We propose a specific-entropy framework based on Poisson jump approximations of continuous martingales. In the Gaussian-mark setting, this leads to explicit entropy functionals on Wiener space whose limits depend on both the martingale law and the microscopic approximation scheme. We then prove a strong duality result and use it to derive a Sinkhorn-type method based on coupled HJB/FP equations. A mirror-flow formulation reveals exponential contraction of the ideal terminal marginal, while a practical linearized scheme admits local tracking error bounds. Numerical experiments in one and two dimensions illustrate the approach and demonstrate substantial computational gains over gradient flow.

5:00 p.m.

Zhenjie Ren

Professor at LaMME, Université Évry Paris-Saclay

Title

Policy gradient flow for stochastic control

Abstract

Reinforcement learning has achieved remarkable success in landmark applications such as AlphaGo and AlphaFold, and is increasingly expected to play a central role in emerging domains such as autonomous driving. A substantial body of work has established theoretical foundations for classical reinforcement learning algorithms, including temporal-difference learning, policy iteration, and Q-learning. By contrast, policy gradient descent, despite its widespread use, remains less well understood from a theoretical perspective, largely because of its non-convex structure and the complexity of the underlying functional space. Existing convergence analyses typically require uniform regularity assumptions on the policy, viewed as a feedback control function, along the gradient flow; moreover, the resulting convergence rates depend on these regularity bounds. In this work, we significantly strengthen the existing convergence theory. Our key insight is to relate policy gradient descent to a mirror flow on the space of probability measures over controlled trajectories, where the Bregman divergence is induced by the convex control cost. We rigorously prove a JKO-type convergence result showing that the discrete-time mirror flow converges to its continuous-time counterpart, and we identify the limiting dynamics as a preconditioned policy gradient descent flow on the space of control processes. Leveraging this mirror-flow perspective, we establish an exponential convergence rate that is independent of policy regularity. The convergence holds both in Bregman divergence and in the value of the associated control problem.

Bio

Zhenjie Ren is a Professor at LaMME, Université Évry Paris-Saclay since 2024. Prior to this, he was an Associate Professor at CEREMADE, Université Paris Dauphine-PSL, where he served from 2016. He earned his PhD in applied mathematics and subsequently held a postdoctoral fellowship at CMAP, École Polytechnique. His research interests lie in applied mathematics with a focus on quantitative finance and machine learning. His work primarily explores areas at the intersection of stochastic processes and optimal control theory, including McKean-Vlasov diffusions, particle systems, mean-field games, optimal contract theory, backward stochastic differential equations, and viscosity solutions.