Statistics and Probability seminar
Speaker: Chirag Modi (New York University)
Host: Manas Rachh
Title: ATLAS: Adapting Trajectory Lengths and Step-Size for Hamiltonian Monte Carlo
Time, day and date: 3:00:00 PM - 4:00:00 PM, Tuesday, September 2
Venue: Ramanujan Hall
Abstract: Hamiltonian Monte Carlo (HMC) is the most widely used Markov chain Monte Carlo technique for Bayesian inference in high dimensions. However, the performance of HMC is sensitive to several tuning parameters like step size and trajectory lengths for leapfrog integration that can be difficult to set. As a result, it can still struggle to accurately sample distributions with complex geometries, e.g., varying curvature, due to the constant step size. In this talk, I will present a strategy to locally adapt the step size parameter of HMC at every iteration by evaluating a low-rank approximation of the local Hessian and estimating its largest eigenvalue. I will then combine it with a strategy to similarly adapt the trajectory length by monitoring the no U-turn condition, resulting in an adaptive sampler, ATLAS: adapting trajectory length and step-size. I will further use a delayed rejection framework for making multiple proposals that improve the computational efficiency of ATLAS, and develop an approach for automatically tuning its hyperparameters during warmup. Finally, I will compare ATLAS with NUTS on a suite of synthetic and real-world examples, and show that i) unlike NUTS, ATLAS is able to accurately sample difficult distributions with complex geometries, ii) it is computationally competitive to NUTS for simpler distributions, and iii) it is more robust to the tuning of hyperparameters.