A Hitchhiker's Guide to Momentum
Fabian Pedregosa
2023 Poster
in
Workshop: Blog Track Poster Session
in
Workshop: Blog Track Poster Session
Abstract
Polyak momentum is one of the most iconic methods in optimization. Despite it's simplicit, it features rich dynamics that depend both on the step-size and momentum parameter. In this blog post we identify the different regions of the parameter space and discuss their convergence properties using the theory of Chebyshev polynomials.
Chat is not available.
Successful Page Load