PRIN Grant 202234LKBW (2023-2025) “ Land(e)scapes: Statistical Physics theory and algorithms for Inference and Learning problems"

Coordinator: Carlo Lucibello

ABSTRACT

Optimization algorithms for problems arising in inference and machine learning
navigate complex loss landscapes ridden by many local minima and saddles trying to
retrieve an unknown signal or to find a minimum with good generalization properties.
The statistical physics of disordered systems has a long history of dealing with rough
landscapes providing both. In this project we will develop novel techniques combining
results from random matrix theory dynamic mean-field theory and spin glass theory to
characterize the landscape of a few paradigmatic inference and learning problems. We
will analytically derive exact recovery thresholds and generalization errors under mild
statistical assumptions on the data distribution for commonly used algorithms such as
gradient descent and approximate message passing. Particular attention will be
devoted to characterizing how overparameterization simplifies the landscape and
facilitates the training dynamics. Moreover we will develop a new class of algorithms
that leverage the theoretical insights achieved and optimally exploit the landscape
information on a global scale.