CLTs in deep neural networks: quantitative bounds through coupling, Stein’s method and entropy

 

SPEAKER: Giovanni Peccati (Luxembourg University)

ABSTRACT: 
 

I will discuss several recent results, allowing one to assess the discrepancy between a randomly initialized neural network and its Gaussian counterpart, in the infinite-width limit. In a functional framework, our techniques are based on the use of coupling techniques for Gaussian processes, revolving around some novel variations of the Powers-Stormer inequalities. In a finite-dimensional setting, our results yield optimal bounds in the 1-Wasserstein, 2-Wasserstein and total variation distances, either through Stein’s method,  or via the use of information theoretical tools. If time permits, I will also discuss some consequences of our results in a Bayesian setting. Based on two joint works: (i) with S. Favaro, B. Hanin, D. Marinucci and I. Nourdin (PTRF, 2025), and (ii) with L. Celli (ArXiv preprint, 2025).