PRIN Grant Prot. 20229T9EAT (2023-2025) “Statistical Mechanics of Learning Machines: from algorithmic and information-theoretical limits to new biologically inspired paradigms"

Coordinator: Enrico Malatesta

ABSTRACT

The increasing availability of massive data sets with ever-increasing volume variety
and velocity has signed the direction of the recent ultimate technological progress
where Machine Learning (ML) has emerged as the key paradigm of modern artificial
intelligence systems. However despite the many impressive achievements of ML there
are serious gaps in our theoretical understanding of learning systems: deep learning
architectures are largely overparametrized yet they are able to easily fit the training
data with very greedy algorithms and at the same time they generalize well. This project
will be structured in three main objectives: 1) assess the information-theoretical limit of
learning: given certain models for the learning machine and the environment how much
information (in terms of amount of data) is necessary for the machine to produce a
faithful representation and correct predictions about the environment regardless of any
computational barrier 2) assess the actual limit of algorithms: the eventual information
codified in the dataset has to be efficiently retrieved. It is important to understand
whether existing algorithms are able to extract this information and how they use it to
build a representation of the environment with the machine parameters. 3) propose new
biologically and human intelligence inspired learning approaches: intended as new
learning machine new dynamics of learning and algorithms and new paradigms for
data driven artificial intelligence which are less task-specific more efficient and
parsimonious with data. Following this route the project intends to develop a
mathematically grounded understanding of ML merging ideas from statistical inference
statistical mechanics and information theory. At the same time it aims at proposing
new biologically and human intelligence inspired learning paradigms models and
algorithms.