Online Learning, Physics and Algorithms

In recent years, we have witnessed an increasing cross-fertilization between the fields of computer science, statistics, optimization and the statistical physics of learning. The area of machine learning is at the interface of these subjects. We start with an analysis in the statistical physics of learning, where we analyze some properties of the loss landscape of simple models of neural networks using the computer science formalism of Constraint Satisfaction Problems. Some of the techniques we employ are probabilistic, but others have their root in the studies of disorder systems in the statistical physics literature. After that, we focus mainly on online prediction problems, which were initially investigated in statistics but are now very active areas of research also in computer science and optimization, where they are studied in the adversarial case through the lens of (online) convex optimization. We are particularly interested in the cooperative setting, where we show that cooperation improves learning. More specifically, we give efficient algorithms and unify previous works under a simplified and more general framework.