IRIS Università Commerciale Luigi Bocconihttps://iris.unibocconi.itIl sistema di repository digitale IRIS acquisisce, archivia, indicizza, conserva e rende accessibili prodotti digitali della ricerca.Sat, 22 Jan 2022 01:34:15 GMT2022-01-22T01:34:15Z10221Anomalous finite size corrections in random field modelshttp://hdl.handle.net/11565/4020234Titolo: Anomalous finite size corrections in random field models
Abstract: The presence of a random magnetic field in ferromagnetic systems leads, in the broken phase, to an anomalous convergence of some thermodynamic quantities to their asymptotic limits. Here we show a general method, based on the replica trick, to compute analytically the finite size correction to the average free energy. We apply this method to two mean field Ising models, fully connected and random regular graphs, and compare the results to exact numerical algorithms. We argue that this behaviour is present in finite dimensional models as well.
Wed, 01 Jan 2014 00:00:00 GMThttp://hdl.handle.net/11565/40202342014-01-01T00:00:00ZClustering of solutions in the symmetric binary perceptronhttp://hdl.handle.net/11565/4032710Titolo: Clustering of solutions in the symmetric binary perceptron
Abstract: The geometrical features of the (non-convex) loss landscape of neural network models are crucial in ensuring successful optimization and, most importantly, the capability to generalize well. While minimizers' flatness consistently correlates with good generalization, there has been little rigorous work in exploring the condition of existence of such minimizers, even in toy models. Here we consider a simple neural network model, the symmetric perceptron, with binary weights. Phrasing the learning problem as a constraint satisfaction problem, the analogous of a flat minimizer becomes a large and dense cluster of solutions, while the narrowest minimizers are isolated solutions. We perform the first steps toward the rigorous proof of the existence of a dense cluster in certain regimes of the parameters, by computing the first and second moment upper bounds for the existence of pairs of arbitrarily close solutions. Moreover, we present a non rigorous derivation of the same bounds for sets of $y$ solutions at fixed pairwise distances.
Wed, 01 Jan 2020 00:00:00 GMThttp://hdl.handle.net/11565/40327102020-01-01T00:00:00ZGeneralized approximate survey propagation for high-dimensional estimationhttp://hdl.handle.net/11565/4024163Titolo: Generalized approximate survey propagation for high-dimensional estimation
Tue, 01 Jan 2019 00:00:00 GMThttp://hdl.handle.net/11565/40241632019-01-01T00:00:00ZSubdominant dense clusters allow for simple learning and high computational performance in neural networks with discrete synapseshttp://hdl.handle.net/11565/3996613Titolo: Subdominant dense clusters allow for simple learning and high computational performance in neural networks with discrete synapses
Abstract: We show that discrete synaptic weights can be efficiently used for learning in large scale neural systems, and lead to unanticipated computational performance. We focus on the representative case of learning random patterns with binary synapses in single layer networks. The standard statistical analysis shows that this problem is exponentially dominated by isolated solutions that are extremely hard to find algorithmically. Here, we introduce a novel method that allows us to find analytical evidence for the existence of subdominant and extremely dense regions of solutions. Numerical experiments confirm these findings. We also show that the dense regions are surprisingly accessible by simple learning protocols, and that these synaptic configurations are robust to perturbations and generalize better than typical solutions. These outcomes extend to synapses with multiple states and to deeper neural architectures. The large deviation measure also suggests how to design novel algorithmic schemes for optimization based on local entropy maximization.
Thu, 01 Jan 2015 00:00:00 GMThttp://hdl.handle.net/11565/39966132015-01-01T00:00:00ZLearning may need only a few bits of synaptic precisionhttp://hdl.handle.net/11565/3996608Titolo: Learning may need only a few bits of synaptic precision
Abstract: Learning in neural networks poses peculiar challenges when using discretized rather then continuous synaptic states. The choice of discrete synapses is motivated by biological reasoning and experiments, and possibly by hardware implementation considerations as well. In this paper we extend a previous large deviations analysis which unveiled the existence of peculiar dense regions in the space of synaptic states which accounts for the possibility of learning efficiently in networks with binary synapses. We extend the analysis to synapses with multiple states and generally more plausible biological features. The results clearly indicate that the overall qualitative picture is unchanged with respect to the binary case, and very robust to variation of the details of the model. We also provide quantitative results which suggest that the advantages of increasing the synaptic precision (i.e., the number of internal synaptic states) rapidly vanish after the first few bits, and therefore that, for practical applications, only few bits may be needed for near-optimal performance, consistent with recent biological findings. Finally, we demonstrate how the theoretical analysis can be exploited to design efficient algorithmic search strategies.
Fri, 01 Jan 2016 00:00:00 GMThttp://hdl.handle.net/11565/39966082016-01-01T00:00:00ZOne-loop diagrams in the random Euclidean matching problemhttp://hdl.handle.net/11565/4025607Titolo: One-loop diagrams in the random Euclidean matching problem
Abstract: The matching problem is a notorious combinatorial optimization problem that has attracted for many years the attention of the statistical physics community. Here we analyze the Euclidean version of the problem, i.e., the optimal matching problem between points randomly distributed on a
d -dimensional Euclidean space, where the cost to minimize depends on the points' pairwise distances. Using Mayer's cluster expansion we write a formal expression for the replicated action that is suitable for a saddle point computation. We give the diagrammatic rules for each term of the expansion, and we analyze in detail the one-loop diagrams. A characteristic feature of the theory, when diagrams are perturbatively computed around the mean field part of the action, is the vanishing of the mass at zero momentum. In the non-Euclidean case of uncorrelated costs instead, we predict and numerically verify an anomalous scaling for the sub-sub-leading correction to the asymptotic average cost.
Sun, 01 Jan 2017 00:00:00 GMThttp://hdl.handle.net/11565/40256072017-01-01T00:00:00ZFinite-size corrections to disordered Ising models on random regular graphshttp://hdl.handle.net/11565/4025611Titolo: Finite-size corrections to disordered Ising models on random regular graphs
Abstract: We derive the analytical expression for the first finite-size correction to the average free energy of disordered Ising models on random regular graphs. The formula can be physically interpreted as a weighted sum over all non-self-intersecting loops in the graph, the weight being the free-energy shift due to the addition of the loop to an infinite tree.
Wed, 01 Jan 2014 00:00:00 GMThttp://hdl.handle.net/11565/40256112014-01-01T00:00:00ZScaling hypothesis for the Euclidean bipartite matching problemhttp://hdl.handle.net/11565/4024167Titolo: Scaling hypothesis for the Euclidean bipartite matching problem
Abstract: We propose a simple yet very predictive form, based on a Poisson's equation, for the functional dependence of the cost from the density of points in the Euclidean bipartite matching problem.
Wed, 01 Jan 2014 00:00:00 GMThttp://hdl.handle.net/11565/40241672014-01-01T00:00:00ZThe random fractional matching problemhttp://hdl.handle.net/11565/4020240Titolo: The random fractional matching problem
Abstract: We consider two formulations of the random-link fractional matching problem, a relaxed version of the more standard random-link (integer) matching problem. In one formulation, we allow each node to be linked to itself in the optimal matching configuration. In the other one, on the contrary, such a link is forbidden. Both problems have the same asymptotic average optimal cost of the random-link matching problem on the complete graph. Using a replica approach and previous results of Wastlund (2010 Acta Mathematica 204 91-150), we analytically derive the finite-size corrections to the asymptotic optimal cost. We compare our results with numerical simulations and we discuss the main differences between random-link fractional matching problems and the random-link matching problem.
Mon, 01 Jan 2018 00:00:00 GMThttp://hdl.handle.net/11565/40202402018-01-01T00:00:00ZCritical initialisation in continuous approximations of binary neural networkshttp://hdl.handle.net/11565/4026949Titolo: Critical initialisation in continuous approximations of binary neural networks
Wed, 01 Jan 2020 00:00:00 GMThttp://hdl.handle.net/11565/40269492020-01-01T00:00:00Z