Generative models based on diffusion have become the state of the art in the last few years, notably for image generation. Here, we analyze them in the high-dimensional limit, where data are formed by a very large number of variables. We use methods from statistical physics and focus on two well-controlled high-dimensional cases: a Gaussian model and the Curie-Weiss model of ferromagnetism. In the latter case, we highlight the mechanism of symmetry breaking in the inverse diffusion, and point out that, in order to reconstruct the relative asymmetry of the two low-temperature states, and thus to obtain the correct probability weights, one needs a database with a number of points much larger than the dimension of each data point. We characterize the scaling laws in the number of data and in the number of dimensions for an efficient generation.

Generative diffusion in very large dimensions

Mezard, Marc
2023

Abstract

Generative models based on diffusion have become the state of the art in the last few years, notably for image generation. Here, we analyze them in the high-dimensional limit, where data are formed by a very large number of variables. We use methods from statistical physics and focus on two well-controlled high-dimensional cases: a Gaussian model and the Curie-Weiss model of ferromagnetism. In the latter case, we highlight the mechanism of symmetry breaking in the inverse diffusion, and point out that, in order to reconstruct the relative asymmetry of the two low-temperature states, and thus to obtain the correct probability weights, one needs a database with a number of points much larger than the dimension of each data point. We characterize the scaling laws in the number of data and in the number of dimensions for an efficient generation.
2023
2023
Biroli, Giulio; Mezard, Marc
File in questo prodotto:
File Dimensione Formato  
arxiv-v3.pdf

accesso aperto

Descrizione: arXiv
Tipologia: Documento in Pre-print (Pre-print document)
Licenza: Creative commons
Dimensione 468.21 kB
Formato Adobe PDF
468.21 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11565/4064237
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? 1
social impact