Skip to yearly menu bar Skip to main content


Oral
in
Workshop: Deep Generative Model in Machine Learning: Theory, Principle and Efficacy

DDPM Score Matching Is Asymptotically Efficient

Sinho Chewi · Alkis Kalavasis · Anay Mehrotra · Omar Montasser

Keywords: [ Diffusion ] [ DDPM ] [ Asymptotic Efficiency ] [ Score Matching ]


Abstract:

The success of score-based generative models (SGMs), and particularly denoising diffusion probabilistic models (DDPMs), rests on the statistical technique of score matching, for which rigorous guarantees are nascent. In fact, recent work has shown that for estimation in parametric models, a variant of score matching known as implicit score matching is provably statistically inefficient for multimodal densities that are common in practice. In contrast, under mild conditions, we show that denoising score matching in DDPMs is asymptotically efficient, i.e., the DDPM estimator is asymptotically normal with covariance matrix given by the inverse Fisher information. Our proof is based on a pointwise relationship between the empirical risks of DDPM and maximum likelihood estimation.

Chat is not available.