First-Order Stationarity of Reverse Diffusions
Develop a first‑order theory for diffusion models that connects optimization guarantees to sampling dynamics.
Recent work has highlighted a deep analogy between nonconvex optimization and sampling, motivating a formal first‑order analysis of diffusion models. The goal is to understand how reverse‑time dynamics inherit convergence properties when the forward noising process satisfies strong convexity, independent of the data distribution.
The study examines SDE‑based reverse‑time flows for both overdamped and underdamped Langevin diffusions, showing that relative Fisher divergences contract at explicit exponential rates under a strongly convex stationary potential. It then incorporates discretization effects and establishes averaged first‑order stationarity bounds, mirroring gradient‑norm guarantees in nonconvex optimization, for the resulting samplers.
The analysis demonstrates that SDE reverse diffusions achieve exponential Fisher contraction, a guarantee unavailable to ODE‑based reverse processes, and that discretized samplers satisfy averaged first‑order stationarity, providing a local, convexity‑free certificate of score consistency rather than global mode weight recovery.
TakeawaySDE reverse diffusions contract Fisher divergence exponentially under strong convexity and their discretized samplers enjoy averaged first‑order stationarity guarantees.