Accelerated Training for Drifting Models

Faster convergence and stronger performance for drifting generative models, with theoretical guarantees.

Intuition

Goal: train a generator that maps Gaussian noise directly to data samples in a single forward pass, instead of the many small denoising steps used by diffusion or flow-matching models — and do so with faster, more reliable training and provable convergence.

We learn a one-step generative model by training representations to approach a fixed point of a learned update map. Each iterate is pulled toward — or pushed away from — other representations through a similarity / dissimilarity map, so that at convergence the representation sits at a stable fixed point that encodes the data manifold.

Multi-step generators iteratively transport noise to data through many small updates. A one-step generator collapses the entire trajectory into a single learned map.

Mathematics

$$\epsilon_k \sim \mathcal{N}(0, I), \qquad x_k = f_\theta(\epsilon_k)$$
$$x_{k+1} = x_k + V(x_k, p)$$
$$\mathcal{L}(\theta) \;=\; \bigl\| f_\theta(\epsilon) \;-\; \mathrm{sg}\!\left[\, f_\theta(\epsilon) + V\bigl(f_\theta(\epsilon), p\bigr) \,\right] \bigr\|^2$$

$f_\theta$ is a parameterized one-step transport map: it pushes a Gaussian sample $\epsilon \sim \mathcal{N}(0, I)$ directly to a representation $x = f_\theta(\epsilon)$. The similarity / dissimilarity map $V(\cdot, p)$ defines the fixed-point iteration the representations should converge under. The training loss matches $f_\theta(\epsilon)$ to its image one step further along the iteration, with a stop-gradient on the target so the network learns to be the fixed point rather than chase a moving one. Exploiting additional structure in $V$ accelerates this convergence and steers the iterates toward better local optima, with guarantees under standard regularity conditions.

Results

Note: Model specifications and experimental details are excluded until after paper decisions.

Why It Matters

Faster training of drifting models
Provable convergence guarantees
Improved sample quality
Structure-aware optimization
Scalable update rules
Stronger theoretical grounding