Conditional Dynamical Systems for Image Generation

Abstract

Image generation has been dominated by deep generative models running on GPUs, a paradigm whose computational and energy costs raise growing sustainability concerns. Emerging non-von Neumann computing substrates, including quantum, compute-in-memory, photonic, and thermodynamic platforms, promise greater efficiency, yet much of the existing work ports conventional neural architectures onto them and primarily accelerates operations such as matrix multiplication. This does not fully exploit a native capability of many emerging computing substrates: relaxation toward low-energy states can itself perform computation at negligible cost. We develop a family of continuous dynamical systems for image generation, built around this primitive to better harness its computational power. The proposed generator evolves an internal state under dynamics admitting an explicit Lyapunov energy and then renders the resulting state through a compact, class-agnostic decoder. For conditional generation, we introduce energy tilting: programmed pairwise interactions remain fixed and shared across classes, while a class-dependent linear field reshapes the energy without reprogramming the interaction array. An Ising-inspired design reaches a clean-FID of 9.71 on CIFAR-10 with 4096 spin variables. These results suggest that the energy-descending dynamics can serve directly as a generative computation and offer a promising path toward efficient generative tasks beyond GPUs.