Disentanglement as Identifiable Pushforward Factorisation

Abstract

We characterise disentanglement in smooth generative pushforward models, such as in VAEs and GANs. For a generator/decoder $g:Z\to X$ and factorised prior $p(z)=\prod_i p_i(z_i)$, we define disentanglement as factorisation of the pushforward density $p_\mu= g_\#p$ into one-dimensional "seam" factors, where each latent dimension controls an independent generative factor of the data. We prove that $p_\mu$ factorises according to the SVD of $g$'s Jacobian; that disentanglement equates to two conditions on $g$ (C1-C2); and that under those conditions the seam factors are identifiable, up to permutation and sign. In the particular case of Gaussian ($\beta$-)VAEs, we show via an identity how diagonal posteriors promote C1-C2, in expectation, explaining why disentanglement arises modulated by $\beta$. Experiments illustrate this mechanism on Gaussian data, dSprites, and CelebA.