Constructive Cavity Method for Ising Restricted and Deep Boltzmann Machines with Gaussian Couplings

Abstract

We apply the Constructive Cavity Method (CCM) to the dense, Ising type, restricted and deep Boltzmann machine with Gaussian couplings between adjacent layers. After identifying the core and interface contributions, CCM identifies the blanket decomposition of the cavity variables with the Aizenman--Sims--Starr random overlap structure. We apply the Random Energy Model (REM) universality ansatz to represent the cavity free energies by those of a generalized random energy model (GREM) and evaluate them recursively through Poisson point processes (PPPs). In the layer-wise gauge each layer is cut with its own cumulative fraction $|U_\ell\cap L_p|=λ_p q_\ell^{(p)}N+o(N)$, with $\mathbf 0=\mathbf q_0\le\mathbf q_1\le\cdots\le\mathbf q_r=\mathbf 1$, where $\mathbf q_\ell=(q_\ell^{(1)},\ldots,q_\ell^{(K)})$. Here $U_\ell$ is the cumulative blanket union up to blanket $\ell$. The index $p$ labels architectural layers, whereas $\ell$ labels the blankets. The dependence on the multilayer geometry is carried by the two matrices $M=2M_0Λ$ and $M_1=ΛM_0Λ$. The constructive cavity field in layer $p$ has blanket variances $Q_\ell^{(p)}-Q_{\ell-1}^{(p)}=(M(\mathbf q_\ell-\mathbf q_{\ell-1}))_p$, while the correction field has blanket variances $Q_\ell-Q_{\ell-1}=\mathbf q_\ell^\top M_1\mathbf q_\ell-\mathbf q_{\ell-1}^\top M_1\mathbf q_{\ell-1}$. The nested REM--PPP average gives the finite-level constructive cavity functional $\mathcal{P}_{\mathrm{RBM/DBM}}^{\mathrm{CCM}}(ζ,\mathbf q)=\log2+\sum_{p=1}^Kλ_pX_0^{(p)}-\frac12\sum_{\ell=0}^{r-1}ζ_\ell(\mathbf q_{\ell+1}^\top M_1\mathbf q_{\ell+1}-\mathbf q_\ell^\top M_1\mathbf q_\ell)$. The balanced scalar gauge is recovered by imposing $\mathbf q_\ell=q_\ell\mathbf 1$. Selecting the physical value in the non-convex multilayer regime is a separate problem.