Sigmoid Tensor Decomposition for Compact Representations of Binary Data

Abstract

Nonlinear low-rank decompositions are receiving growing attention as a way to represent structured data with fewer parameters than conventional linear methods. Existing nonlinear approaches have focused mainly on matrices, which limits their ability to capture multidimensional structure. We propose a nonlinear tensor decomposition based on sigmoid nonlinearization in the Tucker and tensor-train formats. A binary tensor is represented by a low-rank real-valued score tensor, whose parameters are estimated by minimizing the binary cross-entropy loss. The resulting optimization problem is jointly nonconvex, but becomes convex with respect to one factor or score block when the remaining factors are fixed, as in the corresponding matrix case. We evaluate the method on several structured binary tensors and compare it with conventional Tucker and tensor-train decompositions across a range of common ranks. These results indicate that sigmoid nonlinearization is particularly effective in the low-rank regime, where it can provide more accurate binary reconstructions than conventional linear tensor decompositions at comparable model sizes.