An improved periodic activation for PINNs reconstructing convective flows

Abstract

Architectures with periodic activation functions have already been shown to be beneficial in comparison to monotonic counterparts for a wide range of applications of physics-informed neural networks. Here, we investigate a network architecture which uses the complex exponential function, generating pairs of sine and cosine outputs as activation functions. Testing it against comparable, sine-activated multi-layer perceptrons for the task of temperature reconstruction from sparse velocity data for cubic Rayleigh-Bénard convection reveals significant improvements in the reconstruction quality without a substantial increase in computational cost per training step. Vice versa, the improved architecture enables reaching similar reconstruction qualities for a fraction of the expense. Analyzing the mathematical structure of these networks points to the improvements being rooted in the property of passing both a sine and cosine function forward. This way, the subsequent layer is able to adapt the phase of the provided latent periodic functions, and doing it individually for each of its neurons.