A Bregman Proximal Viewpoint on Neural Operators
Abdel-Rahim Mezidi, Jordan Patracone, Saverio Salzo, Amaury Habrard, Massimiliano Pontil, Remi Emonet, Marc Sebban
Abstract
We present several advances on neural operators by viewing the action of operator layers as the minimizers of Bregman regularized optimization problems over Banach function spaces. The proposed framework allows interpreting the activation operators as Bregman proximity operators from dual to primal space. This novel viewpoint is general enough to recover classical neural operators as well as a new variant, coined Bregman neural operators, which includes the inverse activation operator and features the same expressivity of standard neural operators. Numerical experiments support the added benefits of the Bregman variant of Fourier neural operators for training deeper and more accurate models.
Create a lesson
Related papers
Your Transformer Is Secretly Elastic
Walter Simoncini, Dheeraj Varghese, Gertjan J. Burghouts et al.
Continuum Attention
Afshin Khadangi
Learnable Hodge-Laplacian Driven Mixture of Experts for Steady-PDE Surrogates on Coarse Unstructured Meshes
Stavros Nousias, Akis Nousias, Konstantinos Gkrispanis
One Task, Many Interfaces: Simple Cross-Interface Consistency Training Improves LLM Agent Robustness
Artin Tajdini, Manpreet Kaur, Peng Qi
Recovering Reliable Supervision via Critique-Guided On-Policy Distillation
Kaixiang Zhao, Tianrun Yu, Shangzhe Li et al.
Detection of Potato Plant Leaf Diseases using Convolutional Neural Networks
Haimanot Bitew Alemayehu, Simachew Melaku Ayalew, Chilotie Desalew Mengesha et al.