Estimating the Jacobian matrix of an unknown multivariate function from sample values by means of a neural network
Frédéric Latrémolière, Sadananda Narayanappa, Petr Vojtěchovský
Abstract
We describe, implement and test a novel method for training neural networks to estimate the Jacobian matrix J of an unknown multivariate function F. The training set is constructed from finitely many pairs (x,F(x)) and it contains no explicit information about J. The loss function for backpropagation is based on linear approximations and on a nearest neighbor search in the sample data. We formally establish an upper bound on the uniform norm of the error, in operator norm, between the estimated Jacobian matrix provided by the algorithm and the actual Jacobian matrix, under natural assumptions on the function, on the training set and on the loss of the neural network during training. The Jacobian matrix of a multivariate function contains a wealth of information about the function and it has numerous applications in science and engineering. The method given here represents a step in moving from black-box approximations of functions by neural networks to approximations that provide some structural information about the function in question.
Create a lesson
Related papers
Evaluating Compositional Generalization in Transformers: The Role of Composition Equivalence and Module Coverage
Purva Pruthi, Andrew Yuan, Alexander D'Amour et al.
Sharp Sample Complexity of KL-Regularized Reinforcement Learning
Andrew Liu
Leveraging Psychophysical Attentional Distribution for Gaze-Augmented Reward Modeling
Chaeho Lee, Junyup Kim, Wansoo Kim et al.
Deep Personalization: Continuous Context Observation for Local Proactive Agents
Yijia Shao, Song Bian, Junxian Guo et al.
SLIC: Reinforcement Fine-Tuning Small LMs for Multi-Turn Analog Circuit Optimization
Yan Xu, Zhengqi Gao, Ching-Yun Ko et al.
Fermat ProbCover: Coverage-Based Active Learning on Fermat Distance Graphs
Gokul Bhusal, Delanna Do, Kevin Miller et al.