Introducing the Second-Order Features Adjoint Sensitivity Analysis Methodology for Fredholm-Type Neural Integral Equations

This work presents the “First-Order Features Adjoint Sensitivity Analysis Methodology for Fredholm-Type Neural Integral Equations” (1st-FASAM-NIE-Fredholm) and the “Second-Order Features Adjoint Sensitivity Analysis Methodology for Fredholm-Type Neural Integral Equations” (2nd-FASAM-NIE-Fredholm). It is shown that the 1st-FASAM-NIE-Fredholm methodology enables the efficient computation of exactly determined first-order sensitivities of decoder response with respect to the optimized NIE-parameters, requiring a single “large-scale” computation for solving the First-Level Adjoint Sensitivity System (1st-LASS), regardless of the number of weights/parameters underlying the NIE-net. The 2nd-FASAM-NIE-Fredholm methodology enables the computation, with unparalleled efficiency, of the second-order sensitivities of decoder responses with respect to the optimized/trained weights involved in the NIE’s decoder, hidden layers, and encoder, requiring only as many “large-scale” computations as there are first-order sensitivities with respect to the feature functions. The application of both the 1st-FASAM-NIE-Fredholm and the 2nd-FASAM-NIE-Fredholm methodologies is illustrated by considering a system of nonlinear Fredholm-type NIE that admits analytical solutions, thereby facilitating the verification of the expressions obtained for the first- and second-order sensitivities of NIE-decoder responses with respect to the model parameters (weights) that characterize the respective NIE-net.