SFU MOCAD Seminar: Marta Ghirardelli

We consider neural networks (NN) as discretizations of continuous dynamical systems. There are two relevant systems: the NN architecture on one side and the gradient flow for optimizing the parameters on the other. In both cases, stability properties of the discretization methods can be relevant e.g. for adversarial robustness. Moreover, to prevent the problem of exploding or vanishing gradients, it is common to consider NNs whose feature space and/or parameter space is a Riemannian manifold. We investigate the stability of the explicit Euler method defined on Riemannian manifolds, namely the Geodesic Explicit Euler (GEE). We provide a general sufficient condition which ensures stability in any Riemannian manifold. Whenever the manifold has constant sectional curvature, such condition can be turned into a rule for choosing the stepsize.