Scientific Computation and Applied & Industrial Mathematics: Michael Overton

In many applications one wishes to minimize an objective function that is not convex and is not differentiable at its minimizers. We discuss two algorithms for minimization of nonsmooth, nonconvex functions. Gradient Sampling is a simple method that, although computationally intensive, has a nice convergence theory. The method is robust and the convergence theory has recently been extended to constrained problems. BFGS is a well known method, developed for smooth problems, but which is remarkably effective for nonsmooth problems too. Although our theoretical results in the nonsmooth case are quite limited, we have made some remarkable empirical observations and have had broad success in applications. Limited Memory BFGS is a popular extension for large problems, and it is also applicable to the nonsmooth case, although our experience with it is more mixed. Throughout the talk we illustrate the ideas through examples, some very easy and some very challenging. Our work is with Jim Burke (U. Washington) and Adrian Lewis (Cornell).