Statistical mechanics of learning: both shallow and deep

21 Mar
Widely Applied Mathematics Seminar
Surya Ganguli- Stanford University
Tuesday, March 21, 2017 -
3:00pm to 4:00pm
Pierce 209

A recent exciting area of intellectual activity involves a synthesis of machine learning, theoretical physics, and neuroscience.  The unification of these fields will likely enable us to exploit the power of complex systems analysis, developed in theoretical physics and applied mathematics, to elucidate the design principles governing neural systems, both biological and artificial, and to exploit these principles to develop better algorithms in machine learning.  We will give several vignettes in this latter direction, including:  (1) determining the best optimization problem to solve in order to perform regression in high dimensions, (2) learning generative, non-equilibrium probabilistic models of the world by reversing the flow of time in diffusion processes,  (3) exploiting the geometry of high dimensional error surfaces to speed up training, and (4) combining Riemannian geometry and dynamical mean field theory to understand how the expressive power of deep networks originates in the theory of chaos.