Deep neural networks are a rich class of function approximators that are now ubiquitous in a number of domains and enable new frontiers for prediction and modeling in physics and other sciences, but their function, limitations, and governing principles are not fully understood. I will review results from a research program over the past few years seeking to understand supervised deep learning by proceeding scientifically. These investigations draw some ideas and tools from theoretical physics, with close guidance from computational experiments, and integrate together perspectives from computer science and statistics. I will cover some past highlights from the study of overparameterized neural networks — such as exact connections to Gaussian processes and linear models — and then focus on emerging questions surrounding the role of scale (so-called “scaling laws”).
Yasaman Bahri is a Research Scientist at Google Research. She has broad interests within machine learning, with a current focus on the foundations of deep learning. She has contributed to the theory of overparameterized neural networks and statistical mechanics approaches to deep learning and is interested in using machine learning to advance condensed matter physics (many-body systems, materials) and in connections between the disciplines. She received her Ph.D. in physics (2017), in the area of theoretical condensed matter, from the University of California, Berkeley.
The College of Arts