CIS & MINDS Seminar - Pratik Chaudary

<p><strong>Recorded Senminar Link:</strong></p><p><a href=" link:</strong></p><p><a href="">... <b>“</b><b>A Picture of the Prediction Space ofDeep Networks”</b></p><p><b>Abstract:</b><span> </span><span> </span><span>Thistalk will develop information-geometric techniques to analyze the predictionspace of deep networks. By examining the underlying high-dimensionalprobabilistic models, we will reveal that the training process explores aneffectively low dimensional manifold. Networks with a wide range ofarchitectures, sizes, trained using different optimization methods,regularization techniques, data augmentation techniques, and weightinitializations lie on the same manifold in the prediction space. We will alsoshow that predictions of networks being trained on different tasks (e.g.,different subsets of ImageNet) using different representation learning methods(e.g., supervised, meta-, semi supervised and contrastive learning) also lie ona low-dimensional manifold. This indicates that seemingly different tasksexhibit a strong shared structure. Finally, we will discuss ideas that exploitthis low-dimensional structure to build effective priors that can enablelearning from very few samples.</span><br></p><p><br>References:<br>1. The Training Process of Many Deep Networks Explores the Same Low-DimensionalManifold. Jialin Mao, Itay Griniasty, Han Kheng Teoh, Rahul Ramesh, RubingYang, Mark K. Transtrum, James P. Sethna, Pratik Chaudhari. 2023. <a href=" A picture of the space of typical learnable tasks. Rahul Ramesh, Jialin Mao,Itay Griniasty, Rubing Yang, Han Kheng Teoh, Mark Transtrum, James P. Sethna,and Pratik Chaudhari [ICML ’23]. <a href=" Deep Reference Priors: What is the best way to pretrain a model? YansongGao, Rahul Ramesh, and Pratik Chaudhari. [ICML '22] <a href=" Chaudhari is an Assistant Professor in Electrical and SystemsEngineering and Computer and Information Science at the University ofPennsylvania. He is a core member of the General Robotics, Automation, Sensingand Perception (GRASP) Laboratory. From 2018-19, he was a Senior AppliedScientist at Amazon Web Services and a Postdoctoral Scholar in Computing andMathematical Sciences at Caltech. Pratik received his PhD (2018) in ComputerScience from UCLA, and his Master's (2012) and Engineer's (2014) degrees inAeronautics and Astronautics from MIT. He was a part of NuTonomy Inc. (nowHyundai-Aptiv Motional) from 2014-16. He is the recipient of the Amazon MachineLearning Research Award (2020), NSF CAREER award (2022) and the Intel RisingStar Faculty Award (2022).</span></p><p></p><p> </p>

Tuesday, December 5, 2023 - 12:00 to 13:00

Clark, 110