Inducing Robust Algorithmic Reasoning in Vision Language Models
Simon Park$\text{ }^{*}$, Abhishek Panigrahi$\text{ }^{*}$, Yun CHeng$\text{ }^{*}$, Dingli Yu, Anirudh Goyal, Sanjeev Arora Posted on:
Publications
Progressive distillation induces an implicit curriculum
Abhishek Panigrahi$\text{ }^{*}$, Bingbin Liu$\text{ }^{*}$, Sadhika Malladi, Andrej Risteski, Surbhi Goel Posted on:
[paper]
Efficient Stagewise Pretraining via Progressive Subnetworks
Abhishek Panigrahi$\text{ }^{*}$, Nikunj Saunshi$\text{ }^{*}$, Kaifeng Lyu, Sobhan Miryoosefi, Sashank Reddi, Satyen Kale, Sanjiv Kumar Posted on:
[paper]
Trainable Transformer in Transformer
Abhishek Panigrahi$\text{ }^{*}$, Sadhika Malladi$\text{ }^{*}$, Mengzhou Xia, Sanjeev Arora Published at: International Conference on Machine Learning (ICML), 2024
[paper]
Do Transformers Parse while Predicting the Masked Word?
Haoyu Zhao$\text{ }^{*}$, Abhishek Panigrahi$\text{ }^{*}$, Rong Ge, Sanjeev Arora Published at: Empirical Methods in Natural Language Processing (EMNLP), 2023
[paper]
Task-Specific Skill Localization in Fine-tuned Language Models
Abhishek Panigrahi$\text{ }^{*}$, Nikunj Saunshi$\text{ }^{*}$, Haoyu Zhao, Sanjeev Arora Published at: International Conference on Machine Learning (ICML), 2023
[paper]
On the SDEs and Scaling Rules for Adaptive Gradient Algorithms
Sadhika Malladi$\text{ }^{*}$, Kaifeng Lyu$\text{ }^{*}$, Abhishek Panigrahi, Sanjeev Arora Published at: Neural Information Processing Systems (NeurIPS), 2022
[paper]
Understanding Gradient Descent on Edge of Stability in Deep Learning
Sanjeev Arora, Zhiyuan Li, Abhishek Panigrahi $\text{ }^{(\alpha-\beta)}$ Published at: International Conference on Machine Learning (ICML), 2022
[paper]
Learning and Generalization in RNNs
Abhishek Panigrahi, Navin Goyal Published at: Neural Information Processing Systems (NeurIPS), 2021
Effect of Activation Functions on the Training of Overparametrized Neural Nets
Abhishek Panigrahi, Abhishek Shetty, Navin Goyal Published at: International Conference on Learning Representations (ICLR), 2020
[paper]
Word2Sense: Sparse Interpretable Word Embeddings
Abhishek Panigrahi, Harsha Vardhan Simhadri, Chiranjib Bhattacharyya Published at: Association for Computational Linguistics (ACL), 2019
Oral presentation (270/3000 submissions ≈ 9% Acceptance Rate).