Do Transformers Parse while Predicting the Masked Word?
Haoyu Zhao$\text{ }^{*}$, Abhishek Panigrahi$\text{ }^{*}$, Rong Ge, Sanjeev Arora Posted on:
[paper]
Publications
Task-Specific Skill Localization in Fine-tuned Language Models
Abhishek Panigrahi$\text{ }^{*}$, Nikunj Saunshi$\text{ }^{*}$, Haoyu Zhao, Sanjeev Arora Published at: International Conference on Machine Learning (ICML), 2023
[paper]
On the SDEs and Scaling Rules for Adaptive Gradient Algorithms
Sadhika Malladi$\text{ }^{*}$, Kaifeng Lyu$\text{ }^{*}$, Abhishek Panigrahi, Sanjeev Arora Published at: Neural Information Processing Systems (NeurIPS), 2022
[paper]
Understanding Gradient Descent on Edge of Stability in Deep Learning
Sanjeev Arora, Zhiyuan Li, Abhishek Panigrahi $\text{ }^{(\alpha-\beta)}$ Published at: International Conference on Machine Learning (ICML), 2022
[paper]
Learning and Generalization in RNNs
Abhishek Panigrahi, Navin Goyal Published at: Neural Information Processing Systems (NeurIPS), 2021
Effect of Activation Functions on the Training of Overparametrized Neural Nets
Abhishek Panigrahi, Abhishek Shetty, Navin Goyal Published at: International Conference on Learning Representations (ICLR), 2020
[paper]
Word2Sense: Sparse Interpretable Word Embeddings
Abhishek Panigrahi, Harsha Vardhan Simhadri, Chiranjib Bhattacharyya Published at: Association for Computational Linguistics (ACL), 2019
Oral presentation (270/3000 submissions ≈ 9% Acceptance Rate).