Papers
Relevant links for papers and preprints.
Google Scholar might be more up-to-date.
-
Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law
-
FK,
Francis Bach
2025 NeurIPS
-
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models
-
FK,
Robin Yadav,
Alan Milligan,
Mark Schmidt,
Alberto Bietti
2024 NeurIPS
-
Normalization Matters for Optimization Performance on Graph Neural Networks
-
Alan Milligan,
FK,
Hamed Shirzad,
Mark Schmidt,
Danica J. Sutherland
2024 NeurIPS OPTML Workshop
-
Noise is not the main factor behind the gap between SGD and Adam on transformers, but sign descent might be
-
FK,
Jacques Chen,
J. Wilder Lavington,
Mark Schmidt
2023 ICLR
-
Variance Reduced Model Based Methods: New rates and adaptive step sizes
-
Robert M. Gower,
FK,
Mark Schmidt
2023 NeurIPS OPTML Workshop
-
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking
-
FK,
Victor Sanches Portella,
Mark Schmidt,
Nick Harvey
2023 NeurIPS
-
Convergence Rates for the MAP of an Exponential Family and Stochastic Mirror Descent - an Open Problem
-
Rémi Le Priol,
FK,
Damien Scieur,
Simon Lacoste-Julien
2021 arXiv
-
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent
-
FK,
Raunak Kumar,
Mark Schmidt
2021 AISTATS
-
Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search)
-
Sharan Vaswani,
Issam Laradji,
FK,
Si Yi Meng,
Mark Schmidt,
Simon Lacoste-Julien
2020 NeurIPS OPTML Workshop
-
BackPACK: Packing more into backprop
-
Felix Dangel,
FK,
Philipp Hennig
2020 ICLR
-
Limitations of the empirical Fisher approximation
-
FK,
Lukas Balles,
Philipp Hennig
2019 NeurIPS
-
SLANG: fast structured covariance approximations for Bayesian deep learning with natural gradient
-
Aaron Mishkin,
FK,
Diedrik Nielsen,
Mark Schmidt,
Emtiyaz Khan
2018 NeurIPS
-
Fully Quantized Distributed Gradient Descent
-
FK,
Sebastian Stich,
Martin Jaggi
Tech Report
Collaborators
Alberto Bietti, Alan Milligan, Aaron Mishkin, Diedrik Nielsen, Damien Scieur, Danica J. Sutherland, Francis Bach, Felix Dangel, Hamed Shirzad, Issam Laradji, Jacques Chen, Lukas Balles, Emtiyaz Khan, Martin Jaggi, Mark Schmidt, Nick Harvey, Philipp Hennig, Robert M. Gower, Raunak Kumar, Rémi Le Priol, Robin Yadav, Simon Lacoste-Julien, Si Yi Meng, Sebastian Stich, Sharan Vaswani, Victor Sanches Portella, J. Wilder Lavington.
