I am a PhD student at UBC working with Mark Schmidt at the intersection of optimization and machine learning. Prior to UBC, I did my BS and MS at EPFL, with Martin Jaggi. I had the chance to intern at the MPI with Philipp Hennig and at RIKEN with Emtiyaz Khan.
Selected works
Why Adam Outperforms Gradient Descent on Language Models: A Heavy-Tailed Class Imbalance Problem
F. Kunstner, R. Yadav, A. Milligan, M. Schmidt, A. Bietti.
arXiv 2024 [arXiv, code, workshop version, .bib]
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking
F. Kunstner, V. S. Portella, M. Schmidt, N. Harvey.
NeurIPS 2023 [arXiv, code, OpenReview, proceedings, poster, .bib]
Noise is not the main factor behind the gap between SGD and Adam on transformers, but sign descent might be
F. Kunstner, J. Chen, J. W. Lavington, M. Schmidt.
ICLR 2023 [arXiv, code, OpenReview, workshop version, poster, .bib]
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent
F. Kunstner, R. Kumar, M. Schmidt.
AISTATS 2021 [arXiv, poster, .bib]
BackPACK: Packing more into backprop
F. Dangel, F. Kunstner, P. Hennig.
ICLR 2020 [arXiv, code, website, OpenReview, poster, .bib]
Software utilities
- Tex2UTF8: For places that do not support Latex but happily render UTF8
- DSDL: An automated dataset downloader for libsvm datasets