Konstantin Mishchenko

Konstantin Mishchenko

Postdoctoral Researcher

Inria Sierra

Bio

I’m a postdoc at Inria Sierra working with Alexandre d’Aspremont and Francis Bach. I received my PhD in computer science from KAUST, where I worked under the supervision of Peter Richtárik on optimization theory and its applications in machine learning. In 2020, I interned at Google Brain hosted by Nicolas Le Roux and Courtney Paquette. Prior to that, I obtained my double degree MSc diploma from École Normale Supérieure Paris-Saclay and Paris-Dauphine, and a BSc from Moscow Institute of Physics and Technology.

My hobbies include squash, ultimate frisbee, and bouldering.

Interests
  • Optimization
  • Deep learning
  • Federated and distributed learning
Education

  • PhD in Computer Science, 2021

    KAUST

  • MSc in Data Science, 2017

    École normale supérieure Paris-Saclay and Paris-Dauphine

  • BSc in Computer Science and Physics, 2016

    Moscow Institute of Physics and Technology

Experience

 
 
 
 
 
Inria Sierra
Postdoc
Inria Sierra
Dec 2021 – Present Paris, France

Research directions:

  • Adaptive algorithms
  • Second-order algorithms
  • Distributed training

Recent Posts

2 new papers: Newton and adaptive gradient
We recently released two new papers that are available on arxiv as preprints. The first work considers the global convergence of regularized Newton method (short summary in this twitter thread) and the second work is about adaptive stepsizes for stochastic gradient descent.
Last updated on Aug 17, 2022 1 min read
ICML 2022: Outstanding Reviewer
I was selected among top 10% of reviewers at ICML this year.
Last updated on Aug 17, 2022 1 min read
Visiting Sebastian Stich in Germany
Between 22 June and 28 June, I visited Sebastian Stich, who works at CISPA Helmholtz Center for Information Security near Saarbrücken in Germany. We discusses new ways to solve nonconvex problems, and we already have some interesting results. Sebastian is also looking for PhD students, so if you or someone you know is looking for a good supervisor working on optimization, Sebastian should be the first person to contact!
Last updated on Jun 30, 2022 1 min read
New paper: Asynchronous SGD with arbitrary delays
My first ever optimization project was an ICML paper about an asynchronous gradient method. At the time, I was quite confused by the fact that no matter what I was doing, Asynchronous gradient descent still converged. Five years later, I can finally give an answer: Because Asynchronous SGD doesn’t care about the delays, which we proved in https://arxiv.org/abs/2206.07638 our new paper. For a short summary, you can read my twitter thread about the paper or check my slides.
Last updated on Aug 1, 2022 1 min read
I'm at attending SICO conference 12-15 June

In a few days, I am travelling to Autrans, Vercors near Grenoble for the SICO conference dedicated to the 60th birthday of Anatoli Juditsky. The conference will feature a number of speakers working on optimization and statistics. As I did my master’s thesis at the University of Grenoble, I’m really happy to go there again after having been away for almost 5 years.

On the last day of the conference, I will give a talk about a new paper on Asynchronous SGD. The work that I will present is also going to appear online quite soon.

Last updated on Jun 10, 2022 1 min read
See all posts

Recent Papers

Quickly discover relevant content by filtering publications.
Nikita Doikov, Konstantin Mishchenko, Yurii Nesterov (2022). Super-Universal Regularized Newton Method.

PDF Cite arXiv

Samuel Horváth, Konstantin Mishchenko, Peter Richtárik (2022). Adaptive Learning Rates for Faster Stochastic Gradient Methods.

PDF Cite arXiv

Konstantin Mishchenko, Francis Bach, Mathieu Even, Blake Woodworth (2022). Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays.

PDF Cite Slides arXiv

Konstantin Mishchenko, Grigory Malinovsky, Sebastian Stich, Peter Richtárik (2022). ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!. ICML.

PDF Cite Video arXiv ICML

Grigory Malinovsky, Konstantin Mishchenko, Peter Richtárik (2022). Server-Side Stepsizes and Sampling Without Replacement Provably Help in Federated Optimization.

PDF Cite arXiv

Konstantin Mishchenko (2021). Regularized Newton Method with Global $O(1/k^2)$ Convergence.

PDF Cite Code Slides Video arXiv

Konstantin Mishchenko, Bokun Wang, Dmitry Kovalev, Peter Richtárik (2021). IntSGD: Adaptive Floatless Compression of Stochastic Gradients. ICLR.

PDF Cite Code Poster Slides arXiv ICLR

Konstantin Mishchenko, Ahmed Khaled, Peter Richtárik (2021). Proximal and Federated Random Reshuffling. ICML.

PDF Cite Code Slides Video arXiv ICML

Konstantin Mishchenko, Ahmed Khaled, Peter Richtárik (2020). Random Reshuffling: Simple Analysis with Vast Improvements. NeurIPS.

PDF Cite Code Poster Slides arXiv NeurIPS

Adil Salim, Laurent Condat, Konstantin Mishchenko, Peter Richtárik (2020). Dualize, Split, Randomize: Toward Fast Nonsmooth Optimization Algorithms. JOTA.

PDF Cite Poster arXiv JOTA

Dmitry Kovalev, Konstantin Mishchenko, Peter Richtárik (2019). Stochastic Newton and Cubic Newton Methods with Simple Local Linear-Quadratic Rates.

PDF Cite Poster arXiv NeurIPS

Yura Malitsky, Konstantin Mishchenko (2019). Adaptive Gradient Descent without Descent. ICML.

PDF Cite Code Poster Slides arXiv ICML Video

Konstantin Mishchenko (2019). Sinkhorn Algorithm as a Special Case of Stochastic Mirror Descent.

PDF Cite Poster Slides arXiv NeurIPS

Ahmed Khaled, Konstantin Mishchenko, Peter Richtárik (2019). Tighter Theory for Local SGD on Identical and Heterogeneous Data. AISTATS.

PDF Cite Slides arXiv AISTATS

Ahmed Khaled, Konstantin Mishchenko, Peter Richtárik (2019). First Analysis of Local GD on Heterogeneous Data.

PDF Cite Slides arXiv NeurIPS

Konstantin Mishchenko, Mallory Montgomery, Federico Vaggi (2019). A Self-supervised Approach to Hierarchical Forecasting with Applications to Groupwise Synthetic Controls.

PDF Cite arXiv ICML

Xun Qian, Alibek Sailanbayev, Konstantin Mishchenko, Peter Richtárik (2019). MISO is Making a Comeback With Better Proofs and Rates.

PDF Cite arXiv

Saeed Soori, Konstantin Mishchenko, Aryan Mokhtari, Maryam Mehri Dehnavi, Mert Gürbüzbalaban (2019). DAve-QN: A Distributed Averaged Quasi-Newton Method with Local Superlinear Convergence Rate. AISTATS.

PDF Cite arXiv AISTATS

Konstantin Mishchenko, Peter Richtárik (2019). A Stochastic Decoupling Method for Minimizing the Sum of Smooth and Non-Smooth Functions.

PDF Cite Slides arXiv

Konstantin Mishchenko, Dmitry Kovalev, Egor Shulgin, Peter Richtárik, Yura Malitsky (2019). Revisiting Stochastic Extragradient. AISTATS.

PDF Cite Slides arXiv AISTATS

Samuel Horváth, Dmitry Kovalev, Konstantin Mishchenko, Sebastian Stich, Peter Richtárik (2019). Stochastic Distributed Learning with Gradient Quantization and Double Variance Reduction. Optimization Methods and Software.

PDF Cite arXiv

Konstantin Mishchenko, Filip Hanzely, Peter Richtárik (2019). 99% of Worker-Master Communication in Distributed Optimization Is Not Needed. UAI.

PDF Cite arXiv UAI

Konstantin Mishchenko, Eduard Gorbunov, Martin Takáč, Peter Richtárik (2019). Distributed Learning with Compressed Gradient Differences.

PDF Cite arXiv

Konstantin Mishchenko, Peter Richtárik (2018). A Stochastic Penalty Model for Convex and Nonconvex Optimization with Big Constraints.

PDF Cite Poster arXiv

Filip Hanzely, Konstantin Mishchenko, Peter Richtárik (2018). SEGA: Variance Reduction via Gradient Sketching. NeurIPS.

PDF Cite arXiv NIPS

Konstantin Mishchenko, Franck Iutzeler, Jérôme Malick, Massih-Reza Amini (2018). A Delay-tolerant Proximal-Gradient Algorithm for Distributed Learning. ICML.

PDF Cite ICML

Konstantin Mishchenko, Franck Iutzeler, Jérôme Malick (2018). A Distributed Flexible Delay-tolerant Proximal Gradient Algorithm. SIOPT.

PDF Cite arXiv SIAM

Contact