Posts

I was invited by Panos Parpas to deliver a talk at Imperial College London, which I had the pleasure to do today. I talked about my perspective on the optimization theory and how it can be used to approach deep learning. While training deep networks is defined by nonconvex non-smooth losses, which are nearly impossible to minimize in general, the practical performance of optimization methods is nowhere as pessimistic. At the same time, smooth optimization framework seems to suggest that methods like SVRG, which do not work well in deep learning, are the best we can use. The purpose of my talk was, therefore, to outline directions where I believe useful theory can be developed and identify some promising ways of bridging theory with practice.

Today was my last day working as a research scientist at Samsung AI Center in Cambridge, UK. It was a pleasure to be there, we wrote a few papers and one patent application (currently under review) and I had a lot of fun collaborating with people at the office. However, I want to explore new directions and I no longer want to stay in the UK. I will announce my next steps later and I’m taking a break for now to work a little bit on composin electronic music in Ableton Live, which is something I’ve been really enjoying in the last 6 months.

In this post, I share my thoughts on some interesting research direction. Not an exhaustive list, just something I came up with in 30 minutes when responding to an email from a student.

Today I gave an invited lecture at the University of Cambridge in the Federated Learning class taught by Nic Lane. My presentation was structured around the optimization aspects of federated learning, with the emphasis on what theoretical aspects of a method matter when we are trying to understand how it works in practice.

In this post, I would like to share some thoughts on whether it’s a good idea to choose optimization as one’s PhD topic. The text was written as a response to a MSc student, who reached out asking this question, but my response is general enough to share it here for anyone thinking about doing a PhD.

I accepted an invitation to serve in the role of Action Editor to handle a small number of submissions for Transactions on Machine Learning Research (TMLR). I received the invitation shorty after I became an Expert Reviewer for TMLR, but I will only handle submissions as an editor from now on.