I’m happy to announce our new book project - Dive into Deep Learning. It’s still in beta stage, i.e. we’re still working on it, but I think that it’s good enough to share with friends and colleagues. This is joint work with Aston Zhang, Mu Li, and Zachary Lipton. There’s an obvious question - why yet another machine learning book? After all, there’s...

Dear Friends,As some of you may have already heard, I’m leaving CMU to join Amazon, effective July 1, 2016. There I will be in charge of Amazon’s Cloud Machine Learning Platform with the task to make machine learning as easy to use and widespread as it could possibly be. This is a terrific task and it was an offer that I could not turn down. Our lab...

One of the key features of a parameter server is that it, well, serves parameters. In particular, it serves more parameters than a single machine can typically hold and provides more bandwidth than what a single machine offers.  A sensible strategy to increase both aspects is to arrange data in the form of a bipartite graph with clients on one side...

We’ve been busy building the next generation of a Parameter Server and it’s finally ready. It’s quite different from our previous designs, the main improvements being fault tolerance and self repair, a much improved network protocol, flexible consistency models, and a much more general interface. In the next few posts I’ll explain the engineering decisions...

Disks are slow and RAM is fast. Everyone knows that. But many optimization algorithms don’t take advantage of this. More to the point, disks currently stream at about 100-200 MB/s, solid state drives stream at over 500 MB/s with 1000x lower latency than disks, and main memory reigns supreme at about 10-100 GB/s bandwidth (depending on how many memory...

Zico Kolter and I proudly announce the 2014 Machine Learning Summer School in Pittsburgh. It will be held at Carnegie Mellon University in July 7-18, 2014. Our focus is on scalable data analysis and its applications, largely in the internet domain. So, if this is you PhD topic or if you’re planning on a startup in this area, come along.  Registration...