The latest three points on that graph, interestingly show reinforcement learning related projects, applied to games by Deepmind and OpenAI. Particularly AlphaGo Zero and slightly more general AlphaZero take ridiculous amount of compute, but are not applicable in the real world applications because much of that compute is needed to simulate and generate the data these data hungry models need. OK, so we can now train AlexNet in minutes rather than days, but can we train a 1000x bigger AlexNet in days and get qualitatively better results? Apparently not…
“This gradual shift from rich, big corporations to government sponsored institutes suggests to me that the interest in this kind of research within these corporations (I think of Google and Facebook) is actually slowly winding down.”
tinfoilparachute 