Minerva: A Scalable and Highly Efficient Training Platform for Deep Learning

Speaker: Zheng Zhang, Microsoft Research Asia

Abstract:

This talk introduces Minerva, a system we built for deep learning. Minerva addresses the widening gap between productivity-oriented tools (e.g. Matlab/Octave) that are ideal breeding ground for algorithmic innovations, and task-specific ones (e.g. Cuda-Convnet/Caffe/DistBelief) that excel in speed and scale needed to run real-world applications. The challenge is to achieve programmability and performance simultaneously in one coherent framework, a common theme among many if not all domain-specific compute engines. We approach the problem with a modular and layered design, separating the support of language flexibility and execution efficiency.The language frontend expresses deep learning algorithms with a matrix-based API, results in compact codes and preserves the Matlab-like, imperative and procedural coding style. The dynamically generated dataflow representation is then mapped to different hardware. As a result, the same user code runs on modern laptop and workstation, high-end multi-core server, or server clusters, with and without GPU acceleration, delivering better or competitive performance and scalability than existing task-specific tools. We conclude the talk with lessons we gained after building several domain-specific engines, and observations of working across different research domains  

If time allows, we will briefly describe a few on-going works, including a system called Impression Store. Impression Store raises the question that if a great number of Big Data workloads are analytics that an approximate, rather than precise result suffice, what architectural decisions in the underlying infrastructure we should revisit to lift constraints that we have
taken for granted. 

Bio:

Zheng Zhang is a Principal Researcher at Microsoft Research Asia. He was founder of the System Research Group, and later served as Research Area Manager to oversee the system, wireless and networking area of MSRA. His field of research includes computer architecture, large-scale distributed system (storage and computing) and debugging tools, and lately the intersection area of deep learning and big data computing. He is also the founding member and on the Steering Committee of ACM SIGOPS Asian Pacific System Workshop (APSys).

Before joining MSRA, he was a Member of Technical Staff in HP-Labs. He graduated from Fudan University with a BS in 1987, and PhD in ECE from UIUC in 1996.