Artificial Music Intelligence: Expressive and Interactive Music Performance via Machine Learning

Lecture / Panel
For NYU Community

NYU Shanghai presents NYU Data Science Professor Faculty Candidate Gus Xia, PhD Candidate, Machine Learning,  Machine Learning Department, Carnegie Mellon University

As both a computer scientist and a musician, I design intelligent systems that understand and extend human musical expression. To "understand" means to model the musical expression conveyed through acoustic, gestural, and emotional signals. To "extend" means to use this understanding to create expressive, interactive, and autonomous agents, serving both amateur and professional musicians. In particular, I create interactive performers that are able to perform expressively in concert with humans by learning musicianship from rehearsal experience. This study unifies machine learning and knowledge representation of music structure and performance skills in an HCI framework. In this talk, I will go over the learning techniques and present robot musicians capable of playing interactively with humans while generating facial and gestural expressions. At the end, I will show the potential impacts of this study on the future of music appreciation, performance, and education.


Gus Xia is completing his Ph.D. in the Machine Learning Department at Carnegie Mellon University where he studies Machine Learning and Computer Music. His interdisciplinary research on Artificial Music Intelligence connects with Machine Learning, Computer Music. HCI, and Robotics. His papers have been published in top conferences and journals in Computer Music and Intelligent Systems, including NIME, ISMIR, AAMAS, and Computer Music Journal. In 2010, he received his undergraduate degree in Information Science with a minor in Psychology at Peking University. 

He is also a professional DI and XIAO (Chinese flute and vertical flute) player and currently a soloist of the Pitt Carpathian Ensemble. Prior to that, he was the prime soloist of the Chinese Music Institute (CMI) in Peking University, where he also served as the president and assistant conductor. 

Refreshments will be provided.