NYU Tandon researchers selected for National Artificial Intelligence Research Resource Pilot to enhance AI vision models 

An NYU Tandon School of Engineering project, led by Associate Professor of Computer Science and Engineering and Electrical and Computer Engineering Chinmay Hegde, is one of the first 35 initiatives selected for the National Artificial Intelligence Research Resource (NAIRR) Pilot by the U.S. National Science Foundation (NSF) and Department of Energy, a result of President Biden's Executive Order on the Safe, Secure and Trustworthy Development and Use of AI.

The NAIRR Pilot aims to connect U.S. researchers and educators with the computational power, data resources, and training capabilities essential for advancing responsible AI research. Federal agencies are collaborating with government-supported and non-governmental partners in this preparatory step toward an eventual full NAIRR implementation.

The NYU Tandon team chosen for the NAIRR Pilot will develop techniques to ensure advanced AI models that understand both images/video and text can be safely deployed in real-world applications like self-driving vehicles. 

These "vision-language" AI models are extremely powerful but need customization to reliably follow safety rules and ethical principles. The team hopes to introduce editing approaches that will systematically adjust the AI's internal parameters to instill new capabilities that conform to specified safety constraints, societal values, and intended operating behaviors, without the need for full retraining. 

"Rather than laboriously re-training these massive models from scratch, we want to edit and specialize them to behave as intended while aligning with desired requirements," said Associate Professor Hegde. "This editing capability is crucial for responsible use of vision-language AI, especially in data-scarce domains like autonomous vehicles used in construction, mining, and agriculture."

"The invaluable resources provided through the NAIRR Pilot allow Professor Hegde's research to realize its full potential in advancing responsible AI development," said Linda Boyle, NYU Tandon’s Vice Dean for Research. "His work under the NAIRR program aligns with our vision to position NYU Tandon at the forefront of ethical and impactful AI innovation. We’re proud of Professor Hegde's contributions and the other groundbreaking NAIRR Pilot initiatives.  They are paving the way for AI to deliver transformative benefits to society.”

The announcement of the NAIRR Pilot roster comes on the heels of New York State establishing Empire AI Consortium, a public-private partnership with plans to build an AI computing hub in Buffalo, NY. There, NYU and six partner institutions will conduct responsible AI research and development and advance AI for the public good, goals that align with those of the Pilot. 

“NAIRR’s selection of Professor Hegde’s project, along with one from NYU Langone, signals the value of responsible AI research and NSF’s commitment to its development,” said Stacie Bloom, Chief Research Officer & Vice Provost for Research, New York University and NYU representative to the Empire AI consortium. “This Pilot aligns with New York State’s work in creating Empire AI and the commitments made to research that serves the public interest.” 

“NYU Tandon plays a vital role in ensuring New York is the epicenter of responsible AI development,” said Tech:NYC President & CEO Julie Samuels. “NYU’s pioneering researchers, along with their counterparts statewide who have come together to form the Empire AI consortium, are spearheading safe, equitable, and accessible AI innovations that will benefit every sector of New York's economy."

As a participant in the NAIRR pilot, Hegde will gain access to significant computing resources, including an allocation of 50,000 GPU hours on Delta, the powerful NSF-funded advanced computing system at the National Center for Supercomputing Applications (NCSA) at the University of Illinois Urbana-Champaign.

In addition to NYU Tandon and NYU Langone, Cornell University, one of NYU’s fellow founding research institutions in the Empire AI Consortium, is also part of the NAIRR Pilot.