Pushing the envelope on building inspections with AI and robotics

Poorly maintained building envelopes — the outer shell of structures —  have long been silent contributors to New York City’s greenhouse gas emissions and quality of life problems, but a groundbreaking project from NYU Tandon School of Engineering is helping change that.

Chen Feng, assistant professor in both the Departments of Civil and Urban Engineering (CUE) and Mechanical and Aerospace Engineering and Semiha Ergan, associate professor in CUE,  are developing AI-driven technology that identifies building envelope leaks through robot and drone scans, offering a cost-effective and efficient alternative to traditional inspections.

The project aims to provide easier compliance with the soon-to-be-enforced NYC Local Law 97, which limits the emissions large buildings are allowed to produce. The team just received a one-year $1 million Civic Innovation Challenge (CIVIC) award from the National Science Foundation (NSF) and the Department of Homeland Security to support their work. The New York City Office of Technology and Innovation (NYC OTI) also selected the team as an inaugural project in the new NYC Smart City Testbed Program.

“Building envelopes have a significant impact on energy efficiency, occupant comfort, and overall structural integrity,” said Ergan. “Poor air and moisture sealing within these envelopes contribute to worse building emissions and can lead to adverse health outcomes for those who reside within.” 

Leveraging the robot perception expertise in the project Principal Investigator (PI) Feng’s AI4CE lab, this project’s technology offers a transformative solution that is cost-effective, labor-efficient, non-invasive, and scalable. 

“It will enable building inspection practitioners to perform more inspections than traditional methods, significantly reducing time and cost, especially for locations that are not easily accessible. Moreover, our deep learning methods will allow the novel use of unconventional sensors to pinpoint various building envelope defects,” said Feng.

Last year, the team partnered with the New York City Department of Citywide Administrative Services (NYC DCAS), the NYC OTI, and New York City District 2030 to secure a Stage 1 CIVIC award. NSF earmarks CIVIC funding for academic researchers who collaborate with civic partners – including local government agencies and nonprofits – to address community-identified priorities.

The team's Stage 1 work included a pilot assessment of buildings at the Brooklyn Army Terminal, using thermal-sensing drones and a preliminary version of a robotic data collection platform. It also included a team building activity to broaden the research impact to more low-income communities through the team’s civic partners such as the non-profit RETI Center and the Archdiocese of New York City.

The success of Stage 1 laid the foundation for the Stage 2 $1 million award announced last month, which allows the team to continue its research.

Over the next year, the team will continue to develop and deploy an autonomous robotic data collection system equipped with advanced sensors such as GPR, Light Detection and Ranging (LiDAR), GPS, optical cameras, and thermal cameras. Drones will be employed to capture thermal and color data of building exteriors. This technology enables the non-invasive detection of issues like moisture and thermal irregularities in building envelopes, reducing the need for physical visits by engineers. The collected data is sent to cloud-based servers, where AI-powered software analyzes it to generate detailed reports on building conditions.

Among its inspection testing, the team plans to scan various schools owned and operated by the Archdiocese of New York City. Moreover, the team’s social science research partner, a team led by Diana Hernandez, associate professor of sociomedical sciences at the Columbia University Mailman School of Public Health, will investigate various barriers and opportunities for this new technology to be adopted, from academic, social, and political vantage points.

The start-up company Building Diagnostic Robotics, founded by NYU Tandon alumni Bilal Sher MS CUE ’22, played a pivotal role in Stage 1 that continues into Stage 2. As a graduate student from the AI4CE lab, Sher developed EASEEbot, a precursor to his eventual company that offers commercialized AI-powered robotic building inspections. Building Diagnostic Robotics will continue the team’s work after the CIVIC funding concludes.

The research being conducted under this CIVIC award supports NYU Tandon’s commitment to finding engineering solutions to urban infrastructure problems, reflecting one of the School’s seven interdisciplinary Areas of Excellence.