Developing a Multimodal LLM for Home and Neighborhood Safety Assessment in Older Adult Households | NYU Tandon School of Engineering

Developing a Multimodal LLM for Home and Neighborhood Safety Assessment in Older Adult Households

Center for Urban Science + Progress
Health & Wellness,
Urban
Project Sponsor:

Hanxue Wei, Industry Assistant Professor at CUSP, NYU Tandon
Bei Wu, Dean’s Professor in Global Health; Vice Dean for Research at the NYU Rory Meyers College of Nursing; and Co-Director of the NYU Aging Incubator
 

MENTOR:

Qi Sun, Assistant Professor in the Department of Computer Science and Engineering and at CUSP, NYU Tandon
Hanxue Wei, Industry Assistant Professor at CUSP, NYU Tandon
Bei Wu, Dean’s Professor in Global Health; Vice Dean for Research at the NYU Rory Meyers College of Nursing; and Co-Director of the NYU Aging Incubator

Authors

Yuqi Wen, Tianle Cheng

Research Question

How can AI-powered multimodal systems be used to assess and improve environmental safety in older adult households—both indoors and in surrounding neighborhoods?

Background

This interdisciplinary project blends public health, AI, and urban planning to promote safe aging-in-place. It offers an opportunity to work with faculty experts from computer science, nursing, and urban health, and to develop real-world tools with policy relevance. The prototype will be piloted in older American households in New York City. Students are learning to translate urban data science into socially impactful interventions, with emphasis on usability and community engagement.

Methodology

This capstone project supports the development of a prototype system that uses multimodal large language models (LLMs) to assess environmental risks in older adults’ living environments. Students are designing an Agentic AI framework that allows users to upload photos of home or neighborhood environments and receive: (1) a safety score; (2) narrative hazard feedback; and (3) generated visuals illustrating recommended modifications, such as grab bar installation or sidewalk repair.

This project is informed by the Centers for Disease Control and Prevention (CDC) and National Institute on Aging (NIA) home safety guidelines and uses Retrieval-Augmented Generation (RAG) to improve contextual accuracy. Students are contributing to image processing, safety scoring algorithms, prompt engineering for LLMs, or user interface design.

Deliverables

  • A prototype AI tool that allows users to upload home or neighborhood photos and receive a safety score, narrative feedback, and visual recommendations based on LLM and vision-language modeling.

  • A final report and presentation detailing the technical approach, data integration process, model evaluation, and policy or real-world implications for aging-in-place interventions.

Data Sources

The project relies on publicly available datasets. Public datasets include:

  1. PHELE dataset, which contains annotated images of indoor environments relevant to aging-in-place safety

  2. American Housing Survey (AHS), especially the Functional Difficulty and Home Modification modules

  3. CDC home safety guidelines and checklists

  4. NIA home safety guidelines and checklists