NYU team releases open-source database from Woven Planet to help visually impaired pedestrians navigate cities

BROOKLYN, New York, Feb 15, 2022 – A new dataset released by a New York University Tandon School of Engineering research team and Woven Planet Holdings, Inc., a Toyota subsidiary dedicated to building the safest mobility in the world, promises to help visually impaired pedestrians and autonomous vehicles (AVs) alike better navigate complex urban settings. 

Woven Planet partnered with NYU Tandon’s Visualization, Imaging and Data Analytics Research Center (VIDA) to compile a dataset of more than 200,000 outdoor images over the course of a year. The dataset is being used to test a range of visual place recognition (VPR) technologies that can improve the accuracy of personal and automotive navigation applications and promote independence for a variety of users. 

Developed by a team from the Automation and Intelligence for Civil Engineering (AI4CE) lab led by Chen Feng, assistant professor of civil and urban engineering, mechanical and aerospace engineering, and computer science and engineering, this dataset uses side-view images of sidewalks and storefronts in addition to forward-facing imagery, allowing researchers to test more applications than traditional mono-perspective sources. For example, side views support navigation for people with impaired vision who navigate in 360 degrees across busy city sidewalks. The data could also help improve delivery robotics, which must move forward and back as well as side to side to reach homes and businesses.

“This is the first work to systematically analyze some of the biggest challenges of visual place recognition,” said Dr. Feng. “We believe we are the first to make such data available free for education and research purposes, which is critical to diagnose and solve pressing problems with visual place recognition. Vast datasets like this one from Woven Planet can provide critical variety and diversity to inform data-driven systems and speed machine learning at scale.” 

Researchers at NYU led by John-Ross Rizzo, professor of biomedical engineering, and mechanical and aerospace engineering at NYU Tandon, and Vice Chair of Innovation for Rehabilitation Medicine at the NYU Grossman School of Medicine, are already using this dataset to help develop technologies that will help visually impaired individuals better navigate complex urban environments. 

“As a visually impaired person myself, I’ve long been frustrated that our population hasn’t seen more innovation in the navigation space; sure, solutions exist, but apply them in our urban canyons and accuracy, precision and reliability are all compromised,” said Dr. Rizzo. “Image-based wearable navigation assistance is set to make significant breakthroughs for everyone from the blind to the cognitively impaired to the elderly, helping with safe navigation in congested, complicated and often dangerous outdoor environments and also in unfamiliar indoor environments. Ultimately, this project has the potential to redefine accessibility, helping millions of people expand their horizons and better interact with the world.”

This project, which is being sponsored by C2SMART Center (the Connected Cities for Smart Mobility Toward Accessible and Resilient Transportation) — a USDOT Tier 1 University Transportation Center led by NYU Tandon — uses images originally provided by CARMERA Inc., an automotive mapping company and former participant in NYU Tandon Future Labs that was acquired by Woven Planet in 2021.

“NYU has long been one of our core academic partners, in no small part because of our shared commitment to delivering social impact through mobility,” said Ro Gupta, senior director at Woven Planet and head of the company’s Automated Mapping Platform (AMP) North America team. “It’s gratifying to see our data, which is core to our commercial mapping products, being used to help researchers around the world develop tools that will ultimately make mobility more accessible and equitable for all.” 

In addition to providing data from multiple viewpoints, this dataset offers several more unique features:

• Captures long-term changes of the same urban area for a year, so researchers can improve VPR under varied conditions like snow and heavy foliage.
• Anonymizes images to protect the privacy of pedestrians and cars. The anonymized images also provide VPR algorithms static and environment-only information.

Besides Feng and Rizzo, the NYU-VPR team includes Claudio Silva, Institute Professor in the Department of Computer Science and Engineering, who — with VIDA, which he directs — collaborated with CARMERA on the initial image-analysis research. 

Full details of the project are outlined in the paper, “NYU-VPR: Long-Term Visual Place Recognition Benchmark with View Direction and Data Anonymization Influences,” which has been published in IEEE IROS 2021 and can be viewed at arxiv.org/pdf/2110.09004.pdf.

The NYU-VPR dataset is available for research purposes at: https://ai4ce.github.io/NYU-VPR/#download 

About the New York University Tandon School of Engineering

The NYU Tandon School of Engineering dates to 1854, the founding date for both the New York University School of Civil Engineering and Architecture and the Brooklyn Collegiate and Polytechnic Institute. A January 2014 merger created a comprehensive school of education and research in engineering and applied sciences as part of a global university, with close connections to engineering programs at NYU Abu Dhabi and NYU Shanghai. NYU Tandon is rooted in a vibrant tradition of entrepreneurship, intellectual curiosity, and innovative solutions to humanity’s most pressing global challenges. Research at Tandon focuses on vital intersections between communications/IT, cybersecurity, and data science/AI/robotics systems and tools and critical areas of society that they influence, including emerging media, health, sustainability, and urban living. We believe diversity is integral to excellence, and are creating a vibrant, inclusive, and equitable environment for all of our students, faculty and staff. For more information, visit engineering.nyu.edu.

Woven Planet

Woven Planet is building the safest mobility in the world. A subsidiary of Toyota, Woven Planet innovates and invests in new technologies, software, and business models that transform how we live, work and move. With a focus on automated driving, smart cities, robotics and more, Woven Planet builds on Toyota's legacy of trust to deliver secure, connected, reliable, and sustainable mobility solutions for all. Learn more at woven-planet.global.