High-Flying (Eye-Flying) Robots

If you ever scrolled Facebook or browsed any number of lighthearted online news sites, you’ve probably seen at least one video in which a drone piloted by a clueless user suffers some mishap — landing at the bottom of a lake, caught high up in the branches of a tree, or tangled in the hair of some unfortunate bystander.

Simply put, it can be hard to efficiently operate a flying robot, no matter what the instruction booklet claims. That may be amusing when the drone in question is a newly purchased birthday gift for an avid hobbyist: it becomes no laughing matter, however, in the case of a drone being used to look for survivors after a natural disaster or to check pollution levels at an environmentally sensitive site.

Giuseppe Loianno, who heads the new Agile Robotics and Perception Lab at the NYU Tandon School of Engineering, is collaborating with colleagues from the University of Pennsylvania and the U.S. Army Research Laboratory to develop an easy-to-use, self-contained system that will allow pilots with even limited experience to safely and reliably control a drone with just a pair of gaze-tracking glasses and a small computing unit. Notably, no motion-capture system or even GPS is needed. Because of that, the system as a whole is the first of its kind ever to be demonstrated.

Slip on the glasses — in this case a pair of lightweight Tobii Pro Glasses 2 with an inertial measurement unit (IMU) that serves as an orientation sensor and an HD camera — and simply look at the drone. The camera will detect its location with help from the IMU data, and when the user then fixes their gaze elsewhere, the glasses calculate a vector and send a command to the drone to fly there.

A system relying on gaze alone has numerous advantages: it could, for example, provide a boon to users with mobility issues or could allow for the use of drones during emergency situations in which an experienced pilot is not readily available. Additionally, the system could be useful in the rehabilitation field, for people with eye or motion diseases.

Soon, it seems, those drone “blooper” reels may be a thing of the past.