The Birds and the Bees
(Of Engineering)

Porfiri is the latest in a long line of scientists studying glass sponges. But he’s the first to simulate, using an Italian supercomputer, them in situ—modeling the flow of water through their bodies. His team’s models solved the mystery of sponge bouquets—they grow together because each individual sponge in a line benefits from a reduction in drag and easier feeding as some food passes through the sponge in front. The work led to a grant from the U.S. Navy to investigate whether glass sponge-inspired bulb structures attached to a ship’s hull could reduce weight and hydrodynamic drag to save fuel.

Why does that matter? Research shows that an abundant and diverse environmental microbiome is important for human health—inhabitants of environments with low microbial diversity have higher rates of autoimmune conditions. An environment with ecological niches thoroughly occupied by commensal microbes is more resilient to pathogens than a sterile one. Microbial life predates multicellular life like us by billions of years—they oxygenated the atmosphere and engineered our planet’s surface. Designers of the built environment, Hénaff says, should consider how to foster microbially diverse environments in cities. 

As artificial intelligence technologies advance, computer scientists and engineers at Tandon are increasingly teaming up with biologists to help unlock the natural world’s mysteries. To help ecologists track migratory songbirds, Professor Juan Pablo Bello (CSE, ECE, CUSP) set out to build machine learning software that could identify birds by listening to the short calls they emit in flight. He didn’t expect lessons from those birds on how to better design AI systems. But that’s exactly what happened.

A major challenge in machine listening is training a system to recognize patterns independently of context—a system designed to detect human speech needs to be able to identify a person speaking if they’re in a quiet forest or a loud construction site. Computer scientists call that intentional blindness to context “invariance.” With Bello’s bird detection bioacoustics project, called BirdVox, much of the complex signal processing under the hood went toward automatically filtering background noise. But listening to birds taught Bello that fundamental goal might be misguided. 

“With birds a very interesting phenomenon in cities is they modify their song as a function of the noise where they are,” says Bello. Some birds change their pitch enough to compete with the din of city noise that the system misidentified their calls based on frequency. The environmental noise is more than noise—it’s important context for understanding the range in variability within a species, he says. Filtering it out means losing key data.

Computer scientists “have a lot to learn about adaptability in biological systems,” says Bello. The BirdVox project highlights a need to “treat variation as a function of context,” in AI systems, which is not the norm now. “There is a relationship between context and content here, which in machine learning you try to ignore, right?” says Bello. “But in the real world, those relationships are actually important to understand what is going on.” Moving machine learning forward from classification and identification to true reasoning, the next paradigm shift, will mean rethinking assumptions about data and signals, he says.

While developing BirdVox, Bello also studied urban noise in New York City through a project called SONYC. The team’s sensors went up as pandemic lockdowns began and quickly detected a pattern many New Yorkers noticed—birds returning to spaces temporarily abandoned by cars and humans. From the birds, Bello learned another lesson for machine learning—the importance of cyclicity. 

“You naturally think of cyclical behavior as hours in the day, and for birds that’s a lot less relevant than hours as a function of sunup,” says Bello. Since birds are most active at dawn, a machine listening system has to be on its game then more than other times of day when there are fewer simultaneous calls to differentiate. In the city studying urban noise, Bello noticed similar circadian rhythms in sound that varied by day of the week, month and season. Those rhythms made self-supervised learning possible—his team could tell the SONYC system that a synagogue call happens at a specific time every day, and the system could use that intrinsic element in the data to learn to identify the sound of synagogue calls. “That was an interesting thing grounded on what we observed with birds early on,” he says.

In 2019, Brian Weeks, an evolutionary ecologist at the University of Michigan, reached out to Fouhey. He had a problem—he wanted to study changes in bird skeletons over time using museum collections. He knew museums had centuries of specimens stored in little boxes—tons of data. But going through that data proved immensely challenging.

Like Porfiri, Fouhey found himself well outside his comfort zone. “The big challenge is all of a sudden you’re doing stuff that you have no training in at all,” he says. He asked a lot of “stupid” questions, like “Why on earth are you so obsessed with the length of bird femurs?” But also, like Porfiri, he loved it. “What’s so fun is you chat with these scientists and they have all these interesting practices and they’re things you never knew about, like wait, that’s how this works?” He learned each museum keeps a drawer of beetles on hand to clean the flesh off bird bones, for example. Fouhey used to take birds for granted, but now he’s an avid birder on vacations, he says. And he’s worked on other projects for scientists, including computer vision approaches to solar physics data.