Sound and Vision Meet Art and Science at the Integrated Digital Media Showcase

Giant robots clashed in a city-demolishing battle, a virtual piano teacher tutored amid falling snow, and a sage meditated amid glowing lights at the NYU Tandon student-curated Integrated Digital Media (IDM) Showcase. The May 12 event — comprising 91 projects by IDM undergraduates, recent graduates, and grad students — was a panoply of motion capture, physical computing, websites, mobile apps, games, interactive installations, video, animation, virtual and augmented reality, and even performance art. And it was a showcase for NYU Tandon itself, and how IDM is developing into a hotbed for new-media research and practice.

The "Awkward Giant Robot Battles" game was one of the first displays visitors encountered upon entering the exhibition, which took up the entire floor of NYU Media and Games Network (MAGNET). Developed by IDM seniors Olivier Garcia-Borg and Matthew Conto, who created the Tandon Vision app, the virtual reality (VR) game let players become giant robots fighting in a doomed metropolis. Partially inspired by Kaiju films (Japanese creature features starring building-sized monsters) the game started as a parody of the genre, peopled by Godzilla-like giants whose punches are seismic but sluggish. Said Gracia-Borg of the pyrrhic end-game, “You have to destroy the city to save it.”

In a different space, a darkened room, one encountered IDM senior Lajune Mcmillian, who sat utterly still, enshrouded in a white, diaphanous down, glowing with LED diodes. Her piece, “528,” was an exploration of meditation and mindfulness, set to an empyreal soundtrack.

IDM senior Lajune Mcmillian in her piece, "528"

De Angela Duff, co-director and industry associate professor at IDM, said the degree-track IDM program puts students in the nexus of the humanities, art and technology, a space that — thanks to internet-of-things, gaming, and all forms of technological interactivity — most of us inhabit today. “What's unique about IDM is that our program gives our students the freedom to choose their own paths in the intersection of design, art, and technology,” she said. “IDM prepares students to adapt to constantly changing technological paradigms by teaching them to learn how to learn.”

Mahe Dewan, an IDM sophomore and one of the students overseeing the show, said the program teaches skills at the industry level “with the added uniqueness that artists possess.” Dewan continued, “Students will leave as user-experience (UX) designers or cinematographers, to name a few. Some, like me, will leave as video game developers. But each of these students will share the knowledge of what it takes to have good UX, make amazing games, and shoot brilliant movies; it's a matter of choice for the student at the end.”

Sophomore Spencer Capiello said his installation, called "Sonancy," was inspired by sound projects like Janet Cardiff's “Sound Walks.” Capiello’s exhibit, a room-sized immersive audio experience using software developed by IDM Co-Director Luke DuBois, immersed visitors in a 360-degree soundscape. After donning a pair of glowing headsets, attendees wandered through the darkened space, past illuminated, interactive units dangling like ornaments, each one connected to a distinct instrument track or voice within the sound design. If the wearer drew close to one of the hanging activators, the sound of instrument it channeled would increase. Said Capiello, “It’s like virtual reality for audio.”

Shen-Fang Cheng exploited a VR headset and keyboard setup to create an immersive piano tutorial, which superimposed virtual hands over the participant’s actual hands. The effect was to see oneself play as virtual hands floated above, mirroring one’s hand movements. If one hit a sour note the visual palette — snowflakes and swirling images — would shift. Cheng, a senior, whose next gig is as a digital designer with a firm in Rochester, New York, said his next step in the VR tutorial will be to create experiences to teach users how to play other instruments.

Graduate Student Mattie Brice’s exhibit was an examination of gender, identity, and privacy. Brice, who had been a participant in the NYU Tisch School of the Arts Different Games Conference, and was an adjunct teacher at the NYU Tisch Game Center, made her own body a centerpiece of a performance art demonstration. Having wrapped strips of fabric around her thighs, breasts, arms and neck, Brice encouraged people to manipulate the strips of cloth, which were wired to a video-game controller. When a viewer held the controller and moved the fabric, an avatar would move about a virtual apartment depicted on a video display.  

Chun-Fang Huang, an NYU Tandon IDM student, and Lee Hendrickson and Nikolaos Georgantas, industrial design students at nearby Pratt Institute, showcased a conceptual wearable-technology product worn like a necklace, designed to connect people in real physical space and real time. Fang said that, in practice, users would be able to wear the device, sync it with whatever social network you use, and let it use your personal data to customize it to you. “Ideally when you go to a club or social event, you would be assigned a color, and when you come close to someone with similar preferences and social network, both would change from being colored to being white,” said Hendrickson.

Duff said sharing their work is important for students in order to forge the networks that they will require for advancing their career opportunities, “But it is also just fun, social, and celebratory of what they've accomplished all year.”