How Games Can Lead to a Radical Redesign of Everyday Computer Use

For many people, daily computer use might lead to complaints about a sore back, carpal tunnel or lost data. For Polytechnic Institute of New York University (NYU-Poly) Associate Professor Katherine Isbister, glazed eyes and fatigue at the end of the day led her to rethink “our physical relationship to everyday computing.”

In an article in the current issue of Interactions Magazine, published by the Association for Computing Machinery, Isbister discusses a new effort: people who work in HCI (human-computer interaction) are using games to inspire fresh ways to engage with computers physically and emotionally – ways that can lead to positive practical outcomes such as better learning. Conducting research at NYU-Poly’s new Game Innovation Lab, which she directs, Isbister and her students have begun looking at whether — and how — movement can change the way people learn. They focus on the experience people have physically when they interact with computers: “how we feel as we move to interact with the technology.” 

The success of movement-based gaming such as Wii and Kinect, along with the rapid growth in multi-touch device use (smart phones, tablets) leads her to declare that “the era of movement-based interaction has arrived. People have “sophisticated sensor-based ways to interact with the console machines in their living rooms.” 

What next?

“Designers must turn their attention to the qualities of coupling with the system — of the moment-to-moment interaction—to create more supple experiences,” she writes. “I’ve come to realize that this is an excellent organizing principle for approaching movement-based interaction design. What if we place considerable design emphasis on how a movement makes a person feel in her body — as she performs it, and at the end of an extended interaction with the system?” 

This question led to her lab’s efforts to pin down the essential qualities of useful and pleasing movement-based interactions that integrate natural movement. “You might think because it’s movement, and movement is something we do every day, that this is intuitive and easy to do,” she says, “but it’s hard to design good movement and gestural-based interfaces.” 

One project was a simple game called Wriggle. The research team created two versions of the game — identical except that one was controlled with a keyboard, the other with a Wii Remote mounted in a silly hat. This allowed them to isolate the effects of movement on players, with an eye toward building devices and applications beyond gaming.  

In another example, Isbister cites recent research that striking powerful poses for just a couple of minutes enhances testosterone and risk tolerance. What if we could use this to combat math anxiety? she wondered — then designed a math-learning game called Scoop that employed high- and low-power arm sweeps to move fractions along a number line. Scoop recently finished a pilot with middle school girls. If they vary in self-reported math confidence after playing the two versions of the game, Isbister plans to further develop Scoop for an innovative math-curriculum program in New York City. 

Isbister’s research approach:

• Uses research from social science about how movement affects emotions, sense of social connection and well-being; 

• Creates games as research instruments in order to confirm these effects and construct designs that will shape the future of movement-based interactions;

• Applies these methods to design movement-based interactions to accomplish practical aims. 

The way we move contributes to how we feel, Isbister explains, raising the questions: Could email sorting feel more like gardening? Could an online meeting feel more like cooking or playing a sport than chatting through a small window at a prison’s visiting center? “I want to bring my body and its pleasures and capacities back into the mix of doing my daily work,” she says. “Maybe this could even reverse some of the alarming health trends that inactivity has wrought.”

Isbister holds joint appointments in NYU-Poly’s Department of Computer Science and Engineering, and NYU’s Tisch Game Center. She is also an investigator in the Games for Learning Institute housed at NYU and the research director of the Game Innovation Lab at NYU-Poly. Her research focuses on understanding and enhancing the social and emotional expressive range of games and other interactive systems. 

About Polytechnic Institute of New York University

Polytechnic Institute of New York University (formerly Polytechnic University), an affiliate of New York University, is a comprehensive school of engineering, applied sciences, technology and research, and is rooted in a 157-year tradition of invention, innovation and entrepreneurship: i2e. The institution, founded in 1854, is the nation’s second-oldest private engineering school. In addition to its main campus in New York City at MetroTech Center in downtown Brooklyn, it also offers programs at sites throughout the region and around the globe. Globally, NYU-Poly has programs in Israel, China and is an integral part of NYU's campus in Abu Dhabi. For more information, visit engineering.nyu.edu.