Yell/IO: Play video games with the sound of your voice
The ecosystem for video games has perhaps never been more diverse. Mobile phones, personal computers and home consoles present a tremendous variety of form factors and control schemes, with games on each platform that might attract “serious” and “casual” players. However, even as game hardware has incorporated gyroscopes (Nintendo Wii and PlayStation 3 controllers), microphones and cameras (Nintendo DSi, mobile phones), these hardware expansions have done little to add user-generated audio to gameplay.
With Yell/IO, Sander de Jong and I sought to understand how gamers might respond to using audio as the sole form of input for gameplay. We set out to test the limits the player’s voice for providing directional control for on-screen action, as opposed to games for which the ultimate goal is singing or providing certain voice commands.
Our prototype
We created a mock-up controller and interface called Yell/IO. The hardware consisted of a handheld controller built around four electret microphones. This device interfaced with two inexpensive Behringer USB sound cards. The audio from these sound cards was used to control the action of a pair of simple games programmed using the Processing IDE.
The hardware system consists of a controller that can be handheld or placed on a tabletop (depending on the game), and a small box to contain external hardware (sound cards, power source). Currently, the system is designed to plug into a desktop or laptop PC, but a final system could easily incorporate computing hardware and interface directly with a user’s television. The controller’s design incorporates four microphones pointing in the cardinal directions, relative to how the user might hold it.
To build the controller, I soldered four small amplifier circuits with electret microphones, sending the output of each to a USB sound card. From the four channels, a fast Fourier transform is used to calculate the dominant frequencies in each microphone and their respective amplitudes.
We included visual cues on the initial hardware to indicate where the microphones are, in order to help orient the user and provide a clue about where the most sensitive input points are. However, our final design obscures the microphones from view in order to encourage the user to access the whole range of the controller and provide a physical barrier for “cheating” the system by tapping on microphones or blowing directly into them—both of which might damage the hardware over time.
The controller was designed with directional control using the four microphones in mind. Feedback during evaluations made us change the focus to a multiplayer prototype where every player uses one microphone as input. Using amplitude, frequency and direction, several game mechanics could be implemented, allowing for more interesting game concepts. An important question would be whether players are able to adjust to this combination without it being overwhelming.