Digital Wand Gives Texture to Sound

July 10, 2007 — A new, digital wand-like instrument gives new meaning to "textured" sound by letting people record a noise and then brush, scrape, or tap it against objects to create effects.

Its makers hope the Sound of Touch device will eventually be used by everyone from musicians and kids, to sound-designers and scientists — anyone interested in "painting" with sound.

"It's about sound and the kind of intuition people have about sound," said David Merrill, a Ph.D. candidate at the Massachusetts Institute of Technology, who along with fellow Ph.D. candidate Hayes Raffle, invented the device.

For decades, people have been using and manipulating digital audio, said Merrill. But they haven't had a way to combine their intuition about how physical materials sound with the experience of manipulating sounds with those materials.

Take sandpaper. You can imagine what it might sound like to rub your finger across it. But what would it sound like to rub your voice against it?

"The voice will sound gritty. Tap it against glass and it sounds glassy. Rub it against felt and it sounds fuzzy," said Raffle.

To record a sound sample, the user presses a button on the stem of the wand. A microphone at the tip records the sound — for example the hum of a car engine — as long as the button is pressed. Once recorded, the sample is ready for manipulation.

To add effects, the user runs the end of the wand across a texture — for example, a dish of glass marbles. A sensor at the tip feels and measures the stimulation and sends the signal to a computer.

Specialized software that builds on original work by Roberto Aimi, an MIT alum who invented hybrid digital instruments, merges the recorded sound of the car engine with the marbles. The result sounds like a car engine tinkling through glass marbles.

"It reflects a natural and intuitive way of thinking about sound," said Michael Gurevich, a post-doctoral research scholar at Stanford University specializing in physical interaction design for music. "You are taking a thing and going out in the world and probing it and tapping it the way you would a drum stick," he said.

The challenge, he said, is making the device — meant for rubbing and scraping — robust enough.

"I like to talk about the kindergarten test. If you give the wand to a six-year old, how long is it going to last?" he asked.

Merrill and Raffle will be testing the device's robustness at this years SIGGRAPH conference in August, where they expect thousands of attendants to scrape, rub, brush and tap the Sound of Touch to its limit.