A Musical Instrument That's Making (Brain) Waves

By Rebecca Sutton
Reclining man wearing EEG cap

Dr. Thomas Deuel performs on the encephalophone at 9e2 Seattle, a festival that celebrates art, science, and technology. Photo by Alibi Pictures

It would seem a tall order to musically impress Renée Fleming, an opera legend who has heard and sung some of the world’s most compelling performances. But on a recent visit to the University of Washington’s Center for Digital Arts and Experimental Media (DXARTS), she experienced something even she has not come across in her illustrious career: an instrument played not with the voice or the hands or the feet, but simply with the brain.

The instrument Fleming played, a Yamaha Disklavier operated by brain waves using custom hardware and software, is part of a novel research program at DXARTS’s Art + Brain Lab, where professors and musicians Juan Pampin and Richard Karpen are developing and studying instruments operated by Electroencephalography (EEG) and Electromyography (EMG). For the past several years, the two have collaborated with neuroscientist and musician Thomas Deuel of the Swedish Neuroscience Institute, also in Seattle, who is pursuing similar research with the Encephalophone, another EEG-operated instrument he helped develop. Now, with the aid of an NEA Creativity Connects grant, the three men are not only pushing the boundaries of creative expression, but are developing ways for people with motor disability to play music, and potentially rehabilitate damaged areas of the brain.

To play the Encephalophone, a person wears an EEG cap, which transmits brain waves associated with movement to the instrument. “When you go to move your arm, we see a change in signal in the motor cortex,” said Deuel, who is also a professor at DXARTS. “What makes [the encephalophone] interesting and usable, particularly for people who aren't moving, is that you can think about moving and you get the same effect. You see the same signal change.” This allows people who might have limited movement or even paralysis to produce the same musical output as an able-bodied person.

This rehabilitation would be closely linked with training the brain to play the Encephalophone, which just like any instrument, requires a degree of practice and skill. But because the Encephalophone is brand new, there are no music books to consult, or techniques to study. “In a sense it works a lot like any other instrument, except that there's not already a knowledge base of someone being able to teach you how to use it,” said Deuel. “You have to experiment and learn how these new instruments work.”

Karpen and Pampin have been critical in this regard, responding to artists like Fleming and others who have requested to visit to their lab so they can determine how the Encephalophone and other EEG-operated instruments can best be learned and played. The JACK String Quartet have been among the most actively involved in the research, visiting DXARTS on a regular basis.

“These are musicians who expect to have a high degree of expressive control when they're making music,” said Karpen. “They're not that happy when they put on an EEG cap and stuff happens and they have no control over it. So what we're trying to do is figure out how musicians can develop a practice where they can control their thoughts and some of their other muscles that they're used to using to make music, like they do when they're playing viola or cello or piano or some other instrument.”

Clinical trials began last Friday to begin exploring the effects that the Encephalophone might have on people with limited movement. All 15 patients involved in this pilot trial have a background in music, and moderate to severe motor disability. The first patient is a former composer and multi-instrumentalist with Amyotrophic Lateral Sclerosis (ALS), or Lou Gherig’s Disease. He is paralyzed except for his eyes, and communicates through blinks and eye movements.

“I think he'll be really enthralled, and really empowered by the ability to suddenly make music again in real time, which he hasn't been able to do for years,” said Deuel. “So that's one level right there, just being able to bring this to people who otherwise have lost their ability to create music.” Regardless of neurologic rehabilitation, the sheer act of playing music could have a potentially powerful impact on patients.

Eventually, the researchers hope that patients involved in the clinical trials will reach a level of proficiency on brain-operated instruments that will allow them to perform in concert with other able-bodied musicians. The goal is to have patients “be part of an ensemble, so they're not so alone playing music by themselves,” said Karpen. “They’ll be able to play as a musician with other musicians, and have that level of communication, that's non-verbal and doesn't require speech. They'll be able to speak as musicians with each other. That's something that's really beautiful and exciting to us.”

Already, Karpen and Pampin have held performances where they and other musicians have played brain-operated instruments for audiences. At one such concert last spring, “The audience couldn’t believe what they were seeing,” said Pampin, as they witnessed a piano producing sound without anyone touching it. “The experience is a lot more contemplative in a sense. There's something emphasizing that it's not just your hands or your legs or your limbs that you can use for making music. But the brain is part of your body, and can be extended into making music. People were really curious and very excited.”

As pioneering and groundbreaking as it is, Karpen noted that the integration of art and science has a long and rich tradition going back millennia, continuing to and beyond the development of instruments like the piano—at one time considered an engineering marvel—or the advent of electronic music. More recently art and science have been viewed as much less connected; their reintegration is something Karpen said he, Pampin, and Deuel have been dreaming of all their lives. “Art as a discovery of knowledge goes back thousands of years,” said Karpen. “To us, the starting point is as simple as we can do this now, so it's our job to explore it, to see what's really possible, and to push the science and the art by taking some risks.”

All three researchers noted that this re-integration of art and science has been advanced by the NEA’s Creativity Connects initiative. “It's almost ironic that the first grant to help with these clinical trials comes from an arts organization,” said Deuel, who hopes the Creativity Connects award will pave the way for other, future grants from science and medical organizations. “I applaud the NEA for taking a bit of a chance out there and supporting something that I think is really important. Hopefully we'll prime the pump. It’s a big step forward.”

Click here to watch a video of Dr. Thomas Deuel performing on the Encephalophone with The Encephalophone Ensemble at the 9e2 Seattle festival.