A Musical Instrument That's Making (Brain) Waves
It would seem a tall order to musically impress Renée Fleming, an opera legend who has heard and sung some of the world’s most compelling performances. But on a recent visit to the University of Washington’s Center for Digital Arts and Experimental Media (DXARTS), she experienced something even she has not come across in her illustrious career: an instrument played not with the voice or the hands or the feet, but simply with the brain.
The instrument Fleming played is a Yamaha Disklavier, which is operated by brain waves using custom hardware and software. It is part of a novel research program at DXARTS’s Art + Brain Lab, where professors and musicians Juan Pampin and Richard Karpen are developing and studying instruments operated by Electroencephalography (EEG) and Electromyography (EMG). For the past several years, the two have also collaborated with neuroscientist and musician Thomas Deuel, who is pursuing similar research with the Encephalophone, another EEG-operated instrument he helped develop. Now, with the aid of an NEA Creativity Connects grant, the three men are not only pushing the boundaries of creative expression, but are developing ways for people with motor disability to play music, and potentially rehabilitate damaged areas of the brain.
To play the Encephalophone, a person wears an EEG cap, which transmits brain waves associated with movement to the instrument. “When you go to move your arm, we see a change in signal in the motor cortex,” said Deuel, who is also a professor at DXARTS. “What makes [the encephalophone] interesting and usable, particularly for people who aren't moving, is that you can think about moving and you get the same effect. You see the same signal change.” This allows people who might have limited movement or even paralysis to produce the same musical output as an able-bodied person.
Just like any instrument, the Encephalophone requires a degree of practice and skill. But because the Encephalophone is brand new, there are no music books to consult, or techniques to study. “In a sense it works a lot like any other instrument, except that there's not already a knowledge base of someone being able to teach you how to use it,” said Deuel. “You have to experiment and learn how these new instruments work.”
Karpen and Pampin, as well as visiting artists like Fleming, have been critical in this regard, and have helped determine how the Encephalophone and other EEG-operated instruments can best be learned and played. The JACK String Quartet has been among the most actively involved in the research, visiting DXARTS on a regular basis.
“These are musicians who expect to have a high degree of expressive control when they're making music,” said Karpen. “They're not that happy when they put on an EEG cap and stuff happens and they have no control over it. So what we're trying to do is figure out how musicians can develop a practice where they can control their thoughts and some of their other muscles that they're used to using to make music, like they do when they're playing viola or cello or piano or some other instrument.”
Clinical trials began last Friday to begin exploring the effects that the Encephalophone might have on people with limited movement. All 15 patients involved in this pilot trial have a background in music, and moderate to severe motor disability. The first patient is a former composer and multi-instrumentalist with Amyotrophic Lateral Sclerosis (ALS), or Lou Gherig’s Disease. He is paralyzed except for his eyes, and communicates through blinks and eye movements. Although there is hope that such instruments could help with neurologic rehabilitation, the sheer act of playing music could have a potentially powerful impact on patients.
“I think [this first patient] will be really enthralled, and really empowered by the ability to suddenly make music again in real time, which he hasn't been able to do for years,” said Deuel. “So that's one level right there, just being able to bring this to people who otherwise have lost their ability to create music.”
Eventually, the researchers hope that patients involved in the clinical trials will reach a level of proficiency on brain-operated instruments that will allow them to perform in concert with other able-bodied musicians. The goal is to have patients “be part of an ensemble, so they're not so alone playing music by themselves,” said Karpen. “They’ll be able to play as a musician with other musicians, and have that level of communication, that's non-verbal and doesn't require speech. They'll be able to speak as musicians with each other. That's something that's really beautiful and exciting to us.”
Already, Karpen and Pampin have held performances where they and other able-bodied musicians have played brain-operated instruments for audiences. At one such concert last spring, “The audience couldn’t believe what they were seeing,” said Pampin. “The experience emphasizes that it's not just your hands or your legs or your limbs that you can use for making music. But the brain is part of your body, and can be extended into making music. People were really curious and very excited.”
As pioneering and groundbreaking as it is, Karpen noted that the integration of art and science has a long and rich tradition going back millennia, from the development of instruments like the piano—at one time considered an engineering marvel—to the advent of electronic music. More recently, art and science are usually viewed as entirely separate fields; their reintegration is something Karpen said he, Pampin, and Deuel have been dreaming of all their lives.
“Art as a discovery of knowledge goes back thousands of years,” said Karpen. “To us, the starting point is as simple as we can do this now, so it's our job to explore it, to see what's really possible, and to push the science and the art by taking some risks.”
All three researchers noted that this re-integration of art and science has been advanced by the NEA’s Creativity Connects initiative. “It's almost ironic that the first grant to help with these clinical trials comes from an arts organization,” said Deuel, who hopes the Creativity Connects award will pave the way for other, future grants from science and medical organizations. “I applaud the NEA for taking a bit of a chance out there and supporting something that I think is really important. Hopefully we'll prime the pump. It’s a big step forward.”
