The Georgia Institute of Technology in the US has done it again. A research team led by Professor Gil Weinberg have created a robotic arm that is designed to play alongside and attached to a human drummer.
First the arm is secured to the drummer’s shoulder with a harness. Then, as the human drummer begins to play, the third arm is able to determine where it needs to be. If the drummer moves to play the high hat cymbal, the robotic arm moves to the play the ride cymbal. If the drummer moves to the snare, the arm switches to the tom drum.
Once it has approached the correct drum, the arm uses its built-in accelerometers to sense the distance and proximity to the drum. On-board motors ensure the stick is parallel to the the playing surface. And since the arm was programmed using human motion capture technology, it is able to move naturally and intuitively allowing the arm to rise, lower, and twist to make solid contact with the drum.
The robotic arm is even able to determine what it is supposed to be playing to accompany the drummer. The arm listens to the music in the room and then begins to improvise based on the beat and the rhythm. If the musician plays slowly, the arm slows down. If the drummer speeds up, the arm plays a faster tempo.
This isn’t the first time Professor Gil Weinberg and his team have joined music and robotics together. In 2014, his team developed a robotic prosthesis for a drummer named Jason Barnes. Barnes was forced to have his arm amputated after being electrocuted in a work related accident. Though grateful he survived the accident, Barnes felt as if all his dreams had died when he lost his arm. When Dr. Weinberg was told of Jason’s story: “I couldn’t say no,” wrote Weinberg in an email to NBC News. “So I wrote an NSF [National Science Foundation] proposal to fund it, recruited a team…”
Barnes is now a student playing at the Atlanta Institute of Music and Media. Thanks to his prosthetic he is able to play in ways that no human drummer can. “I’ll bet a lot of metal drummers might be jealous of what I can do now,” says Jason. “Speed is good. Faster is always better.”
After creating Barnes’ prosthetic, Weinberg used what they had learned for his team’s next, even more ambitious project. Weinberg was very excited to use robotics to augment humans.
“If you have a robotic device that is part of your body, it’s a completely different feeling from working alongside a regular robot,” said Weinberg. “The machine learns how your body moves and can augment and complement your activity. It becomes a part of you.”
The next step for the team is to link the robotic arm’s movements to the musician’s brain activity. They are already experimenting with an EEG headband to detect the drummer’s brain patterns, hoping that they well be able to identify certain patterns that would allow the arm to react to the musician’s thoughts rather than improvise based on what the arm is “hearing”.
But this research isn’t being done just to augment musicians. Weinberg is hopeful that their research will lead to other applications of the technology.
“Imagine if doctors could use a third arm to bring them tools, supplies or even participate in surgeries. Technicians could use an extra hand to help with repairs and experiments,” he said. “Music is based on very timely, precise movements. It’s the perfect medium to try this concept of human augmentation and a third arm.”
If Dr. Weinberg and his team are able to continue having such success in their research, it won’t be hard to dream of a future that was once only the realm of science fiction. Someday, a factory worker arriving at the plant and putting on their third arm could be as normal as their putting on a hard hat is today.