SYDNEY – A team of experts and students from National ICT Australia and the University of New South Wales have won first place in a major international technology competition for developing a robotically operated, computer-driven clarinet that runs on Linux.
Developed over the last eight months, the automated clarinetist beat a Dutch developed guitar playing robot to the top gong in the Artemis Orchestra competition, thanks to its playing ability and the high level of complexity in its mouthpiece design.
Head of the project, NICTA’s Dr. John Judge, described the robot as an embedded computer system connected via specially constructed electronics to actuators – brass plungers with rubber nylon feet – that control the keys and mouthpiece of the clarinet.
The robot is controlled via an off-the-shelf microcontroller board from embedded systems company, Gumstix.
“To get networking we plugged in a daughter board, and another board for the console, so we ended up with a stack of boards. The Gumstix board is an ARM processor running an Open Embedded Linux distribution,” Judge told Computerworld.
“There’s actually two CPUs controlling the robot: We’ve got Linux running on the ARM which is doing the processing of the music and setting up the series of events for the robot to react to. Those events get streamed and piped very accurately out to a microcontroller – another Gumstix board – called Robostix, which is an ATMEL, ATmega128 AVR microcontroller. The microcontroller is then told what pressure to exert on the mouthpiece and what keys to depress, and presto, the robot begins to play.
“So it’s actuating the mechanism, all the timing is being done on the Linux side. We’re actually sending a stream of midi-events to the microcontroller and it just reacts to each event as a node-on/node-off type thing. The software running on the microcontroller is our code written in C,” Judge said.
But the real beauty and complexity of the robot lies in the design of its mouthpiece. Successfully controlling the reed and air pressure flowing through the clarinet was achieved through collaboration with students and Professor Joe Wolfe from UNSW’s School of Physic’s music acoustics laboratory.
“The robot basically has two parts. There’s the parts with the actuators to control the fingering – that’s all the brass tubes in the photo (click here). And then behind that there is something that looks like it is stuffed with white padding – that is actually an air pressure chamber surrounding the entire mouthpiece of the clarinet acting as a mouth,” Judge explains.
An air pump pushes air into the chamber, which is then pressurized around the clarinet’s mouthpiece by the microcontroller in order to achieve a pleasant, correct sound.
“To get the clarinet to play a pleasant note at all you need the right air pressure and the right dampening at the same time, and there is only a small range at which the clarinet will produce a pleasant sound, otherwise it will squeak or not sound pleasant at all.”
One servo motor that pushes down on a sponge-pad inside the air chamber cylinder applies pressure to the reed at the point the lip would, imitating the embouchure and dampness of a human mouth. A second servo motor serves to mimic the musical style of tonguing in order to articulate between notes.
But can it play better than a human, or outperform the human mouth?
“No, it’s not as good as a human- there are tricky things a human can do with a clarinet that it cant do, yet. We can actually step through three octaves of range, but it’s very difficult to jump from a very low note to a high note in one jump, and there are some notes we have trouble jumping to and some that are difficult to play cold.”
In the Artemis competition, the robot played Rimsky-Korsakov’s “Flight of the Bumblebee” and Ravel’s “Bolero”, both of which avoid large octave jumps or notes that are difficult for the robot to play cold.
Considering the team had all systems working together only three weeks before the competition, the ability of the robot to overcome these weaknesses in the future is likely.
Judge admits that when you close your eyes the clarinet playing robot doesn’t yet have the warmth a human player would, but the acoustics laboratory at UNSW is working to improve its authenticity.
He heaped praise on the runners-up to the Artemis competition – a group of robotics enthusiasts from Eindhoven who developed a guitar playing robot that “when you shut your eyes sounded like it was a human playing”.
“But most of the coolness in this clarinet is in the mouthpiece, that’s why we won the prize.”
Judge said that NICTA is looking to return to the competition in subsequent years, and hopes to setup a local competition for tertiary students in their final year of high school or at university to compete for selection of their robotic project as Australia’s representative in the ARTEMIS competition.
NICTA’s chief technology officer, Dr. Chris Nicol, said that it is conceivable in the near future that we could see an entire orchestra made up of computer driven instruments like the clarinet playing robot.
A short or long video demonstration of the robot is available on the NICTA Web site
