George Tzanetakis

George Tzanetakis

Rise of the robot musicians

UVic professor George Tzanetakis on track to create computer music systems that can jam with humans

The thing about robot musicians is they’ll never trash a hotel room, show up to a gig drunk or break up a band after meeting an eccentric performance artist.

At least not yet.

The music-computer lab of George Tzanetakis at the University of Victoria is ground zero for the rise of robot musicians. But don’t think of it like anthropomorphic C-3PO strumming a mandolin – it’s more like a series of mallets linked to solenoids on a drum, backed by sophisticated software. Tzanetakis effectively wants to teach music to a computer, and for the computer to pick up on musical cues while jamming with humans.

“When you play sound, a musician hears what is happening. We are trying to add the ability to understand music to an artificial agent that performs,” says Tzanetakis, an assistant professor in the department of computer science, and in the department electrical and computer engineering – and the school of music.

“The idea is to make the system musically intelligent, to have robotic musicianship.”

It remains early days for robot jazz bands. Mechanical musicians are largely limited to percussion instruments due to increasing engineering complexity as each mallet and its actuator is added to the mix, and to the software that crunches the music in real time to adjust the tempo and volume.

“We want the musicality of the system to learn how to slow down, speed up, understand if it is too loud. We are still working on making sure it doesn’t get drunk,” Tzanetakis jokes.

Of course computers, even advanced learning systems, don’t understand music or sound or much of anything else – they just crunch data in sophsticated ways. Gabrielle Odowichuk, who worked in UVic computer-music lab as a masters student, said completely computer-driven music seems unlikely. Computers still can’t connect on an emotional level.

“You need a human component or its boring. All robots need a human component,” says Odowichuk who worked on gestural control of sound. “Often you can program in more randomness to make it more human. But when it comes to conveying something and connecting with an audience, computers lack expressiveness.”

Tzanetakis sees robots and computer musicality as the natural evolution in how technology influences how music is made. To him, it’s no different than the person who first strung wires and called it a piano, or plugged in a wah-wah pedal.

“Using computers and robots is the same process,” he says. “Musicians accept synthesizers, drum machines and DJ sets. If anything a robot is more innocent than a drum machine. It’s just another technical dimension.”

How this work will eventually influence the broader music world is hard to predict. Tzanetakis, a 37-year-old native of Greece who earned his PhD in computer science from Princeton University in 2002, tends to be about five years ahead of current popular technology.

Tzanetakis’ early work on acoustic signal processing laid the groundwork for popular apps such as Shazam, which can recognize songs by holding a smartphone up to a speaker. He’s helped develop melody-matching systems that identify songs by users humming or singing. His music sensing algorithms are at the heart Smule’s AutoRap app, which creates a rap song out of any set of sounds, and the Ocarina app, which transforms an iPhone into a flute.

Of course, when he began working on what’s called audio fingerprinting – systems that analyze, dissect and find patterns in acoustic signals – smartphone technology didn’t exist.

“If I do my job right, stuff I’m doing now will be commercial in about five years,” he says. “What is commercial now is stuff myself and others were doing 10 years ago. Ten years ago there was no iPod, no iTunes. The world was a different place.”

Tzanetakis expects the next big creative leaps to involve computer systems that can analyze and extract individual instruments and voices from complex music scores. The human ear can pick our individual conversations in a noisy party – computers cannot.

“It’s really a hard problem, far from being solved, but is actively researched,” he said.

Despite being at the forefront of intertwining computers and music, Tzanetakis, a jazz musician and pianist before he became a computer scientist and engineer, admits he doesn’t always practice what he preaches.

“I’m a sax player. I have typically avoided playing with technology.”

editor@saanichnews.com

 

 

Just Posted

An example of the forest land in the Port Renfrew and Fairy Creek area of Vancouver Island is shown on May 11, 2021. THE CANADIAN PRESS/Jen Osborne
Old-growth logging opponents launch hunger strike as arrests continue at Fairy Creek

Zain Haq says the hunger strikers will gather today at Burrard Street in downtown Vancouver

The City of Victoria is hoping to ring in the summer by celebrating local art and offering some distanced, live music to surprise people in parks, plazas and other public spaces. (Photo courtesy of the City of Victoria)
Live, pop-up concerts and local art being showcased in Victoria this summer

People will see surprise serenades at 16 locations throughout the summer

An example of the forest land in the Port Renfrew and Fairy Creek area of Vancouver Island is shown on May 11, 2021. THE CANADIAN PRESS/Jen Osborne
Old-growth logging opponents launch hunger strike as arrests continue at Fairy Creek

Zain Haq says the hunger strikers will gather today at Burrard Street in downtown Vancouver

The 14th annual Oak Bay Young Exceptional Star (YES) awards June 3. (Christine van Reeuwyk/News Staff)
Oak Bay celebrates its Young Exceptional Stars with outdoor award ceremony

Nine young people recognized in 14th annual awards

Jada Benwell and Connor Larkey are the valedictorians of the 2021 graduating class at Parkland Secondary School. (Wolf Depner/News Staff)
Pandemic taught lessons in perseverance for North Saanich high schoolers

Parkland Secondary School to release 2021 grad ceremony video on June 25

Canadian Prime Minister Justin Trudeau participates in a plenary session at the G7 Summit in Carbis Bay, England on Friday June 11, 2021. THE CANADIAN PRESS/Adrian Wyld
Canada donating 13M surplus COVID-19 vaccine doses to poor countries

Trudeau says the government will pay for 87 million shots to be distributed to poor countries

Premier John Horgan speaks as provincial health officer Dr. Bonnie Henry, right, and health minister Adrian Dix look on during a press conference to update on the province’s fall pandemic preparedness plan during a press conference from the press theatre at Legislature in Victoria, Wednesday, Sept. 9, 2020. THE CANADIAN PRESS/Chad Hipolito
B.C. officials to provide details on Step 2 of COVID reopening plan Monday

Step 2 could allow for larger gatherings and a resumption of recreational travel

Indigenous Services Minister Marc Miller is seen during a news conference, Wednesday May 19, 2021 in Ottawa. THE CANADIAN PRESS/Adrian Wyld
Indigenous child-welfare battle heads to court despite calls for Ottawa to drop cases

Feds are poised to argue against two Canadian Human Rights Tribunal rulings

The Great Ogopogo Bathtub Race has been held in Summerland as a fundraising event. Do you know which Canadian city introduced this sport? (Black Press file photo)
QUIZ: A summer’s day at the water

How much do you know about boats, lakes and water?

Two-year-old Ivy McLeod laughs while playing with Lucky the puppy outside their Chilliwack home on Thursday, June 10, 2021. (Jenna Hauck/ Chilliwack Progress)
VIDEO: B.C. family finds ‘perfect’ puppy with limb difference for 2-year-old Ivy

Ivy has special bond with Lucky the puppy who was also born with limb difference

A million-dollar ticket was sold to an individual in Vernon from the Lotto Max draw Friday, June 11, 2021. (Photo courtesy of BCLC)
Lottery ticket worth $1 million sold in Vernon

One lucky individual holds one of 20 tickets worth $1 million from Friday’s Lotto Max draw

“65 years, I’ve carried the stories in my mind and live it every day,” says Jack Kruger. (Athena Bonneau)
‘Maybe this time they will listen’: Survivor shares stories from B.C. residential school

Jack Kruger, living in Syilx territory, wasn’t surprised by news of 215 children’s remains found on the grounds of the former Kamloops Indian Residential School

A logging truck carries its load down the Elaho Valley near in Squamish, B.C. in this file photo. THE CANADIAN PRESS/Chuck Stoody
Squamish Nation calls for old-growth logging moratorium in its territory

The nation says 44% of old-growth forests in its 6,900-square kilometre territory are protected while the rest remain at risk

Most Read