George Tzanetakis

George Tzanetakis

Rise of the robot musicians

UVic professor George Tzanetakis on track to create computer music systems that can jam with humans

The thing about robot musicians is they’ll never trash a hotel room, show up to a gig drunk or break up a band after meeting an eccentric performance artist.

At least not yet.

The music-computer lab of George Tzanetakis at the University of Victoria is ground zero for the rise of robot musicians. But don’t think of it like anthropomorphic C-3PO strumming a mandolin – it’s more like a series of mallets linked to solenoids on a drum, backed by sophisticated software. Tzanetakis effectively wants to teach music to a computer, and for the computer to pick up on musical cues while jamming with humans.

“When you play sound, a musician hears what is happening. We are trying to add the ability to understand music to an artificial agent that performs,” says Tzanetakis, an assistant professor in the department of computer science, and in the department electrical and computer engineering – and the school of music.

“The idea is to make the system musically intelligent, to have robotic musicianship.”

It remains early days for robot jazz bands. Mechanical musicians are largely limited to percussion instruments due to increasing engineering complexity as each mallet and its actuator is added to the mix, and to the software that crunches the music in real time to adjust the tempo and volume.

“We want the musicality of the system to learn how to slow down, speed up, understand if it is too loud. We are still working on making sure it doesn’t get drunk,” Tzanetakis jokes.

Of course computers, even advanced learning systems, don’t understand music or sound or much of anything else – they just crunch data in sophsticated ways. Gabrielle Odowichuk, who worked in UVic computer-music lab as a masters student, said completely computer-driven music seems unlikely. Computers still can’t connect on an emotional level.

“You need a human component or its boring. All robots need a human component,” says Odowichuk who worked on gestural control of sound. “Often you can program in more randomness to make it more human. But when it comes to conveying something and connecting with an audience, computers lack expressiveness.”

Tzanetakis sees robots and computer musicality as the natural evolution in how technology influences how music is made. To him, it’s no different than the person who first strung wires and called it a piano, or plugged in a wah-wah pedal.

“Using computers and robots is the same process,” he says. “Musicians accept synthesizers, drum machines and DJ sets. If anything a robot is more innocent than a drum machine. It’s just another technical dimension.”

How this work will eventually influence the broader music world is hard to predict. Tzanetakis, a 37-year-old native of Greece who earned his PhD in computer science from Princeton University in 2002, tends to be about five years ahead of current popular technology.

Tzanetakis’ early work on acoustic signal processing laid the groundwork for popular apps such as Shazam, which can recognize songs by holding a smartphone up to a speaker. He’s helped develop melody-matching systems that identify songs by users humming or singing. His music sensing algorithms are at the heart Smule’s AutoRap app, which creates a rap song out of any set of sounds, and the Ocarina app, which transforms an iPhone into a flute.

Of course, when he began working on what’s called audio fingerprinting – systems that analyze, dissect and find patterns in acoustic signals – smartphone technology didn’t exist.

“If I do my job right, stuff I’m doing now will be commercial in about five years,” he says. “What is commercial now is stuff myself and others were doing 10 years ago. Ten years ago there was no iPod, no iTunes. The world was a different place.”

Tzanetakis expects the next big creative leaps to involve computer systems that can analyze and extract individual instruments and voices from complex music scores. The human ear can pick our individual conversations in a noisy party – computers cannot.

“It’s really a hard problem, far from being solved, but is actively researched,” he said.

Despite being at the forefront of intertwining computers and music, Tzanetakis, a jazz musician and pianist before he became a computer scientist and engineer, admits he doesn’t always practice what he preaches.

“I’m a sax player. I have typically avoided playing with technology.”

editor@saanichnews.com

 

 

Get local stories you won't find anywhere else right to your inbox.
Sign up here

Just Posted

An aerial view over Oak Bay. (Black Press Media File Photo)
An aerial view over Oak Bay and the Marina. Oak Bay residents pay the highest taxes on Vancouver Island. Don Denton/Black Press
Oak Bay secondary suites study considers units old and new

Secondary suites draft report due in new year

Itty, a Siamese cat, has been missing since a house fire in Victoria’s Fernwood neighbourhood on Friday, Nov. 27. Her owner says she has white fur with blonde and grey markings. (Facebook/ROAM)
Cat goes missing after house fire in Fernwood neighbourhood

‘Itty’ has white fur, blonde and grey markings and blue eyes

Island Health has confirmed the first long term care facility outbreak in Greater Victoria at Veterans Memorial Lodge in Saanich. (Google Maps)
Island Health records first long-term care COVID outbreak in Greater Victoria

Veterans Memorial Lodge in Saanich confirms one positive staff member

Westcoast Impressions plans to organize a COVID-19 friendly version of the event in 2021 at the Mary Winspear Centre after having cancelled the 2020 version against the backdrop of pandemic. The opening night of the 2019 Sidney Street Market featured more than 150 vendors lined along Beacon Avenue. (Black Press Media file photo)
Sidney Street Market plans for 2021 return at Mary Winspear Centre

Tentative plan calls for the event’s return to Beacon Avenue after COVID-19 pandemic

(Dave Landine/Facebook)
VIDEO: Dashcam captures head-on crash between snowplow and truck on northern B.C. highway

Driver posted to social media that he walked away largely unscathed

Black Press Media and BraveFace have come together to support children facing life-threatening conditions. Net proceeds from these washable, reusable, three-layer masks go to Make-A-Wish Foundation BC & Yukon.
Put on a BraveFace: Help make children’s wishes come true

Black Press Media, BraveFace host mask fundraiser for Make-A-Wish Foundation

A B.C. Ambulance Service paramedic wearing a face mask to curb the spread of COVID-19 moves a stretcher outside an ambulance at Royal Columbia Hospital, in New Westminster, B.C., on Sunday, November 29, 2020. THE CANADIAN PRESS/Darryl Dyck
Top doctor urges Canadians to limit gatherings as ‘deeply concerning’ outbreaks continue

Canada’s active cases currently stand at 63,835, compared to 53,907 a week prior

A Canadian Pacific freight train travels around Morant’s Curve near Lake Louise, Alta., on Monday, Dec. 1, 2014. A study looking at 646 wildlife deaths along the railway tracks in Banff and Yoho national parks in Alberta and British Columbia has found that train speed is one of the biggest factors. THE CANADIAN PRESS/Frank Gunn
Study finds train speed a top factor in wildlife deaths in Banff, Yoho national parks

Research concludes effective mitigation could address train speed and ability of wildlife to see trains

A airport worker is pictured at Vancouver International Airport in Richmond, B.C. Wednesday, March 18, 2020. THE CANADIAN PRESS/Jonathan Hayward
Canada extends COVID restrictions for non-U.S. travellers until Jan. 21 amid second wave

This ban is separate from the one restricting non-essential U.S. travel

Menno Place. (Google Street View image.)
B.C. care home looks to hire residents’ family members amid COVID-19-related staff shortage

Family would get paid as temporary workers, while having chance to see loved ones while wearing PPE

A man walks by a COVID-19 test pod at the Vancouver airport in this undated handout photo. A study has launched to investigate the safest and most efficient way to rapidly test for COVID-19 in people taking off from the Vancouver airport. The airport authority says the study that got underway Friday at WestJet’s domestic check-in area is the first of its kind in Canada. THE CANADIAN PRESS/HO, Vancouver Airport Authority *MANDATORY CREDIT*
COVID-19 rapid test study launches at Vancouver airport for departing passengers

Airport authority says that a positive rapid test result does not constitute a medical diagnosis for COVID-19

A small crash in the water south of Courtenay Saturday afternoon. Two men had to be rescued, but reports indicate there were no serious injuries. Photo by Mike Chouinard
Small plane crash in Comox Valley waters Saturday afternoon

Two rescued from plane that had flipped in water; no serious injuries reported

A photo from 2017, of Nuchatlaht First Nation members outside court after filing a land title case in B.C. ( Submitted photo/Nuchatlaht First Nation).
Vancouver Island First Nation calls on B.C. to honour UNDRIP in historic title case

Nuchatlaht First Nation says Crown counsel continues to stall the case using the ‘distasteful’ argument that the Nation ‘abandoned’ their land

Most Read