South Korea has used AI to bring a superstar back to life, but ethical concerns abound

For the first time in 25 years, the distinctive vocals of South Korean superstar Kim Kwang-seok will be heard on national television singing new material.

It’s a feat made all the more extraordinary because the famous folk singer is dead.

National broadcaster SBS plans to use artificial intelligence (AI) to bring Kim’s voice back to life on a new program, “Competition of the Century: AI vs Human,” to air later this week.

It’s not the first time AI has been used to revive a famous singer in South Korea.

In December, music channel Mnet aired “One More Time,” a show that used AI and holograms of late artists to pay tribute to their work.

And, on New Year’s Eve, K-Pop superstars BTS performed online with an AI version of singer Shin Hae-chul, who died after surgery in 2014.

Recent AI performances have excited fans of their music and the technology, but others have raised concerns about the ethics and legalities of resurrecting the voices of the dead. The creation of new works — or revived voices — by AI also raises copyright issues. Who is considered the owner? The creator of the AI program or the AI system itself?

Legendary Korean singer

Kim was aged just 31 when he died in 1996 at the peak of his career after a string of hits including “A Letter From a Private,” “Song of My Life” and “In the Wilderness.”

His death was considered to be so shocking that some have never accepted the official cause of suicide, instead choosing to believe conspiracy theories that he was murdered.

Decades later, fans still gather in a street named in his honor near his childhood home in the city of Daegu, though this year’s tributes were mostly held online due to the coronavirus pandemic.

Murals of the artist gaze over the street, where tourists sit on benches shaped like guitars and listen to musicians playing his greatest hits.

So, when national broadcaster SBS announced that it was using AI to recreate the Kim’s voice on a new program set to air this month, it sent fans into a frenzy.

A one-minute promotional clip of Kim singing “I Miss You,” a ballad released by Kim Bum-soo in 2002, has been viewed more than 145,000 times on YouTube since December. And a video showing the process behind the episode has been viewed more than 750,000 times since it was posted on January 6.

“The recovered voice sounds very much alike him, as if Kim recorded it alive,” said Kim Jou-yeon, a Kim fan for 30 years.

It even impressed a young fan Park Hye-hyun, who was just two years old when Kim died. “One of my wishes was to hear more songs in Kim’s voice … I had tears,” she said.

Show inspired by a Go match

SBS Producer Nam Sang-moon said the idea for an AI versus human competition came to him after he watched world champion Lee Se-dol take on South Korean AI program HanDol in the ancient strategy game of Go in 2019.

Surprisingly, Lee won one of their three matches.

Just a month earlier, Lee had announced his retirement from professional Go competition, saying AI was “an entity that cannot be defeated.”

Go requires two players to place black and white stones on a 19-by-19 grid. Each competitor tries to claim the most territory by surrounding their opponent’s pieces so they’re removed from the board.

Lee’s game against HanDol reminded Nam of Lee’s earlier match against AlphaGo, an AI program developed by Google DeepMind, in 2016. That time, AlphaGo won four out of five games, and afterward Lee said he had “misjudged” the machine’s capabilities.

“Lee-AlphaGo was a big shock followed by short but huge interest in AI, which is still being developed somewhere, but we’re unfamiliar with it yet,” SBS producer Nam said in a phone interview.

Nam sensed the Korean public was ready for an update and began pulling together the six-part AI competition series, including the AI performance from the late Kim Kwang-seok.

Recreating Kim

The audio AI company behind Kim’s reincarnated voice is Supertone, a South Korean startup founded in 2020 that provides AI audio solutions for content creators, according to co-founder and Chief Operating Officer Choi Hee-doo.

“For example, BTS is really busy these days, and it’d be unfortunate if they can’t participate in content due to lack of time. So, if BTS uses our technology when making games or audiobooks or dubbing an animation, for instance, they wouldn’t necessarily have to record in person,” Choi said.

Supertone’s Singing Voice Synthesis (SVS) technology learns voices by listening to multiple songs with corresponding notes and lyrics, Choi explained.

The system learned 100 songs by 20 singers before it was given 10 Kim Kwang-seok songs to learn. It now knows his voice well enough to mimic the singer’s unique style and pronunciation, Choi said.

In the SBS show, the AI Kim won’t be competing against a human singer — he’ll be singing a duet with one. “We revived Kim Kwang-seok to show the capability of it,” said Nam, the show’s producer.

Instead, Ock Joo-hyun, the former leader singer of girl band Fin.K.L, will be taking on the AI machine. Just as the AI system learned Kim’s voice, it will be trained to mimic her, too.

While some may consider a competition between an AI singer and a human as harmless fun, others warn the technology poses threats that need to be addressed by stricter guidelines and regulations.

Potential dangers of AI

Tech-savvy South Korea is considered to be at the forefront of AI technology, but there are calls for more regulation.

Within days of the launch of a petition last week, more than 376,000 people had put their signature to calls for tougher punishments for the creators of deepfake videos, particularly those that use the faces of female celebrities for pornography.

Amsterdam-based researchers Deeptrace counted more than 14,000 deepfake videos online in 2019, and found that almost all of them — 96% — were non-consensual deepfake pornography using the images of female celebrities, many from South Korea.

Replicating someone’s voice also poses serious risks, especially if fake audio is used in misinformation campaigns and fraud. In 2019, scammers convinced an executive to transfer hundreds of thousands of dollars in cash to an account, by faking the voice of his boss using audio software.

“Every year, we see about $470 million in fraud losses, including from wire transfer and phone scams. It’s a massive scale,” Vijay Balasubramaniyan, the CEO and co-founder of Pindrop, told CNN last year.

Worldwide, nations and organizations are calling for legal reform, ethical guidelines and social agreements to prevent the misuse of AI.

UNESCO is working with its 193 member states to lay the ethical foundations for regulating the technology. Last September, it released a draft of its “Recommendation on the Ethics of Artificial Intelligence.” The final report is expected to be presented to the UNESCO’s General Conference later this year.

In December, South Korea’s Ministry of Science and Information, Communication Technology released the “National AI Ethical Guideline,” a document that lays out basic standards for people involved in the development and application of AI. It says AI should be “developed and used in accord with its purpose and intention as a tool for human life, and that process should also be ethical.”

The same month, the ministry released what it calls a roadmap for AI legislation, which proposes to expand South Korea’s intellectual property rights beyond works created by humans to the investors and inventors behind AI creations.

Supertone’s Chief Technology Officer Heo Hoon said he supports laws to regulate the industry. He said to mitigate the current risks the company doesn’t deal directly with the public, preferring to work with businesses that share its ethics.

“We are clearly aware of the possibility of our technology being misused when in hands of the public,” Heo said. “I think reviving the voice of the deceased would bring backlash, which hopefully would begin a social debate and conclusively lead to legislation.”

The company has built a level of protection into its recordings by marking the audio as being produced by its AI, though casual listeners are unlikely to know.

“We have watermarking technology, which is an information planted in audio, that can’t be heard, but tracks where it was made and how it’s being distributed,” Heo said.

Questions of ownership

The use of AI to create works traditionally produced by human creativity raises the issue of ownership. If AI creates the works, does the program hold the copyright, or does it belong to the programmer?

Lawyer Ko Hwan-kyoung, an expert in AI and data protection, said the issue needs to be addressed as AI becomes more advanced.

“What’s interesting about AI is that it’s evolving to a level where it can write and compose by studying datas,” Ko said.

Last year, singer Hayeon released “Eyes on You,” a single composed by an AI program. Human producers later refined the song, according to her record label, Enterarts.

Elsewhere, AI has been used to create news stories, books and even art. Typically, the creator is considered to own the work, but in the case of AI it’s less clear.

“Are we going to acknowledge AI as legal entity with legal personality like humans, and grant copyright?” Ko, the lawyer, asked. “We need necessary regulation that guarantees human safety, not an excessive one that hinders development of AI technology.”

In the case of Kim Kwang-seok, producer Nam said SBS gained the consent of Kim’s family to reproduce his voice before proceeding with the show.

SBS paid a one-off fee to his family for featuring his voice in the show, as they did with other cast members, Nam said. Parts of the show will be published on YouTube after it airs, but neither SBS nor Supertone plan to release Kim’s song as a single.

“We’ve agreed not to officially release the songs introduced in the show,” Supertone’s Choi said.

So for now, Kim fans will have to be content with playing the old songs recorded by the real singer at a microphone more than 20 years ago.