T-Pain in your living room: Are virtual concerts the future of live entertainment?

By Samantha Bresnahan, CNN

(CNN) — In a studio in Culver City, California, T-Pain is ready to put on a show. The American rapper and singer-songwriter, known for popularizing Auto-Tune with his hits such as “Buy U a Drank” and “Bartender,” is used to performing in front of thousands of screaming fans. But today, there’s no crowd; instead, it’s just T-Pain, a camera crew and a green screen.

It’s a departure from the past, but a nod to the future. T-Pain is performing songs that will become a virtual reality (VR) concert, delivered directly to fans via VR headsets.

“I’m definitely used to bigger stages,” T-Pain (whose real name is Faheem Rashad Najm) tells CNN during an interview on set, adding that he typically uses feedback from the crowd to “really gain my energy … (so) it’s just a lot more awkward.”

That doesn’t mean he isn’t excited about the opportunity. “I’m just a nerd,” he says, who dabbles in VR and 3D software as a hobby at home. When AmazeVR, the company behind what was billed as the world’s first VR concert tour with artist Megan Thee Stallion last year, approached him for a collaboration, the Florida-born artist was ready to try a new experience.

“The fans are going to be a lot closer,” T-Pain says. “You get to see different angles, you get to see a different kind of performance … AmazeVR is kind of like putting (me) in your living room.”

Capturing the “perfect” take

AmazeVR was founded in 2015, and has raised more than $50 million in the last four years, including a recent boost in December after Apple announced its Vision Pro mixed reality headset, according to the company.

“(Virtual reality) is a very big shift,” says Kyung Kuk Kim, vice president and head of production for AmazeVR. “It’s a new kind of medium. People are not used to it, but it’s very powerful. It really just brings you right (to the) middle of any scene or any situation.”

The virtual concert process begins with meetings with the artist to decide on everything from song choice to visuals, scripts, wardrobe, and choreography. The performance is recorded in one 12-hour session using an 8K stereo camera and motion-controlled crane.

In-house artificial intelligence (AI) helps enhance the 3D video in post-production, Kim says. A combination of that in-house AI and other platforms such as Unreal Engine are used to create the virtual stage, lighting, and special effects.

Post-production typically takes six to eight weeks, though release dates also coincide with what works for the individual artists. (AmazeVR says it plans to launch the T-Pain concert this Fall.)

“The biggest technical challenge is that the viewer is the camera,” says Lance Drake, creative director for the company. “Instead of cutting or hiding (parts of the) performance, this has to essentially be a truly live performance. And the viewer is given permission to look wherever they want – they can look and take in the world, they can look at the artist’s shoes, they can look at the artist’s face.”

That means it must be the “perfect” single take, Drake adds, “so when the artist is performing to camera, they’re performing to you. When the artist is looking into camera, they’re looking into the viewer’s eyes.”

Studying the virtual fan experience

Still, the big question remains: how closely can a virtual concert mimic what fans love about live music?

It’s one Dr. Mel Slater is trying to answer. A researcher who has studied the world of VR and our brains since the early 1990s, Slater and his team recently conducted a study using a 1983 Dire Straits concert Slater came across on YouTube.

“I thought, ‘I wish I could go to that concert … so that gave me the idea of reproducing a Dire Straits performance in virtual reality,’” says Slater, distinguished investigator in the Department of Clinical Psychology at the University of Barcelona and co-director of its Event Lab (Experimental Virtual Environments for Neuroscience and Technology).

He was interested in testing several elements, including under what conditions people would have the illusion they were at a concert, and how to take an old 2D video and create new 3D elements, he says.

“I thought it was a very innocent study, that you put people in a music concert and let’s see what happens … they should just enjoy it and that’s it,” Slater says. “But the results are very interesting. And it’s always good when you get results you don’t expect – because that’s when you learn something.”

The team found that many women in the virtual audience actually felt uncomfortable. “(They) would say things like, ‘there were these men around me, and I got worried that they were going to come over and start talking to me. And they kept looking at me,’” Slater explains. “This wasn’t actually true – we hadn’t programmed these characters around to be looking at them, but they had the illusion that they were.” (AmazeVR’s concert experience, by contrast, does not include any other virtual audience members at this time).

In a way, that was a good sign for the concept of virtual concerts in general, he says, because that meant “there was a strong sense of what we call presence – the illusion of being there and that these events were really happening.”

Another observation was that people said that “in reality, when they go to a concert, they never go alone – they always go with friends and family,” he adds. “So, we built another environment where you can attend the concert, but with other people that you know,” using avatars (virtual characters) that look like them.

That model awaits study, Slater says, and the goal is to test if this alleviates some of the experience issues they observed. “My intuition is that this will quite radically change how people respond to the overall audience and to the concert itself,” he says.

What also takes a virtual concert to the next level is the live filming – a key difference between the quality of the Dire Straits version (using computer graphics to turn an old concert video into a virtual experience) and AmazeVR’s approach of having the musician perform directly to the camera in 3D. But what remains tricky is convincing the brain it’s really happening right in front of you, according to Slater.

“We look at television and we go to the movies and everything we see is very realistic. But we know the people in the movies are not seeing us,” he says. “(Virtual video) is not completely effective because some part of your brain is saying, this is video, they’re not seeing me … So there are advantages and disadvantages.”

Interest on the rise

While the idea of a virtual concert versus the real deal could seem like a tough sell for hardcore music fans, there are benefits to scaling back in the physical world.

According to the UN Environment Programme, “live concerts and tours can contribute to the climate crisis, driving up emissions through fan and artist travel, energy consumption and the mass production of merchandise.” Meanwhile, rising ticket prices to some of today’s most popular concerts, including Taylor Swift’s “Eras” tour and Beyoncé’s “Renaissance World Tour,” are pricing people out (though also boosting local economies).

“A lot of artists don’t have a chance to have a big concert because it needs a big budget, all the production and everything is so complicated,” says AmazeVR’s Kim. “But even small or big artists, (with VR), it doesn’t matter. You can always make your own VR concert and share it to all the fans all over the world, and I think it’s giving a very good, fair opportunity to every artist and every fan.”

AmazeVR’s app debuted on Meta App Lab and SteamVR on July 13, before it releases to all major extended reality (XR) stores, including Apple Vision Pro. According to AmazeVR, pricing is currently per concert; it will cost $6.99 for one year of access to the T-Pain concert, for example, after that content launches.

A collaboration is in the works with K-pop group aespa, while AmazeVR is “in discussions with many labels and management companies,” with interest in VR concerts on the rise since Apple’s headset announcement, the company says.

It is also producing and releasing mixed reality (MR) concerts – a combination of virtual and augmented reality – by overlaying virtual elements onto live, real-world video. This will allow fans to “interact with the music artists using hand gestures,” the company says, an element that Slater believes can go a long way in how our brains perceive and accept virtual environments.

Regardless of the platform or the venue, it’s still all about the music, says T-Pain.

“Even when you’re not at a concert … kicking it with two friends and putting on my music is kind of like a party,” the rapper adds.

“I think the music kind of does that on its own.”

CNN’s Leif Coorlim contributed to this report.

The-CNN-Wire
™ & © 2023 Cable News Network, Inc., a Warner Bros. Discovery Company. All rights reserved.