Filming in the Metaverse

The Metaverse is set to become the home for thousands of virtual communities—especially when the price of the technology comes down. There are already productions taking place within these communities to document the growth and extent of the content inside. But what are the rules for production, and how do you shoot anything?

The Metaverse is a phrase that we should all know by now. Facebook has even changed its name to reflect this virtual world’s powerful potential, and they hope to play a big part in its success. But what is it?

To get a wildly overblown and Hollywood-ised idea, you can watch some of Steven Spielberg‘s Ready Player One film. It’s Hollywood 101, but you’ll get a general idea of what the Metaverse is (or could be).

You can also watch a six-year-old Vimeo film called Hyper-Reality, a version of what’s ahead but, indeed, a chaotic vision of the future. “Where physical and virtual realities have merged.”

2022 will be a massive year of growth for the Metaverse, with major companies announcing their move into the space. Fan-based activities—like sports—are ripe for development. English soccer team Manchester City has recently announced a collaboration with Sony for their flavor of a virtual community.

The official press release states:

Both parties are aiming to create a global online fan community where fans can interact with City and each other within the Metaverse, which will be a virtual recreation of the club’s Etihad Stadium. Sony will support Manchester City to increase further value of its content and engagement with its fans around the world.

– Manchester City

The impression you get by seeing other people’s view of what the Metaverse will be shown that nobody knows yet, but the hype is massive. So, how do you make content for a substantial new 3D content-hungry world?

At Sundance this year, there was a documentary called We Met in Virtual Reality. Director Joe Hunting wanted to cover the beginnings of communities that inhabit the Metaverse now and decided not to shoot some reportage of the real people who become avatars within the world, but shoot the world itself.

For this, he started learning how to shoot within VR Chat, which is an online virtual world platform.

He used a virtual camera or Avatar add-on designed for the platform called VRCLens from a developer and member of the VR Chat community called Hirabiki. The camera is essential, but virtual is open to fast-track development paralleling Metaverse’s trajectory.

Using VRCLens

First off, to use the camera, you have to enter VR Chat after taking on the shell of an Avatar. The camera is summoned within the chat. Like a physical camera, you start by choosing which lens you want for the scene. You choose your exposure and allow for zooming in and out. So, the main features of a standard camera are there to use—apart from that, you can fly your camera like a drone.

Joe quickly realized that he had enough variety within the camera to make his film work. But, the biggest advantage was that he would be shooting from within VR and not looking in.

It was important to me that the place within VR Chat was represented by the people in it.

– Director Joe Hunting

The people inside were not shy to talk about their real feelings and challenges, especially in the shadow of the pandemic.

Joe decided to shoot handheld within VR Chat, like a real documentary self-shooter. He ended up being in nearly every scene. But also the inhabitants of VR Chat could see that Joe was holding a camera, which helped push the reactions that he was after.

It was as organic as a physical process would be.

– Director Joe Hunting

Ironically, the biggest challenge Joe found was recording the sound. The audio recorded is spatialized, so sound comes in from a wide soundstage like an ATMOS recording. To then remix that to stereo was a challenge. Also, the length of filming was sometimes challenging—he’d be shooting for four hours without stopping.

His post-production used extensive grading to lift the aesthetic into something that exuded hope and light, mainly as his interview subjects were encouraged to speak about their mental health in the middle of COVID.

The New VRChat Camera

VRCLens is an Avatar add-on virtual camera for VRChat who has recently improved its camera—CooperTom demonstrates it below. It does introduce more features like control of depth of field and multiple focal lengths. There are also green screen controls, anchoring camera controls, and filters in the last version.

At the moment, it all looks very “Minecraft,” but things will change rapidly, especially when the big game engine companies get involved and bring their expertise in building 3D worlds. More of that later.

Benefits and Problems of a Virtual Shoot

The production of the documentary took a year to achieve. This included location scouting, test shoots, and collating all the data, but Joe concluded that it was easier to achieve than physical production. He aimed to record the highs and lows of why people increasingly inhabit these worlds.

He trained his virtual camera on various activities, including pool games, dances, and educational classes, to tell the stories of several couples who met in VR during the COVID-19 lockdown. It might have been virtual, but the director wanted real stories.

Watching some of Joe’s films, you can’t help but realize how essential the quality is and how early the technology is—it’s a threshold moment.

The movement of the characters is limited by the lack of any in-depth motion capture apart from VR headsets and some devices called Valve Index Controllers. These are like motion capture for your fingers. But, this again is extremely limited—you can’t even cross your fingers.

Enter the Gaming Engines

The two big gaming engines are Unity and Unreal Engine from Epic Games. Unreal has been prolific in entering the film world and driving certain aspects of virtual production for TV and film.

Unity builds worlds but, by their admission, have been dragging their heels with crossing over into film. However, their recent purchase of Weta Digital might change all that.

What is the Metaverse, if not a giant 3D construction? Weta Digital is known to be pretty adept at realizing 3D moving images, and Unity’s billion-dollar buy-out includes the brain’s trust of such 3D assets. Unity had the Metaverse in mind when they spent the money and predicted worldwide access to the assets through cloud-based enterprises.

Marc Whitten, Unity Create SVP, and GM commented on the deal, and its meaning for the Metaverse.

The key for me [is that] the Metaverse is going to need more 3D content. It will need an extraordinary increase in the number of people capable of building in 3D. From a Unity perspective, we started thinking hard about how we could build something that democratizes content creation.

– Marc Whitten and GM

If you’re at all cynical about the potential of the Metaverse, then you’re not alone. It all feels quite nebulous at the moment, and the potential far from the sweet inhabitants of Joe’s new documentary. But a 3D internet is likely to happen, and it’ll need content creators to build the world.

We’re right at the start of the adventure, but development will be swift. And, you can be sure, we’ll be here to share the information when available.

For more filmmaking advice, tips, and tricks, take a look at these gems:

Cover image via A. Solano.

Source link
Studionics The Best Media Production house in Coimbatore

Category: ,
%d bloggers like this: