1 of 3
Lead Pastor Jim Burgen. Image courtesy of Flatirons Community Church.
2 of 3
Lead Pastor Jim Burgen. Image courtesy of Flatirons Community Church.
3 of 3
Extended Reality (XR). Augmented Reality (AR). Virtual Reality (VR). Mixed Reality (MR). Where does one stop and the other begin? Between church content creators, the professional film industry, and universities with top digital teaching studios, who’s using what, to what effect — and how can the average church film team leverage these virtual production tools and techniques?
Christian Wood, VFX supervisor at Crafty Apes Atlanta, a professional visual effects company offering services that span the film industry (The Menu, Where the Crawdads Sing, Fantasy Football, and National Treasure are recent examples), as well as cinematics, episodics, and music videos, explains the basic lay of the land: “XR encompasses augmented reality (AR), virtual reality (VR), and mixed reality (MR). While all share overlapping features and requirements, each has different purposes and underlying technologies.”
It’s actually not a bad time to jump into early virtual filmmaking technologies, because it’s a sea of uncertainty, in some respects, and nobody is alone there.
Following are some basic, distinctive elements of XR, the umbrella term that refers to combining real and virtual worlds through current and future spatial computing technologies, within the parameters of its three digital reality components.
AR: a combination of digital and physical worlds, with interactions made in real-time, and accurate 3D identification of both virtual and real objects.
______________
VR: immersive experiences that help to isolate users from the real world, oftentimes via a headset device and headphones designed specifically for VR.
______________
MR: a combination of AR and VR elements whereby digital objects can interact with the real world.
______________
XR at church
Brando Perez is the video director at Flatirons Church, a multisite in Denver, Colorado. His role encompasses determining and crafting the vision of the church’s video across all its campuses, “the look, the feel of whatever we do,” as he describes it. A good portion of his time these days is devoted to figuring out Flatirons’ XR workflow.
Perez says his team uses XR with the goal of engaging attendees more deeply with the teaching pastor’s message on the real-world stage at the church. He doesn’t want people glued to their mobile devices, as they would be in a virtual reality scenario.
“VR is virtual reality where you pretend to be with people,” as he puts it. “But you’re not with people. And we want to build community and get people to interact with each other.”
“The Bible is so visual, with so many locales. It’s going to change live teaching, and we’re just getting started. It’s a serious tool to help people engage.”
—Brando Perez, Video Director, Flatirons Church
At Flatirons, Perez and his team are pursuing XR head on, as best they can, for teaching purposes. “We wanted to place our teachers in the best possible place to be able to illustrate what they’re saying,” he notes. (He points to a Flatirons 2021 Easter service on YouTube as a good example of what his team is doing in the realm of virtual production:
Flatirons 2021 Easter Service.
In their XR workflow, Perez and his team are using Epic Games’ Unreal Engine v.5 software (real-time 3D creation tool for photoreal visuals and immersive experiences), along with a Disguise media server which Perez says works especially well in XR and AR scenarios. The workflow also includes the church’s existing, expansive LED wall at its main campus, along with RED’s Komodo 6K Digital Cinema Camera, which he says delivers the absolute best image for Flatirons’ LED wall setup.
“Our leadership said to get whatever the best camera would be,” he notes. After testing many high-end options, the Flatirons’ film team came back to the RED Komodo at a church-approachable $5K price point, “easy to use and looks great,” he describes.
For Perez, telling the story should be the goal and the standard, of course, and the technology used to achieve it is only a secondary tool. To be effective, XR can’t be intrusive or tacky.
“Imagine you have a pastor who wants to teach something [specific],” Perez describes, “and they want to go from day to night at a very specific moment while they’re talking. We can do that with our setup—change elements live as he’s talking. We can place people wherever they want to be, in real-time while they’re talking. We don’t have to wait for an editor to render out a video file. We can change it out in real-time on the LED wall.”
Perez notes that he and his team are savoring all the possibilities that XR will enable for unique teaching opportunities at Flatirons moving forward
“The Bible is so visual, with so many locales. It’s going to change live teaching, and we’re just getting started,” he says. “It’s a serious tool to help people engage.”
Out-of-this-world XR
A lot of the recent awareness and buzz around XR stems from the sheer awe-factor elicited by Disney+’s space Western series, The Mandalorian, based on George Lucas’s Star Wars.
When tech types far and wide see the series, their mouths water.
The Filmmaker and Professor of Practice in Virtual Production and VFX, Jeasy Sehgal, who teaches in the Creative Media Industries Institute in the College of Arts and Sciences at Georgia State University in Atlanta, breaks it down. (It’s worth noting that Sehgal is the only Epic Games’ Unreal Engine authorized instructor on the East Coast, along with Virtual Production Dojo being the only Unreal Engine-authorized training center on the East Coast, offering industry professionals expedited learning in virtual production methodologies.)
Explaining The Mandalorian phenomenon, he says, “The magic happens when we’re using virtual pipelines and Unreal Engine technology to create any environment or location, in a high level of detail, as well as digital humans and actors, and capturing performers using motion capture technology without having to go onto a location to film a production.”
In Season I of The Mandalorian, Sehgal notes, the show’s creators and crew were experimenting with using virtual environments with real-life actors, putting real life into the virtual world with in-camera visual effects (ICVFX). And through the process of experimentation, new worlds began to open up for the show’s creators.
“Think about traditional cinematography, pre-production, and post production,” he notes. “And then virtual production—where the post-production is happening at the same time as the production. Now pre-visualization is the final finished visual effects in the camera—fire, rain, snowfall …”
When Sehgal is asked to consider Bible stories and how they might be recreated and enhanced by virtual production, he riffs, “Imagine that the pastor is talking about an apostle. The apostle is created as a digital human with a performer behind the stage. The performer rehearses and the audience sits and sees the pastor on the stage—yet he’s the only person on the stage, the other characters are virtual.”
Then, envision this, Sehgal says: “The pastor says something, and the digital human on the LED screen behind the pastor reacts to that, puppeteered in real-time by a performer. Then it’s filmed by the film crew at the church and they’re using ICVFX.”
Or what about creating a Biblical scene, virtually, with XR?
“The pastor is the protagonist telling the story of the journey of Christ, and what if we could bring Jerusalem to life?” he imagines. “Rather than using the LED walls, the magic happens inside Unreal Engine. You could use satellite technology to create the actual topography, for example, King Solomon’s tomb, with digital humans as the characters in the story that the pastor is talking about in his sermon.”
Yesteryear and present-day ramifications for lighting in XR
Consider, Sehgal says, that virtual production’s roots go way back, and that technological advances like XR are actually part of an evolution in cinematography.
“Virtual production has really been happening since the ‘40s and ‘50s, for example, shots of actors running on-screen with a car speeding up behind them,” he says. “It’s not brand new, but today we can extend worlds and even have real-life lighting come into play.”
For example, “Cinematographers and gaffers can set up lighting, and if I want the aurora borealis, they can control both virtual and physical lighting simultaneously to create it and use both with the actors on-stage.”
In Wood’s world at Crafty Apes, he would advise church technical artists using virtual production to keep it simple to get the best looks with a teaching pastor on-stage.
“As always,” the pro notes, “you want to take your lighting cues from the environment if you’re adding CG objects or adding characters to the environment.”
Age-old wisdom that’s never outdated: the story comes first
With so many virtual production possibilities today, thanks to tools like Unreal Engine that let anybody create virtual worlds from their home office if they wish, Wood reminds eager content creators of the cornerstone of excellent content. Using VR as an example, he notes, “The key is to actually utilize the technology to enhance the story. The interactivity needs to be for a purpose—it needs to help convey the story.”
He also adds, “I think a lot of audiences are used to VR by now, and are not ‘wowed’ by it, but rather are looking for it to add to the experience in other ways which help tell the story.”
While Wood may speak of a broad swath of people who consume content today, that span of people includes anyone sitting in the church seats today. In short, every content consumer you meet today is incredibly media-savvy and discerning. Meaning that effects cannot be simply for effect.
Back to Perez’s example of Flatirons’ use of its video wall as a virtual backdrop display, Wood notes, “I’d keep the background simple and subtle. It should enhance the speaker’s message, not distract from it. Keep it in the same color palette as the rest of the environment, and I wouldn’t have too much movement.”
Rick Ramsey, education director of visual arts at Winter Park, Florida’s Full Sail University, which has a new on-campus facility called Studio V1, built specifically to train students on virtual film production—in many projects people are seeing on-screen today—also reminds students of film that story is boss; it’s the make-or-break element in any XR project’s success. (Before he was in visual education, Ramsey was a minister at a large church in Winston-Salem, North Carolina, in charge of all video and audio.)
“Being in visual arts, to me, the visual is the biggest part of storytelling,” Ramsey notes, and yet, “First you need a script—a story worth telling. We teach our students to tell that story within a frame, but that they must go back to the script and storyboard.”
He adds, “It’s almost like with a comic book. You must be able to tell the story without dialogue, and then you know you nailed it.”
Learning opportunities abound
It’s actually not a bad time to jump into early virtual filmmaking technologies, because it’s a sea of uncertainty, in some respects, and nobody is alone there.
“Industry professionals are scratching their heads at the moment,” Sehgal admits. “They’re oftentimes not ready to go out and join a university, college, or other training program, and this creates a big barrier. I’d love to see people come into the university and do research.”
For those with the time and willingness to learn, though, such as amateurs and students of film, there are professional programs, like Georgia State’s and Full Sail’s, for example.
And yet, there are places you can look and begin to learn right from the same computer in the home office that’s tricked up with Unreal Engine.
“For those who are curious about what this field entails there’s a ton of stuff on YouTube on virtual production,” Ramsey says. “And there are a ton of content creators doing mostly green screen, and you look at it and you can’t tell. Because once it’s composited it looks the same.”
Unreal Marketplace is naturally a treasure trove, with tons of environments, Ramsey notes. “Downloading Unreal 5.1 gives you tons of environments to play with, plus tutorials, but to really embrace and understand the technology, I encourage getting as much time and experience as you can learning on an actual virtual production stage.”
Once amateur content creators understand the principles of cinematography, they can emulate the top film cameras in the industry inside Unreal Engine, without spending a dollar.
Another learning opportunity that’s open to content creators who’re busy working out in the field: Epic Games’ Unreal Fellowships, 30-day intensive blended learning experiences designed to give animation, film, and VFX industry professionals serious mastery of virtual production tools.
For content creators with less time to devote to learning virtual production, though, both Sehgal and Ramsey suggest simply going to unrealengine.com, where you can find everything to get started.
Sehgal adds this of training at its simplest. “The easiest way to learn is just to do. And sometimes the best camera is just the camera in your pocket. But you MUST understand the principles of cinematography.” Once amateur content creators understand those principles, they can emulate the top film cameras in the industry inside Unreal Engine, without spending a dollar, he adds.
For Perez, he and his team at Flatirons are talking to others in their world who are experiencing the same pain points and victories.
“We’re all trying to help the big C church,” he states of Flatirons, whose film team is currently sharing its XR and camera tracking findings and other virtual production resources with fellow churches. “If other people are already starting to figure out what that kind of training looks like, we should start sharing with one another.”