Oscar Raby's The Turning Forest, screening at MIFF via Oculus alongside Raby's Easter Rising: Voice of a Rebel.
Virtual reality looks set to break into the mainstream in 2016, with Australia's film festivals leading the way. The Melbourne International Film Festival kicks off this Thursday, and one of its most exciting new sidebars is its VR program.
In this extract from the latest issue of IF, we chat to those leading the way in the new medium (including Melbourne-based Oscar Raby, whose work is being showcased at MIFF) to get the lowdown on what's happening in VR – and what's ahead.
Australian filmmaker Matthew Bate first experienced virtual reality at last year’s Sundance, where his feature documentary Sam Klemke’s Time Machine played in the festival’s New Frontier sidebar.
“I'd never experienced it before, and I remember watching a couple of VR works and standing up and declaring, ‘this is the future of cinema’!” Bate tells IF.
“Which it isn't, it's definitely not (laughs). But it is an evolution in the way we tell stories.”
Bate returned from Park City and teamed up with choreographer Gideon Obarzanek on a VR work that would put viewers on stage, surrounded by professional dancers.
“I had this horrible analogy of it being like a lap-dance with Sydney Dance Company. Instead of sitting in the seats looking at them, they would be surrounding you.”
Stuck in the Middle With You was commissioned by ACMI, where it becomes part of the permanent Screen Worlds exhibition from late 2016.
Bate worked with Adelaide VR company Jumpgate, and the nature of shooting 360 meant that a regular film crew was impossible.
“We had the same lighting director that had done the stage play. It was a theatre lighting crew, not a film crew.”
“Normally, you sit next to the monitor or look through the camera. With this, you literally have to press record and walk away. That was really quite a scary thing, because you have no idea what you’re actually getting. You can't play it back yet. The technology is so embryonic. We're in the Pong stage, if you compare Pong to PS4.”
American filmmaker Rose Troche (Go Fish, The L Word) also cottoned onto the potential of the new medium at Sundance.
Troche is an old friend of New Frontier chief curator Shari Frilot, and became interesting in exploring VR after a visit in 2013.
“Shari set me up on a blind date with this guy who had been in VFX for a while and then moved into VR, Morris May,” Troche says.
May and Troche have since collaborated on two VR projects: Perspective Chapter 1: The Party is about a rape at a college party, in which the viewer can adopt the perspective of both rapist and victim. Its sequel, Perspective Chapter 2: The Misdemeanor, goes inside a confrontation on the streets of New York City between two black men and two white police officers.
For Troche, the move into VR has allowed her to hit restart on her career, even though, so far, she’s had used her own money to bankroll both projects.
“We're all trying to figure out how to monetize this. The beautiful part about what's happening right now is that nobody quite knows, so we're all creating work because we want to create work in this space.”
“I'm looking at grants that people who are just getting started are looking at. Suddenly I get to be new again, and that's really exciting to me. It's also at times humbling, because I have twenty years under my belt, but I love the evening of the playing field.”
That sentiment is echoed by Martin Taylor, the co-founder and head of content at Sydney-based VR content studio Start VR.
“I was just over at Vegas for NAB, and everyone's having the same problem,” Taylor says.
“It's such a new medium that really large budgets are not getting to the solutions any quicker than smaller budgets. It's really about the strength of ideas, and [having] the grit to solve some of the storytelling problems that this new medium has.”
An opportunity to work through some of those problems arose when Start VR COO Nathan Anderson caught up with AFTRS’ Neil Peplow at last year’s Screen Forever conference in Melbourne.
AFTRS and Start VR have now collaborated on VR Noir, a “VR crime thriller” that launched this month at the film school.
“We went in to teach the film school about virtual reality production techniques and then launched into VR Noir as a practical project, with the teaching staff as crew members. We designed and built it, and are overseeing the project.”
Post was done with Frame Set Match, who climbed onboard as production partners.
For Taylor, the challenge for VR is creating an entirely new grammar, radically different to film.
“All traditional production is designed around film language to be presented on a flat screen. People automatically assumed that all those things would translate into a surround environment but they don't. You can't rely on editing as a basic narrative tool.”
As for the standard parallel made between VR and gaming, Taylor thinks it only applies up to a point – “Even game storytelling is usually done on a screen,” he points out.
“We're finding that VR is more a blend of a radio play, because sound is much more important in an immersive environment, a stage play – because you're plonked in the middle of the action – and a magic trick, because you need to rely on tricks of misdirection.”
According to Oscar Raby, of Melbourne VR house VRTOV, it’s not only the quality of the foley that’s important.
“It’s about how the separation between those two layers, audio and visual, can increase the narrative capacity of the piece. It doesn't have to be diegetic sound all the time, so it's important to have sound behave the way sound behaves in space.”
VRTOV’s work, which has screened at Sundance, Tribeca and now MIFF, is built using the standard game engine Unity.
“We start by doing 3D scans, with an infrared sensor that captures the depth information of whatever object is in front of it. By combining a set of infrared rays and the colour information of the camera, the 3D scanner creates one set of volumetric data.”
“That gets put inside the game engine, and it allows you to move the camera in real time, so whatever the user is doing – say, moving sideways or leaning in to the object – the object would look like it's in place, not flattened like a video.”
Raby’s background is in performance, rather than filmmaking, and he sees that distinction as crucial to his brand of real-time interactive VR.
VRTOV’s work is all about “what you do inside the story, and how your connection to the story comes out of your actions. Which is something you find in games.”
“Using a game engine like Unity, you don't have a timeline. You don't have starting points and you don't have an end point. If you come from a filmmaking tradition, you need to chuck the timeline out the window.”
For all the excitement in the air among creators, the consumer angle is murkier.
“Distribution is a nightmare at the moment,” says Amy Nelson, a producer at ABC Research and Development.
The ABC produced their first VR work, Warwick Gold, last year, after years of pushing from the R&D team.
“We've tried distributing Warwick Gold through the Oculus store and of course it's possible to do but it wasn't straightforward. There were lots of glitches. Every time those guys update their software or the OS updates there are always bugs.”
Fully investing in VR remains tricky for the national broadcaster.
“We're mindful that we have to think not just of our metro audiences – that people in regional areas need to be able to access content too, and broadband speeds are just awful.”
Samsung’s Gear VR headset is already available, with the HTC Vive and Facebook’s Oculus Rift entering the local market and Playstation releasing their very own VR headset in the second half of the year.
Taylor thinks Playstation could dominate.
“They'll be launching with over a hundred titles, and the titles are really great. That's going to be a no-brainer for so many people.”
“The Oculus Rift and the HTC Vive are absolutely incredible pieces of equipment, but you need to know about computers. You need to have a powerful computer, and you need to have the space.
“Samsung are obviously the early adopters in terms of the VR phones, but not everybody likes Samsung phones.”
Then there’s mixed and augmented reality, which many, including ABC R&D’s Nelson, predict may be even bigger than VR.
Nelson points to Microsoft’s HoloLens headset, still available for developers only in the U.S.
Using the HoloLens, “you can still see the room or the space that you're in but you have amazing rich immersive graphics over the top of the environment that's around you.”
“It's not like VR where when you put the headset on everything's blocked out and you're immersed in a completely separate digital world. It's a little more social, and it’s not as taxing physically.”
One of the biggest players in the mixed reality space is the Miami-based Magic Leap, backed by Google and actively working with Peter Jackson and his Weta Workshop to produce content.
The race is on.