Bringing VR ballet to new audiences with a dual fisheye lens

Ballet has long captivated audiences with its mesmerising blend of movement, music and storytelling. However, traditional theatre environments can be overwhelming or inaccessible for many neurodiverse individuals. But what if you could redefine the way ballet is experienced by harnessing the power of virtual reality?

Clive Booth, whose work focuses on atmospheric fashion, beauty and portrait imagery, loves working in the arts and is passionate about embracing the latest technological developments, so he was the ideal filmmaker for an ambitious virtual reality project with the Canon RF 5.2mm F2.8L DUAL FISHEYE lens. His goal was to record an immersive performance of Swan Lake for the Freefall Dance Company, Birmingham Royal Ballet's sister company for neurodiverse dancers, which would also be shown to a thousand students across six schools. "The idea was to bring a neurodivergent audience closer to the dancers," Clive explains. "These are young people and adults who wouldn't normally be able to go and see live theatre."

Clive had worked with Birmingham Royal Ballet in the past, but the launch of the Canon RF 5.2mm F2.8L DUAL FISHEYE lens opened up new opportunities for VR and immersive storytelling. This was an opportunity for Clive to apply his decades of experience to an exciting new technology and create something vitally important. "I immediately thought, 'Wow, what can I do with this lens?'"

Tom Rogers, the creative digital producer at Birmingham Royal Ballet, added a valuable dimension to the creative collaboration. A dancer with the company for 18 years, he could immediately recognise the creative possibilities of filming a live show in VR.

"Clive and I embraced the idea of immersive storytelling," Tom explains. "It makes complete sense because you're putting the audience in the space with us. There is so much going on behind the scenes, so being able to put someone on stage with our dancers in the wings gives the viewer unparalleled access. It's quite an emotional experience."

Like many creators – and viewers – Clive is new to VR. Prior to starting the project, he spent six months working with the product team at Canon to find out what was possible, how to overcome issues such as framing shots so that the filmmaker is not in the scene, and how to edit the resulting files.

Canon's EOS VR Utility software was a key part of the process. It simplifies your workflow by automatically processing compatible RAW video files taken with, for example, a Canon EOS R5 C. After they've been processed, you can make basic adjustments, such as white balance, ISO and brightness, before exporting the files into various resolutions, ready for further editing. EOS VR Utility really is "the glue that brings everything together," adds Clive.

Learning to film in VR requires an understanding of how the human eye and brain perceive movement. "At less than 60 frames per second (fps), the human eye can see if the movement is not fluid," Clive explains. "But over 60fps, we can trick the human brain into thinking that's reality. Canon provided me with the updated EOS VR Utility, which enabled me to process RAW files shot at 60fps on my Canon EOS R5 C and create beautiful, fluid movement from the dancers."

The low-light environment in the theatre posed its own challenges. With his EOS R5 C, Clive had base ISOs of 800 and 3200. He tried to keep it at ISO 800 as much as he could, but there were times when he was filming backstage and in the wings when he had to go up to ISO 3200. However, the camera's excellent noise reduction combined with the use of a plug-in in Adobe Premiere Pro, dramatically reduced the noise in post-production, resulting in clean shots with lots of depth of field.

Clive says the EOS R5 C is undoubtedly his camera of choice for shooting VR content. "I am extracting every ounce of what that camera can do," he says. "There aren't many cameras that can shoot 12-bit 8K Cinema RAW Light at 60fps for 50 minutes non-stop. That camera is faultless.

Take a look behind the scenes of filming Swan Lake in virtual reality:

Immersive audio was crucial to the success of the story. Clive needed to "lock" the sound to the environment, so he used an ambisonic microphone made up of four directional mics to create what he calls a "bubble of sound". "If, for example, there's a grand piano over here, I can hear it through one ear," he explains. "If I move my head, I hear it through the other ear."

The sound bubble [spatial audio] recorded what was on all sides of the camera, but Regan was also interviewed wearing a microphone. When you view the film on a headset, you can see Regan and hear the ambisonic audio, but Clive also got a clean voiceover that he could lay over the scenes without Regan, to help continue the narrative.

Despite expanding into new technologically-advanced platforms and shifting terabytes worth of data in the process, Clive says there is a liberating simplicity to filming in VR. "It's one camera, one lens, a tripod and a mic, and that's pretty much it," he says. "It's quite approachable."

For Clive, though, the most enjoyable aspect of this project was being given the tools to inspire the next generation. "Working on meaningful projects is very important to me at this stage in my career," he explains. "Like many people, I want to leave a legacy, so being able to bring ballet to young and neurodiverse people is wonderful. But to go a step further, the story is about the challenges a ballerina faces, and the way that she faces those challenges. Other young people can look at their own lives and the challenges they face and hopefully learn from Regan."