Shooting immersive 3D VR videos with Canon's revolutionary dual fisheye lens

Destination sports such as skiing, ski touring and paragliding are as much about the setting as the physical activity, but a video will tend to do justice to only one of these elements at a time. What if you could experience the action in immersive 3D VR, with the freedom to look around a full 180°?

Mountain biking enthusiast and Canon Ambassador Martin Bissig built his outdoor photography career with the mountainous region of Davos in Switzerland on his doorstep. His startling images of extreme biking terrain and daring riders such as trials cyclist Danny MacAskill have been published in more than 20 countries, for clients including National Geographic and Red Bull. Whether up his local mountains in Switzerland, further afield atop Croatian cliffs or in the wind-carved sandy deserts of Oman, Martin is known for the jaw-dropping scenery in his photos. Both the extreme sports activities and surrounding environment are crucial story components for his clients in the experiences industry, and the challenge for Martin is to bring both these elements to life.

"When I go out on shoots in nature, it's not something you can easily recreate with photos and video, because you miss the 3D look and that sense of being surrounded by the scenery," he says. This has all changed with the revolutionary Canon RF 5.2mm F2.8L DUAL FISHEYE lens designed for shooting stereoscopic 180° VR.

"As a professional, I'm always adventurous and curious to try out new technologies, techniques and gear," Martin says. "I don't know of any other lens like this, and Canon's EOS VR System gives me everything I need to shoot VR. With the EOS R5 and the EOS VR Utility application from Canon, this system allows me as a photographer and filmmaker to jump into a new field of work."

For his first production with the dual fisheye lens, teamed with the EOS R5 he uses regularly, Martin shot an immersive stereoscopic 3D 180° VR video for the Davos tourism board to showcase some of the winter activities possible amidst stunning snow-covered mountains, including cycling, snowshoeing, ski touring, skiing, paragliding and sledding.

The EOS R5 teamed with the RF 5.2mm F2.8L DUAL FISHEYE lens can create high-resolution 3D VR videos with excellent dynamic range and colour reproduction thanks to the capability of the EOS R5's 8K full-frame sensor. This is the kind of video quality previously possible only by using hefty multi-camera rigs with lengthy set-up times – things that are simply not an option when shooting action on the move, on a deadline, across physically demanding terrain.

"I shoot 75% stills to 25% videos, and after two years with the EOS R5 it is still the best camera I have ever used for the work that I do, when it comes to image quality, speed, size and weight," says Martin. "Now having the possibility to attach the VR lens and transform my favourite mirrorless camera into a VR camera, that's amazing!

"I was surprised when I saw how small and compact the lens is, which makes it super easy for me to carry the kit around and put on a gimbal," he adds.

"What I like most about the concept of VR for sports is this: when you are moving in a really special environment where action is taking place, the viewer has the freedom to look around 180° and check it out," Martin says. "The athlete or sport itself is not the centre of attention, but the whole of the scene."

It took Martin some trial and error to get the balance right between subject and scenery. "For the 3D effect, I found it worked best with the subject in action being around two metres away, with an attractive view in the background to show the beauty of nature."

Working so close to active subjects while he was also on the move required constant communication, and some experimentation with camera movement. Martin found that limiting the speed of his subject and the degree of camera movement helped distinguish immersive 3D VR videos from those that are nauseating to watch. After all, the camera is on the move, but headset-wearing viewers are stationary.

Martin usually shoots regular 4K video, but 3D VR with the dual fisheye lens requires the camera's maximum 8K resolution to produce a high resolution 3D VR file with a final output file size of 4K per eye. That's because the inputs from the dual lenses, which replicate the left and right eye views, are simultaneously recorded onto the EOS R5's sensor, side by side.

To get headset-ready, the left- and right-eye circular fisheye images need to be swapped over in position and converted into two flat, square side-by-side 4K images (known as an equirectangular projection, used for VR output) using the Canon EOS VR Utility software. The exported VR file can then be edited using any suitable video editing application. Alternatively, Canon's EOS VR Plugin for Adobe Premiere Pro* can be used instead to simplify this whole process into one step, by automatically converting the original dual fisheye image and importing it directly into Adobe Premiere Pro's media window at the same time. Once the conversion is done, a regular video editing workflow takes over. "I shoot video with the EOS R5 in Canon Log 3 with the BT2020 colour space, plus Canon LUTs profile to convert straight into Rec. 709 standard footage," Martin says. "If you know how to colour grade regular footage from your camera, there's no real difference when colour grading VR footage. No additional knowledge is needed – it's that easy."

Before the final edit, Martin uploaded some of that day's videos to his headset to evaluate which sequences worked best in 3D VR. He found that the kit worked especially well in the smooth paragliding scenes he had shot, where the viewer can look all around at the equipment and down through his legs to the distant landscape beyond for a real first person view (FPV) experience. Fast movement was more challenging. In one sequence when the skier crossed in front of the camera, the EOS R5's 8K selected frame rate of 25fps (PAL) struggled to create smooth subject movement. The RF 5.2mm F2.8L DUAL FISHEYE lens can be used with the EOS R5 C, which offers 8K at 30fps, or 60fps (using RAW LT with an external power supply and requiring a different RAW workflow/third-party VR conversion application), but this becomes a larger and less nimble setup, while for Martin it was the portability as much as the quality of the camera and lens that made immersive sports videos possible.

"For its size, the image quality of the lens with an EOS R5 is amazing," he says. "There was never a point when looking at the footage I wished for more detail, certainly with the current hardware and quality of the VR glasses.

"I was shooting outside in a sunny environment with an aperture around f/13, so I had a depth of field from my feet to infinity, and all the detail was crystal-clear and sharp. But while the quality of detail is important, you are so drawn into the three-dimensional world that you are looking around to experience the whole scene rather than looking at how sharp the details are. It feels so realistic, and you are transported into that scene."

Martin compiled two or three clips from each activity into a final video no longer than two minutes for comfortable viewing. "I now have a showreel that I can share for potential clients to view on a headset. This is the closest you can possibly get to experiencing an area and activity, and that is going to be the future.

"I would say that we are in the early adoption days of VR, and I think this technology has huge potential in the outdoors, tourism and destination industries."

*Adobe and Adobe Premiere are either registered trademarks or trademarks of Adobe in the United States and/or other countries.