Floating in the clouds with Virtual Reality 360 Video
Our studio wanted to create something classy for Valentine’s Day while having fun with the latest technologies. Some brainstorming and there was no question, we would float with our friends and loved ones in the clouds with balloons
VR is a huge buzz these days with the Cardboard in 5 million user’s hands and Oculus and other hardware about to be released. Mechanism Digital’s team has enjoyed producing spherical content for about 15 years. In the beginning we created imagery with nodal tripods which allows an array of multiple (usually 14 to 38) photographs to be shot in all directions (not at the same time) placed at the same no-parallax point, also known as the “entrance pupil”. The camera is carefully rotated between each press of the shutter to cover all POV angles of the scene. These photos would then be “Stitched” together using software to make a single file to be viewed in QuickTime as a QTVR file or even some Java formats for web allowing the viewer to pan around from a single point to see in all directions including up and down. The last couple of years the content has made great leaps into the motion/video world allowing action to happen all around the viewer. One of the challenges with video is multiple cameras (typically 2 to 16 cameras in a rig) can’t all use the same no-parallax point as the laws of physics still state that two objects cannot occupy the same space at the same time. The bigger the cameras, the further apart they need to physically mounted which causes double vision as objects get close to the camera rig. This physical size problem has made GoPros popular because of their small format, allowing subjects to get within 12 inches without major problems. In addition to the parallax problems, camera lenses naturally have different distortions in their curvature which can also make stitching images together seamlessly a problem.
As a VFX and digital animation studio, Mechanism Digital takes enjoys working in the computer virtually where we can actually have all camera at the same center focal point at the same time and therefore render mathematically perfect spherical images every time and we don’t even have to paint out the tripod!. Objects can be close or far without any parallax problems and can easily produce stereoscopic content (stereo a whole other compromise I’ll save for another post). Computer generated 360 VR Video is great for taking your audience where you can’t put a camera. We use it for medical education inside the human body at the microscopic scale, architectural visualizations, or for telling stories with animated characters and talking animals. Most live action 360/VR video is for putting the viewer in a physical environment with real actors, and there are creative techniques to combine live and virtual cameras for VFX or even broadcast graphics.
For the Valentine’s Balloon project we used our standard phases of production/design including mood boards, style frames and storyboards to nail down the overall look, feel and animation of the experience. In most 360 projects it is important to create storyboards for all views including front, left, right, back and sometimes up and down. For this project we only had to plan forward and downward views as the content was similar all around the viewer.
Our balloon models were modified and textured in Maya 2016 where most of the work was performed in this project. We used particles with random dynamic motion for the balloon’s wobble and rotation. The balloon models were parented to particles and cached out to render easily on our render farm without having to perform particle run-ups for every frame.
The clouds required the most time to figure out and develop their look. Fangge Chen our lead artist/designer started developing her own clouds using Maya Fluids, but ultimately we found and were very happy with the EMFX Clouds script which can be downloaded from Creative Crash. Fangge still spent a couple days testing the shapes and working on ways to bring render times down as Valentine’s day wouldn’t wait for the long render times which were anywhere from one to five hours per 4K frame. We settled on a quality that looked good for wispy edges and would calculate on our farm in time. One sacrifice for the clouds was to bake/freeze the simulation avoiding the need to run-up for each frame, although this means the clouds would not seethe over time.
With our looming deadline we opted to use the Domemaster 360 Mental Ray lens which reduced render time and avoided the LatLong conversion although it does have a thin seam on left and right if you look closely. See Andrew Hazeldon’s blog for many useful VR tools for Fusion, After Effects, Nuke, Maya. For most client productions we use our custom in-house six camera Maya rig and composite the six 960 square images into one perfect equirectangular frame (AKA LatLong) using the Domemaster’s plugin for Blackmagic’s Fusion. This technique produces arguably better results and is more flexible, but the additional step after the 3D render requires considerable time adding another minute per frame in animations which are often 1800 frames per shot.
We decided not to go stereoscopic for this project as it not supported by the iPhone on YouTube and wanted the experience to be viewed by as many friends and colleagues as possible with and easy distribution.
You can view on a standard computer in Youtube and click/drag around to look in all directions, but the most dramatic effect is when using a headset like the Cardboard or Samsung Gear VR viewer. The Samsung Gear is like a high-end Cardboard which is more comfortable and keeps the light out from around your eyes but can only be used with Samsung Phone. The iPhone currently isnt’ working on YouTube with the Cardboard, but you can still look around in the YouTube app on the iPhone (and other phones/tablets) as if you are peering through a window into another “virtual” world.
The project was fun for the whole studio and pushing the envelope further always adds to our experience we can apply for future client projects. Even with the additional render time and technical needs, we are excited about many upcoming projects in 360/VR!
Check out Mechanism Digital’s page for the Valentine’s Balloons and several other 360 VR video experiences our studio has produced:
Originally published: 02/15/2016