360-degree video goes mainstream, this DYI tinkerer offers tips as he finds ways to use the immersive technology for live events like CES 2016.
In just five months, Khoi Nguyen took his curiosity to the cutting edge of 360-degree video making, a fledgling trend poised to become pervasive in 2016. Rather a digital video camera or smartphone, he’s using new technology that allows him to synchronize many cameras to capture everything in front, back, up, down and all sides of the camera.
Nguyen is already moving beyond making 360-degree “spherical” videos to forging a path for others interested in using this immersive technique to cover live events like debates or sports matches.
It requires as many as a dozen — or more — interlocking cameras, a powerful laptop computer, extremely fast data transfer technology, an internet server and lots of trial an error.
“My dream is that one day the capability can fit in the palm of your hand, so people can use their phone to live video chat while the other person controls the 360-degree camera perspective,” said Nguyen, a technical marketing engineer at Intel.
He admits it remains a far-fetched dream today, but nonetheless he’s pushing the limits of technology to make it happen. He believes 2016 will bring a proliferation of ideas, tips, and tools that will attract more filmmakers into this technically challenging medium.
Nguyen was on site to help Intel with video needs at CES 2016, where a team demonstrated how 360 video could be captured and shared live. He connected a camera rig to a laptop powered by 6th Generation Intel Core i7 quad core processor, which captured and fed video to a VR headset and a variety of mobile devices. The VideoStitch Vahana VR app took live 1080p 30fps raw footage via HDMI cables from six GoPro cameras, stitched the live footage into 360 degrees and delivered a 4K quality live video stream. The aim was to show how the technology is evolving and becoming more accessible.
Many 360-degree video makers today, like Nguyen and the tightrope walking Flying Frenchies, rely on ingenuity, existing tech and sheer willpower, but they also seek help from the growing online community that constantly shares new knowledge.
Dive in
Nguyen, an amateur video maker, wanted to find innovative ways to push new PC technologies to their limits. After learning about stitching together 4K quality footage captured simultaneously from several to a dozen or more cameras, he knew he wanted to take 360-degree video making for a spin.
Since then, he has fallen deep into the details of 360-degree video creation, with a growing passion for the art and technology. He pays it forward by sharing tips for creating 360-degree videos with others.
“I geek out on this stuff,” said Nguyen. “When I explain how it works, people don’t really get it until I pull out my Google Cardboard, put in my smartphone and show them the video I made of my daughter at the playground.”
What makes these video unique is their ability to be controlled by viewers, who can change perspective by touching the left or right side of a tablet or computer screen, moving a handheld or Google Cardboard mounted smartphone left, right, up or down or just by moving their heads while wearing a VR headset. It makes viewers feel like they’re standing in the location of the camera, with the ability to look all around.
Unlike true virtual reality experiences, these 360-degree videos can be enjoyed without VR headsets.
Popping up everywhere
Major media players like The New York Times and Wall Street Journal took subscribers into the 360-degree video world this year with behind-the-scenes looks at a street artist and a ballerina at Lincoln Center.
More than 30,000 of these videos by professional and amateur videos are available on YouTube already, and now people can see them on Facebook.
For CCS Insight technology analyst Ben Wood, 360-degree video experiences seemed to come out of nowhere this year and now are everywhere.
“I see 2016 as a watershed year in terms of awareness of these virtual reality experiences,” said Wood.
He expects to see it used extensively by the media and entertainment industries, but also by travel, real estate and other businesses that want to emotionally entice customers with immersive experiences.
“It’s the first technology that I’ve ever seen that’s had such dramatic impact on people after they see it for the first time,” said Wood. “It absolutely blows people’s minds and now it’s utterly accessible.”
Wood expects to see lots of new high-end, professional and prosumer 360-degree video gear, as well as lower cost, simple-to-use gear for consumers at the 2016 Consumer Electronics Show.
Tips for comrades with 360-degree cameras
For the new wave of creators making these videos, there’s a growing collection of software, production knowledge and tricks.
Nguyen has become “the guy writing the book on 360 video” inside Intel.
To get started, Nguyen used six GoPro Hero 3+ cameras and rig from 360Heros in order to capture the Intel Developer Forum event in San Francisco last September.
“We showed how to create these type of 360-degree videos using the stitching software calledKolor AutoPano,” said Nguyen. At that time, Intel was working with Kolor to optimize the software to run on Intel Iris Pro integrated graphics on 5th Generation Intel Core processors.
In November, Nguyen updated his equipment so he could capture footage of the Intel Extreme Masters gaming competition in San Jose. He upgraded to six GoPro Hero4 cameras and a rig from Freedom360.
“I had more experience this time,” said Nguyen “I knew how to configure each of the GoPro cameras, and how to do Motion Synchronization instead of Audio Synchronization. I learned how the cameras capture everything, so this helped me place the camera rig in the right spots to get the best shots. I also used better tripods.”
To prepare for the Consumer Electronics Show in January 2016, Nguyen made his own 360-degree camera rig using a free, open-source 3D-printable design.
“It’s something anyone can do,” he said. “They just have to find the right design they like and a 3D printer.”
While he’s moving into the next frontier of live 360-degree video, Nguyen make time to share insights along the way. He recently met with Christopher Coppola’s students at San Francisco State University, as they prepared for a 360-degree video they were creating to compete in thePAH Fest 2015.
The PAH Fest is among several film festivals and competitions closely followed by Nguyen’s teammate, Mike Gendimenico, a technical marketing manager and seasoned photographer. Gendimenico’s friendship with Coppola over the years led to the idea of getting young film students to take a turn at making 360 videos. The two teamed up with Intel’s Jerry Elkins, Kurt De Buck and Mike Rivet and devised a challenge for students to create a new music video for singer Noa Neal, who worked with Intel last spring to create a 360-degree video for her hit song , Graffiti.
On the day Gendimenico and Elkins equipped Coppola’s students with camera gear, laptops and software, Nguyen talked to students about things he learned in the past six months.
“One of the first things you have to do as a 360-degree camera operator is to get yourself out of the shot,” he said. “Everything is in the shot, including the mess in the background, the gaffer tape and the lighting equipment.”
He said the camera doesn’t pan, it actually captures everything, so strategic placement and stabilization is essential.
“You have to avoid ‘micromovements’ or it’s difficult to stitch together footage from multiple synchronized cameras,” he said.
He also advised students to keep their distance.
“Shoot at least four or five feet away from subjects or else there will be distortion,” he said. “Also, film at an average height of a human to give best viewing experience.”
Since there are multiple cameras, directors and producers need to keep actors away from stitch lines, where the camera views overlap, or else the actors will get cut.
“In the future this problem may disappear as software improves,” he said.
He advised to pre-plan everything and meticulously label captured footage.
“Post production relies on sorting through footage from multiple cameras,” said Nguyen. “Place a premium on your organization early on for success through the stitching and editing process.”
He explained that the standard projection of 360-degree videos is “equirectangular,” and output should be 2:1 ratio. Rather than seeing a flat map of the earth, the earth gets mapped into a sphere using orthogonal coordinates.
“Because of the 2:1 ratio aspect, a 4K quality 360 video has a resolution of 3840 x 1920,” said Nguyen. “By contrast, a 16:9 ratio, which is standard HDTV, 4K quality 360 video resolution is 3840 x 2160.”
To give a sense for what the final file size is like for a 360-degree video, Nguyen pointed to the 43-second video of his daughter playing in the park, which was created with 4K quality cameras. That compressed mp4 file is 157MB. This equirectangular video needs to be played back using a 360 video player app like Kolor Eyes, VideoStitch player, YouTube or on Facebook.
Six weeks after getting equipment and training, the student videos were complete. “San Francisco in 360 Degrees” by Max Serwitz, Jacob Phillips and Diego Murga went on to win a PAH Fest award.
The student filmmakers also won a trip to Las Vegas, Nevada to attend CES 2016, where they helped Intel create a 360 video of Neal performing her new song, Wildheart, live at the Intel booth.
While it’s exciting to watch these videos, Nguyen is getting an even bigger thrill out of making them.
“I get more fascinated the deeper I get into the emotional wonders and technical challenges of the medium,” he said.
