Exciting New VR Cameras Appear at F8 and NAB
Last Wednesday on day two of its F8 conference, Facebook revealed two new VR cameras with an amazing new trick: using Facebook’s machine vision technologies, they can recreate the depth of the scene they are capturing as a 3D point cloud. This is similar to other pointcloud-based volumetric VR video solutions like HypeVR, but does away with the need for bulky, expensive LIDAR sensors.
The point cloud data gives viewers the ability to shift their perspective slightly, creating a far more convincing sense of depth than can stereo alone, and preserves proper stereo if users tilt their head at odd angles, a major weakness of current video-only solutions.
Facebook is partnering with OTOY to create workflows for integrating the camera’s output with Octane Render elements and compositing them in After Effects and Nuke. They are also working together on playback formats that will preserve the point cloud data with 32-bit floating point resolution (a standard video depth map can only preserve 8-bit depth, but they may also support this format for mobile playback).
The new cameras are christened the “Surround 360” and are available in two configurations, the x6 and x24, with 6 and 24 cameras respectively. Each is roughly the size of a soccer ball. The sensors are provided by FLIR (also known as Point Grey) which provided the cameras for Facebook’s previous open-sourced VR camera design.
Coming in second place in the camera race this week was Google, who is offering a revamped version of their Jump camera system. First announced a year ago in the guise of the 16-GoPro Odyssey rig, Jump offers very good automatic stitching using optical flow algorithms to interpret novel viewpoints that the sixteen cameras alone cannot capture. However, Odyssey had an Achilles’ heel of sorts: a blurry hole in the footage at the north pole (directly overhead).
The new camera system adds a 17th sensor to address this oversight. Based on cameras from YI Technology, an action cam company that competes with GoPro and is backed by Chinese handset juggernaut Xiaomi, the new Jump is lighter weight and more refined, with an external control panel and a companion app. It also adds the option to record video at 60 frames per second at 6K, while preserving the original camera’s ability to record 30 frames per second at 8K.
For its part, YI also announced a portable 2D/360 camera called the YI 360 VR, which looks to compete with the likes of the new Samsung Gear 360. The camera features a chunky industrial design reminiscent of a reporter’s cassette recorder, and is capable of live streaming video at a 2.5K resolution.
GoPro meanwhile announced a very similar offering in the form of the GoPro Fusion, which uses back-to-back sensors to produce a 5.2K spherical image. Its differentiating feature is the ability to crop out a standard 1080p frame in post, allowing you to frame action you might have missed with a standard GoPro. It should be noted that any VR camera with sufficient resolution can pull off this trick, but GoPro is betting that the extra 1.2K resolution over similar cameras like the YI and Gear 360, and better software tools for the cropping task, will make the selling point stick.
Digital Domain has been experimenting with custom made VR cameras for years and has now announced a commercial product in the form of a spherical 4K camera with a refined workflow and custom stitching tools.
There were too many VR cameras on display at NAB to list them all here. Cristian Dominguez Rein-Loring posted a photo album on Facebook showing the tableau. You’ve got to see the crazy Arri Alexa rig which looks to be about the same size as a SmartForTwo.
F8’s Day Two Keynote Brings Abrash Wisdom, ReactVR and Mind Control
Facebook’s F8 conference brought even more goodies to VR developers everywhere. React VR, a web-based VR framework from Oculus, got an official release. It was first announced at Oculus Connect 3 back in October.
The keynote video is available here. Michael Abrash gave another of his patented mind-blowing speeches about the future of VR, but in a twist he decided to focus specifically on AR and MR. He comes on 48 minutes into the presentation. He says we’re still 5-10 years away from living in “always-on, go-everywhere mixed reality”, what he calls its “Macintosh moment”. Abrash extends the analogy to the Mac by predicting that AR will be the biggest paradigm shift in computing since the mouse and GUI.
The wildest presentation was on Facebook’s skunk works effort to develop mind-control technologies. Regina Dugan took the stage immediately following Abrash and revealed Facebook’s ambition to enable users to do actions like typing using only their mind, something that would greatly improve the experience of working in VR, where keyboards are invisible.
More Highly-Anticipated Experiences Reach the Oculus Store and Steam
Two highly anticipated releases hit the metaverse this week. Oculus Studios and Twisted Pixel released Wilson’s Heart, a noir-horror puzzle game. Rick and Morty: Virtual Rick-ality comes from Job Simulator developers Owlchemy Labs and co-creator Justin Roiland. Batman: Arkham VR, previously a PSVR exclusive, came to Oculus and Steam as well.
A recent video on Mashable features AppliedVR’s virtual reality experiences designed to lower patients’ pain and anxiety levels. Studio Transcendent, publishers of VR Digest, created Guided Relaxation for AppliedVR. This VR experience is referenced several times in the video, which includes moving patient testimonials and discusses VR’s ability to lower the need for opiates in pain management.