Copy
VR Digest: F8 & NAB Usher in
New Cameras, Far-out Tech

The week's top VR news brought to you
pre-digested and ready for assimilation!
VR Digest: A VR newsletter brought to you by Studio Transcendent
written by John Dewar (@jstarrdewar) Issue 117 – April 26, 2017

VR Digest: A Virtual Reality newsletter brought to you by Studio Transcendent

Hi Friends,


Studio Transcendent just finished a spectacular experience we can’t wait to show to you; John is heading out for a much needed breather, so Aaron will be guest editing next week. Tons of exciting news to report from this past week: this is a monster of an issue. Enjoy!

Best,
Aaron, John, Bowdy, and Elissa 
at Studio Transcendent

Exciting New VR Cameras Appear at F8 and NAB

Last Wednesday on day two of its F8 conference, Facebook revealed two new VR cameras with an amazing new trick: using Facebook’s machine vision technologies, they can recreate the depth of the scene they are capturing as a 3D point cloud. This is similar to other pointcloud-based volumetric VR video solutions like HypeVR, but does away with the need for bulky, expensive LIDAR sensors.

The point cloud data gives viewers the ability to shift their perspective slightly, creating a far more convincing sense of depth than can stereo alone, and preserves proper stereo if users tilt their head at odd angles, a major weakness of current video-only solutions.

Facebook is partnering with OTOY to create workflows for integrating the camera’s output with Octane Render elements and compositing them in After Effects and Nuke. They are also working together on playback formats that will preserve the point cloud data with 32-bit floating point resolution (a standard video depth map can only preserve 8-bit depth, but they may also support this format for mobile playback).

The new cameras are christened the “Surround 360” and are available in two configurations, the x6 and x24, with 6 and 24 cameras respectively. Each is roughly the size of a soccer ball. The sensors are provided by FLIR (also known as Point Grey) which provided the cameras for Facebook’s previous open-sourced VR camera design.

Coming in second place in the camera race this week was Google, who is offering a revamped version of their Jump camera system. First announced a year ago in the guise of the 16-GoPro Odyssey rig, Jump offers very good automatic stitching using optical flow algorithms to interpret novel viewpoints that the sixteen cameras alone cannot capture. However, Odyssey had an Achilles’ heel of sorts: a blurry hole in the footage at the north pole (directly overhead).

The new camera system adds a 17th sensor to address this oversight. Based on cameras from YI Technology, an action cam company that competes with GoPro and is backed by Chinese handset juggernaut Xiaomi, the new Jump is lighter weight and more refined, with an external control panel and a companion app. It also adds the option to record video at 60 frames per second at 6K, while preserving the original camera’s ability to record 30 frames per second at 8K.

For its part, YI also announced a portable 2D/360 camera called the YI 360 VR, which looks to compete with the likes of the new Samsung Gear 360. The camera features a chunky industrial design reminiscent of a reporter’s cassette recorder, and is capable of live streaming video at a 2.5K resolution.

GoPro meanwhile announced a very similar offering in the form of the GoPro Fusion, which uses back-to-back sensors to produce a 5.2K spherical image. Its differentiating feature is the ability to crop out a standard 1080p frame in post, allowing you to frame action you might have missed with a standard GoPro. It should be noted that any VR camera with sufficient resolution can pull off this trick, but GoPro is betting that the extra 1.2K resolution over similar cameras like the YI and Gear 360, and better software tools for the cropping task, will make the selling point stick.

Digital Domain has been experimenting with custom made VR cameras for years and has now announced a commercial product in the form of a spherical 4K camera with a refined workflow and custom stitching tools.

There were too many VR cameras on display at NAB to list them all here. Cristian Dominguez Rein-Loring posted a photo album on Facebook showing the tableau. You’ve got to see the crazy Arri Alexa rig which looks to be about the same size as a SmartForTwo.
 

F8’s Day Two Keynote Brings Abrash Wisdom, ReactVR and Mind Control

Facebook’s F8 conference brought even more goodies to VR developers everywhere. React VR, a web-based VR framework from Oculus, got an official release. It was first announced at Oculus Connect 3 back in October.

The keynote video is available here. Michael Abrash gave another of his patented mind-blowing speeches about the future of VR, but in a twist he decided to focus specifically on AR and MR. He comes on 48 minutes into the presentation. He says we’re still 5-10 years away from living in “always-on, go-everywhere mixed reality”, what he calls its “Macintosh moment”. Abrash extends the analogy to the Mac by predicting that AR will be the biggest paradigm shift in computing since the mouse and GUI.

The wildest presentation was on Facebook’s skunk works effort to develop mind-control technologies. Regina Dugan took the stage immediately following Abrash and revealed Facebook’s ambition to enable users to do actions like typing using only their mind, something that would greatly improve the experience of working in VR, where keyboards are invisible.
 

More Highly-Anticipated Experiences Reach the Oculus Store and Steam

Two highly anticipated releases hit the metaverse this week. Oculus Studios and Twisted Pixel released Wilson’s Heart, a noir-horror puzzle game. Rick and Morty: Virtual Rick-ality comes from Job Simulator developers Owlchemy Labs and co-creator Justin Roiland. Batman: Arkham VR, previously a PSVR exclusive, came to Oculus and Steam as well.
 

Bits

A recent video on Mashable features AppliedVR’s virtual reality experiences designed to lower patients’ pain and anxiety levels. Studio Transcendent, publishers of VR Digest, created Guided Relaxation for AppliedVR. This VR experience is referenced several times in the video, which includes moving patient testimonials and discusses VR’s ability to lower the need for opiates in pain management.

There was one more cool announcement at F8: the Oculus 360 Capture SDK, which enables developers to export 360 video of their real-time rendered experience at 4K resolution while playing the game. Current export solutions for game engines are too slow to allow for real time game play, leading to arduous hours of faking the play or coming up with a replay system. This may also enable end-users to produce their own 360 video captures. The system is supposed to be compatible with Unreal and Unity, but as of now only a Unity example is provided, so it is not yet a plug and play solution.

Lytro posted a behind the scenes video of their latest lightfield-capture VR experience, a music video centered around Leonard Cohen’s Hallelujah. Ecco VR was involved in creating the binaural soundscape of the piece. They previously worked on Studio Transcendent’s Rapid Fire: a brief history of flight. Road to VR’s Ben Lang had a chance to try the experience out.

Road to VR’s Ben Lang also recorded videos showing the present state of the Meta 2’s AR demonstrations, which are quite interesting.

Google has nominated five apps in each of the newly created virtual reality and augmented reality categories in their 2017 Google Play Awards. Winners will be announced on May 18th during Google I/O.

DJI has released more details regarding their FPV goggles. An article in The Verge states that “users can both fly the drone and control the camera gimbal using just their head movement, thanks to an embedded accelerometer and gyroscope.”  This will undoubtedly be incredibly cool, but bring Ginger Ale to the field with you: in addition to the drone’s rapid motion, DJI says the system has 110ms of latency, far higher than the 20ms Oculus targets as a comfortable delay between turning your head and seeing the image update. The $449 headsets became available for presale on Monday and will start shipping in late May.

VR Digest™ is a Virtual Reality Newsletter brought to you by Studio Transcendent, a producer of premium Virtual Reality experiences. We publish weekly on Wednesdays.

 
Subscribe Now
Copyright © 2017 Studio Transcendent, All rights reserved.