Microsoft Opens Volumetric Capture Studios
Microsoft first demonstrated their amazing volumetric capture technology nearly two and a half years ago. Now it will be available for content creators to use as Microsoft is opening two “Mixed Reality Capture” studios in San Francisco and London.
The company has been experimenting with their capture technology since 2010. It differs from competitors like 8i in the presentation step; most solutions use 3D point clouds to represent the capture, which makes playback a challenge. Microsoft’s solution also generates a 3D point cloud, but that’s not where it ends up; their added secret sauce is a system that can determine surface normals from the capture, and using that data, construct a mesh that represents each frame.
Then a second, almost even more delicious secret sauce is added that creates a spatiotemporally coherent mesh—in other words, a single mesh that has its vertices animated across multiple frames. This coherent mesh is replaced every so often when the shape gets too distorted, which is similar to the concept behind MPEG video compression; the differences between each frame are stored, rather than the entire frame. This allows for compression of the mesh data so that it can fit in the Network Abstraction Layer datastream of an MPEG video file. The textures for the mesh form the video stream. That means the volumetric data can be delivered to end users at high framerates, streaming over the internet at normal bitrates.
If you want to see what this looks like in person, you’re in luck, because one of the first VR experiences to use the technology, Blade Runner 2049: Memory Lab VR, is being made available for download starting today on the Oculus Store.
The Kinect Is Dead, Long Live The Kinect
Microsoft is discontinuing the manufacture of the Kinect, much to the chagrin of hackers who enjoyed using it for computer-vision related experiments. The progeny of the Kinect will live on in the form of millions upon millions of iPhone X’s—available for preorder on Friday—which incorporate a miniaturized version of the Primesense sensor array that powered the first-generation Kinect. Unfortunately, it’s unlikely that the iPhone’s sensor can fill the hole in the hearts of the artists and researchers that were building applications with Kinect; but we can hope that developers will find interesting uses for a tiny one in millions of pockets (beyond the animated poop-moji that Apple developed).
Canted-Display VR Headsets on the Move
Pimax’s 5K/8K headset Kickstarter broke $2.3 million in funding, sailing into the stretch goal phase. They also posted several videos on the Kickstarter page showing a “through the lens” view and demonstrating its tracking capability. The most impressive of these is the Project Cars demo, as you can easily read the gauges and thin text.
Meanwhile, the other high resolution canted-display headset, Star VR, posted some news. Acer has invested an additional $5 million into the project in exchange for two-thirds control. They have also let developer Starbreeze off the hook for an additional $7.5 million that they had pledged to the project. So far about $10 million has been spent.
Both endeavors will benefit from efforts to support proper rendering on canted display devices, something that has so far eluded Pimax’s demos. The first bullet point in the VR release notes for Unreal Engine 4.18 is “Added support for non-parallel projections for HMDs that have canted display panels.”
More Unreal Engine 4.18 Improvements
Unreal Engine 4.18 got its official release; as we’ve mentioned previously, this version brings with it support for ARKit and ARCore as well as SteamVR on Mac—all very exciting developments. This release also brings some massive lightmapping upgrades. Volumetric lightmapping that can affect fog and multiple bounces from sky lights may not sound that interesting on paper, but in side-by-side screenshots, they look incredible.
This release also gets support for SteamVR “knuckles” controllers. Developers will be relieved that the stat displays and console can now be read properly through the headset. And Oculus Vulkan support is now available.
Look Ma, Robot Hands!
There is no doubt that telepresence is, in the long term, a tremendously important and exciting application for VR technology. Researchers at the University of Tokyo have taken a step down that road by building an impressive walking robot that can be puppeteered with Vive controllers.