Apple Acquires SMI
"Apple buys smaller technology companies from time to time, and we generally do not discuss our purpose or plans." So goes the oft-repeated boilerplate response out of Cupertino, CA as the tech giant assimilates smaller technology firms. Their latest acquisition has major implications for our industry however.
Apple has purchased SensoMotoric Instruments, better known as SMI, which has been working on ultrafast eye-tracking technology. Macrumors broke the news with a detailed breakdown of the evidence (Apple tries to shield its acquisitions from public view by using shell companies).
High performance eye tracking is crucial to making extremely high resolution head mounted displays a reality. Your eyes paint a 200-degree field of view picture of the world by flicking about rapidly in saccades. At any given moment they can only make out a high resolution image in a 2-degree field of vision called the fovea centralis. Therefore, in theory a sufficiently sophisticated system would allow the computer to render only a tiny percentage of the scene on each frame. This is known as foveated rendering.
SMI announced back in March that it was working with Valve to integrate eye tracking into Valve’s OpenVR API. In January 2016 it demonstrated a working foveated rendering demo with sensors built into a modified DK2. It has also been providing hardware to other companies interested in using eye tracking in Gear VR or DK2 devices for special purposes. One such purpose was covered by The Voices of VR podcast this week. SyncThink is an application that uses a Gear VR to automatically diagnose a concussion within 60 seconds.
Those initiatives will likely be eliminated as Apple moves to integrate SMI’s technology into their own VR/AR initiative. It is the strongest evidence yet that Apple is working on a head-mounted display and not just “windowed AR” like that which is launching on ARKit-compatible devices this fall.
SteamVR Knuckles Now in the Hands of Developers
Valve has started providing developers with prototypes of its new “knuckles” controller design, which is strapped to the hand so that it does not need to be gripped. It also features capacitive sensors for hand presence, similar to Oculus Touch, but with even more sensors so that all five fingers are tracked. Here is a video capture of the accompanying tech demo.
Cloudhead Games is one of the recipients and posted a screenshot of how the controllers appear in SteamVR and a video of the knuckles working in Climbey.