Simulating physical sensations is a growing frontier in VR tech
This focus is arguably long overdue: current VR controllers can only rumble, making them no more advanced than a PlayStation gamepad from two decades ago.
But to judge from some of the prototypes on display at the recent Consumer Electronics Show in Las Vegas – fount of a million breathless press releases – these devices seem very unlikely to see the inside of a living room soon.
Among the line-up: a pneumatics-powered monstrous black gauntlet, created by a company called HaptX, which must be tethered to a box the size of a projector, and a glove called the Maestro that pulls back your fingers using motorised “exotendons” and that is a mess of wires attached to a bulky wrist-mounted control box.
To be fair to these firms, they are for the moment marketing their wares to other companies and theme parks, not consumers (several have released SDKs to developers).
They face, however, a deeper challenge: nobody quite knows what haptic technology to back because “touch” itself comprises so many different sensations. In VR, is it more important to be able to feel hot and cold, detailed textures, or the weight of a weapon in your hand?
Even relatively simple improvements over the current generation of VR controllers seem slow getting off the ground. Valve and HTC’s SteamVR Knuckles, new controllers for its Vive headset that started shipping to developers last June, detect whether you have your individual fingers clenched tight or splayed out, opening the door to all manner VR hand gestures.
But it doesn’t offer anything more advanced than vibration. And there was no sign of Knuckles at CES. HTC said that its new, higher-resolution Vive Pro will stick to using existing vibrating wands.
One step up from this are controllers from Tactical Haptics. This company’s grips are covered in small plates that move up and down under the hand, stretching out your skin to, for example, mimic the recoil of a pistol, for example.
This “shear” force, tangential to the hand’s surface, is similar to “if you’ve ever slid your hand down a stair railing”, according to chief exec William Provancher. The controllers rely on the fact that we are more sensitive to these “shear” forces than when something presses down on them directly, he explains (PDF – there is some research, albeit with a tiny sample size, that backs this up).
Tactical Haptics’ latest idea, unveiled earlier in January, is controllers that magnetically slot together in different ways to simulate a rifle, steering wheel or shovel in VR, for example.
But the company is focused on clients like businesses and arcades (the public can try them out in a Los Angeles IMAX cinema) rather than consumers, he admits. A big barrier to mass adoption is getting developers to provide in-game physical information to drive haptic controllers’ feedback, Provancher says. Without their cooperation, it is “nearly impossible” to create content that works with the controllers.
More advanced still, but probably even further from commercial release, are exoskeleton gloves. HaptX’s glove (below) uses tiny air-filled “pixels” to create sensations across your palm and fingertips, and can push back against individual fingers to create the illusion that the wearer is holding something solid.
Youtube Video
The company has also experimented with temperature by pumping hot and cold water into thermal “pixels” that spread across the hand. One interesting psychological illusion, according to company’s founder and chief executive Jake Rubin, is that when you put two points of hot and cold close enough together on the body, “your brain essentially gets confused, and interpret that as pain or pressure.”
Using this “chequerboard” of hot and cold pixels, “it’s surprisingly easy to trigger the illusion of ‘oh my god, my hand is actually on fire’,” Rubin says. This safe pain sensation could be used in military training, or by particularly hardcore gamers, he thinks.
But although thermal haptics are “very cool” and have a real “gee-whiz” factor, according to Rubin, tactile touch is still much important for creating a sense of realism. The thermal element appears to be absent in the glove now being demo’d to journalists and companies.
A “lighter, smaller, and more ergonomic” version with a full development kit will ship to interested companies next year, Rubin says. This kit will include Unity and Unreal plug-ins, and will “instantly add realistic touch to virtually any 3D model, using the model’s geometry, texture, and physics engine data to calculate appropriate tactile and force feedback”, he says. The Maestro glove already has an SDK available, and supports the same engines.
Yet another take on haptics is being provided by researchers at Stanford University, who have managed to create objects that can change shape in your hand using a technique called “haptic jamming”.
Youtube Video
Different cells in a lattice can either be inflated with air or “jammed” stiff by vacuuming the air out of them – just like a packet of ground coffee, according to PhD student Margaret Koehler working on the project. The idea is that this creates a customisable surface that could mimic textures in VR.
The team, having created 2D lattices of cells, are now working on what Koehler calls a “fully 3D display where you have something more like a ball that you could grasp”. It starts out as a sphere, but cells can jam to change it into different shapes, she says.
However, they require a “big umbilical cord” to supply them with air, meaning the system is much more likely to be table-mounted, perhaps to simulate a human torso during surgery training.
Doing away with squishy spheres or sweaty gloves altogether are “mid-air haptics”: the firing of sound, lasers or air puffs to create touchable textures in space. Ultrahaptics, founded in Bristol in 2013, says it can use ultrasound to create virtual knobs and dials for cars, stoves or mixing decks. But the technology could also be used in VR to simulate rain splashing against your skin, or the blast of air from a passing train, according to Sriram Subramanian, co-founder and a professor of informatics at the University of Sussex.
There are problems, though: one test of the technology reported a “distracting buzz when the ultrasound waves reverberate off the skin”.
One Japanese lab has experimented with mid-air haptics using lasers, Subramanian notes, creating the sensation of sharp points that could mimic the tip of a knife or needle.
Development kits for Ultrahaptics are now available, with particular support for the Unity engine. Subramanian sees the technology being incorporated one day into VR setups “when and if VR takes off” commercially, rather than being directly sold to consumers.
All of the haptic technologies described so far have one thing in common: they can’t stop you walking through a wall in VR (except with their unwieldy cables). The traditional sci-fi solution to this problem is a robotic exoskeleton that envelopes the entire body. But there are a couple of less clunky ways to simulate weight and solidity emerging from university labs.
The first involves gyroscopes (below), which, when spinning, become an effort to rotate against their axis. This property has been put to use by a German team, which strapped a flywheel into a VR headset, making it more difficult for users to turn their head the faster it spins – simulating the inertia of injury or low gravity, for example.
Youtube Video
One of those behind the project Jan Gugenheimer, a researcher at Ulm University, admits that a big downside is the weight this adds to the headset, meaning the prototype is much too heavy and clunky as is for consumers. But he thinks if the motors were made smaller, gyroscopes could add a sense of weight and resistance to VR headsets and controllers.
There is one other unlikely sounding way of moving our legs and arms in VR: electric shocks. One lab near Berlin has rigged up a system that sends currents through your shoulders, wrists, biceps and triceps using 16 electrodes gelled on the skin. This contracts your muscles, meaning that your arms can be shocked backwards when they hit a barrier in VR. It sounds bizarre, dangerous even, but the underlying technique – electric muscle stimulation – is widely used in the rehabilitation of patients who have lost muscle power.
Youtube Video
There are drawbacks. Pedro Lopes, a PhD student at the Hasso Plattner Institute in Potsdam working on the project, reckons you also feel what he describes as a “tingling sensation on your skin” during use. That would be something that has to be ironed out if the tech is to catch on, he thinks. We think he’s right. Also: users have to stick electrodes to their skin using a wet gel. For this technology to actually take off, Lopes reckons, the electrodes would need to be incorporated into some kind of dry suit you can easily wear.
For all the advances in VR, when it comes to the full virtual monty, to touch and physical sensation, it’s difficult to know which haptic tech will break out. VR early adopters, it seems, will be stuck with rumbling grip pads for some time to come. ®
