Microsoft have published details on a force feedback 3D touchscreen project that allows you to feel and manipulate images.
Researchers have combined an LCD flat panel screen, force sensors and a robotic arm that moves forwards and backwards. Microsoft say that by controlling the amount of resistance in a user’s fingertip, then they can simulate the shape and weight of things shown on the screen.
The screen can recreate the feel of an object’s shape by being moved backwards and forwards. When the prototype is touched, it responds by pushing back with a light force so that your fingers stay in contact with the screen. Then, by pressing against it, the robotic arm is programmed to instantly pull the screen backwards in a matching smooth movement.
To create a 3D effect, size and perspective is automatically adjusted.
“As your finger pushes on the touchscreen and the senses merge with stereo vision, if we do the convergence correctly and update the visuals constantly so that they correspond to your finger’s depth perception, this is enough for your brain to accept the virtual world as real,” said senior researcher Michael Pahud.
Using an MRI scan, they demonstrated how a medic could navigate through the various sections of the scan by pushing a finger against the display. It was also possible to draw notes and leave a “haptic detent” (or force-feedback marker) so that it was possible to find specfic layers again.
“I could imagine receiving haptic feedback when you encountered an anomaly, such as a tumour, because we can change the response based on what you touch,” said Mr Pahud.
“You can have different responses for when you touch soft tissue versus hard tissue, which makes for a very rich experience.”
“If you were moving your finger over a surface that was rough the screen would have to go up and down very quickly to be able to give you that impression.”
“I think their example of the brain scan is a bit artificial, but where I could see it being useful would be for a doctor doing teleconsultancy work,” he said.
So while this 3D touchscreen needs some more work, it looks like we will see more of this type of technology in the future.
Researchers have combined an LCD flat panel screen, force sensors and a robotic arm that moves forwards and backwards. Microsoft say that by controlling the amount of resistance in a user’s fingertip, then they can simulate the shape and weight of things shown on the screen.
The screen can recreate the feel of an object’s shape by being moved backwards and forwards. When the prototype is touched, it responds by pushing back with a light force so that your fingers stay in contact with the screen. Then, by pressing against it, the robotic arm is programmed to instantly pull the screen backwards in a matching smooth movement.
To create a 3D effect, size and perspective is automatically adjusted.
“As your finger pushes on the touchscreen and the senses merge with stereo vision, if we do the convergence correctly and update the visuals constantly so that they correspond to your finger’s depth perception, this is enough for your brain to accept the virtual world as real,” said senior researcher Michael Pahud.
Medical Uses
Microsoft have said that the device could be used for medical purposes as well as for gaming. They think it could be a useful tool in aiding doctors explore body scans.Using an MRI scan, they demonstrated how a medic could navigate through the various sections of the scan by pushing a finger against the display. It was also possible to draw notes and leave a “haptic detent” (or force-feedback marker) so that it was possible to find specfic layers again.
“I could imagine receiving haptic feedback when you encountered an anomaly, such as a tumour, because we can change the response based on what you touch,” said Mr Pahud.
“You can have different responses for when you touch soft tissue versus hard tissue, which makes for a very rich experience.”
A Bit Artificial
Although Microsoft are making considerable progress with this device, some feel it is still not responsive enough to make it accurate. Dr Paul Weller, who is head of the Centre for Health Informatics at City University,London, said the screen would not be accurate enough for detecting texture.“If you were moving your finger over a surface that was rough the screen would have to go up and down very quickly to be able to give you that impression.”
“I think their example of the brain scan is a bit artificial, but where I could see it being useful would be for a doctor doing teleconsultancy work,” he said.
So while this 3D touchscreen needs some more work, it looks like we will see more of this type of technology in the future.
