Real Haptics: using physical manipulation to control virtual worlds

In this work we propose Real Haptics, a novel interaction method for Mixed Reality based on two key ideas: better Embodiment by segmenting user real hands and the manipulation of physical objects, which may preserve or change their real appearance in the immersive scenario. First, we deployed a functional prototype of Real Haptics developed with Unity, using color-based algorithms for object segmentation and Aruco library for object tracking. Then, inspired by the benefits of gamification, we built a 5 minutes Escape Room designed to validate the key features of Real Haptics: real Embodiment, real objects, augmented real objects and control devices. We carried out a through evaluation with 53 users and a fair comparison scheme: users had to play twice the Escape Room, once with Real Haptics and a second one with purely virtual objects and avatars. After each run, users had to answer a survey derived from standard questionnaires measuring Presence, Embodiment and Quality of Experience. Some quantitative data was also extracted from the game runs. Results validate our hypothesis that the use of Real Haptics significantly outperforms key factors such as Presence or Embodiment. We foresee our solution to provide a significant advance in the control of virtual environments, with a first potential focus on training applications