Publication

Thuong Hoang, Martin Reinoso, Frank Vetere, Egemen Tanin, “Onebody: Remote Posture Guidance System using First Person View in Virtual Environment”, in Proceedings of Nordic Conference on Human-Computer Interaction,, Gothenburg, Sweden 2016.
[pdf]

Description

This project explores NUI technology for the knowledge transfer of motor skills, in such areas as martial arts, dance and yoga. Traditionally, the teacher demonstrates new movements/poses and skills for students to replicate. The teacher then provides feedback to help students perform the correct postures and movements. The consolidation process of such knowledge requires the student to internalise the movements from the perspective of an observer to a performer of movements.

We have developed ‘Onebody’ a virtual reality training system that helps people understand the correct body movements and posture required for learning sport or other physical activities. Onebody allows a student to step into their instructor’s body. Students wishing to learn activities such as martial arts, will benefit from this system as it provides a first person perspective of their instructor’s movement. The system renders the instructor and the student using avatars via skeletal tracking. Using a virtual reality headset the student can visualise the movement of the instructor’s avatar rendered in place of their own body.

We explored immersive virtual reality and body tracking technology to provide a virtual training environment for motor skills. Body tracking technology, such as the Microsoft Kinect, is used to virtualise the movements of both the teacher and the students, as virtual avatars. The ability to coexist in a virtual world supports remote learning.

The outcomes of this project provide a better understanding on how motor skills are acquired and can be expanded to different areas. Example areas include sports, physical education, physical training, remote training and motor skill rehabilitation.