Current markerless motion capture systems are capable of acquiring faithful biomechanical abstractions of the skeletal system, also known as stick figures. Despite their applicability, conventional markerless motion capture systems perform improper limb orientation tracking.
Marker-based motion capture systems, such as Vicon and Optitrack, perform an improved limb orientation tracking with the consequence of being less cost effective and more intrusive when compared to markerless approaches. To advance human-computer interaction in physical activity settings, it is necessary to have a richer body representation that adds segment orientation information to skeletal tracking. This project proposal seeks to accurately track body segment orientations, for a given arbitrary posture, using machine learning to select the optimal vector orthogonalization technique that best correlates with the posture. By the end of the project, we expect to develop a markerless, cost effective, and reliable motion capture platform composed of an array of Kinect sensors that runs the algorithm in order to build plausible skeletons with anatomically correct body segments.
The potential economic value of the technology resides mostly in the video game and fitness industries as well as rehabilitation and motion capture laboratory settings. The major goal of this project is to develop a new algorithm for skeletal tracking with anatomically correct segment orientation based on the combination of vector orthogonalization and advanced machine learning techniques. We aim to augment current markerless motion capture used in exergames and rehabilitation. To validate the algorithm, two case scenarios will be considered: (1) rehabilitation for individuals post stroke with impaired upper limb movement; and (2) exergaming with punching, jumping jacks, and squatting exercises.
Interested users such as physiotherapists, personal trainers, students and even patients, will be in the same room and interacting with the content. To this end, depth camera sensors will be used to capture the user’s posture.
|Title||Skeletal Tracking Enhanced with Anatomically Correct Kinematics for Exergames and Rehabilitation|
|Scientific Area||Advanced Computing|
|Leading Institution||Instituto de Engenharia de Sistemas e Computadores, Investigação e Desenvolvimento em Lisboa (INESC ID/INESC/IST/ULisboa)|
|Participating Institutions||The University of Austin at Texas (UT Austin)
Hospital Professor Doutor Fernando Fonseca, EPE (HFF, EPE)
Instituto de Engenharia Mecânica (IDMEC)
|Begin date||1st October, 2018|
|End date||31st March, 2020|
|Key Words||Skeletal Tracking, Kinect Sensor, Vector Orthogonalization, Machine Learning|
"The major challenges we will face consist of (i) formulating a robust algorithm that should support arbitrary body postures; and (ii) most importantly, validating the machine learning model. Ultimately, we want to estimate anatomically correct body segment orientations in real-time, for each and every segment of the stick figure representing an arbitrary posture. This underpins the main challenge of skeletal tracking."
Daniel LopesPrincipal Investigator in Portugal
"We expect to advanced markerless motion capture by incorporating our machine learning approach to track body segment orientation, in particular, limb segments that are more challenging to capture. By adding full segment orientation to existing skeletal tracking representations, we expect to develop a markerless, cost effective, and reliable motion capture platform composed by an array of Kinect sensors that runs our algorithm in order to build plausible skeletons with anatomically correct body segments. The potential economic value of the technology resides mostly in the video game and fitness industries as well as rehabilitation and motion capture laboratory niches."