Summary
This project aimed at developing and implementing a machine learning algorithm capable of estimating anatomically correct body segment rotations based on minimal marker-sets, namely, stick-figures acquired from motion capture systems such as the Kinect sensor, Qualisys or Vicon. More specifically, the ill-posed problem at hands consisted of estimating the 6th degree of freedom (rolling angle) from only 2 non-coincident points that belong to a rigid body (body segment).
The main outcome of STREACKER was the positive results obtained from the machine learning algorithm: through a supervised learning approach, the team was able to estimate the longitudinal rotation angles that revealed to be, in general, anatomically correct with a maximum error that is near 10 degrees.
The developed machine learning algorithm is expected to augment current motion capture systems that are used not only in Gait Labs, but also in exergames and rehabilitation settings. In particular, motion capture systems that are ‘markerless’ (VicoVR, Kinect) or perform full-body tracking with a single RGB-camera (PoseNet), which do not require placing optical markers upon the user, will benefit to a great extent from STREACKER’s algorithm as they resort on a minimal set stick-figure model that is similar to the one the team has considered.
The potential economic value of the technology resides mostly in the video game and fitness industries as well as rehabilitation and motion capture laboratory niches.
| Title | Skeletal Tracking Enhanced with Anatomically Correct Kinematics for Exergames and Rehabilitation |
| Reference | UTAP-EXPL/CA/0065/2017 |
| Scientific Area | Advanced Computing |
| Funding | € 93 576,00 plus matched funding at UT Austin |
| Leading Institution | Instituto de Engenharia de Sistemas e Computadores, Investigação e Desenvolvimento em Lisboa (INESC ID/INESC/IST/ULisboa) |
| Participating Institutions | The University of Austin at Texas (UT Austin) Hospital Professor Doutor Fernando Fonseca, EPE (HFF, EPE) Instituto de Engenharia Mecânica (IDMEC) |
| Duration | 18 months |
| Begin date | October 1, 2018 |
| End date | March 31, 2020 |
| Keywords | Skeletal Tracking, Kinect Sensor, Vector Orthogonalization, Machine Learning |
| Website | streacker.inesc-id.pt/home/ |
Main Highlights
- A motion database;
- A machine-learning algorithm to estimate anatomically correct longitudinal rotations;
- Marker set protocol, experimental acquisition model, and biomechanical model.
Indicators
- 10 Published papers in peer-reviewed Journals and Conferences;
- 1 MSc. student funded;
- 1 Mobility exchange supported;
- 1 Scientific event organized;
- 1 Computational application developed.
Papers and Communications
- Duarte, Stéphane, Compreender o Processo de Adoção de Tecnologias Interativas em Centros de Reabilitação, Department of Computer Science and Engineering, Instituto Superior Técnico, Universidade de Lisboa, 2019 (Masters’ Dissertation)
- Paulo, S. F., Relvas, F., Nicolau, H., Rekik, Y., Machado, V., Botelho, J., Mendes, J. J., Grisoni, L., Jorge, J., & Lopes, D. S. (2019). Touchless interaction with medical images based on 3D hand cursors supported by single-foot input: A case study in dentistry. Journal of Biomedical Informatics, 100, 103316. https://doi.org/10.1016/j.jbi.2019.103316
- Lopes, D. S., Pires, S. M., Barata, C. D., Mascarenhas, V. V., & Jorge, J. A. (2020). The hip joint as an egg shape: a comprehensive study of femoral and acetabular morphologies. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 8(4), 411–425. https://doi.org/10.1080/21681163.2019.1709902
- Alves, T., Carvalho, H., & Simões Lopes, D. (2020). Winning compensations: Adaptable gaming approach for upper limb rehabilitation sessions based on compensatory movements. Journal of Biomedical Informatics, 108, 103501. https://doi.org/10.1016/j.jbi.2020.103501
- Lopes, D. S., Faria, A., Barriga, A., Caneira, S., Baptista, F., Matos, C., Neves, A. F., Prates, L., Pereira, Â. M., & Nicolau, H. (2019). Visual Biofeedback for Upper Limb Compensatory Movements: A Preliminary Study Next to Rehabilitation Professionals. EuroVis 2019 – Posters. https://doi.org/10.2312/EURP.20191139
- Lopes, D., Relvas, F., Paulo, S., Rekik, Y., Grisoni, L., & Jorge, J. (2019, November 14). FEETICHE: FEET Input for Contactless Hand gEsture Interaction. The 17th International Conference on Virtual-Reality Continuum and Its Applications in Industry. VRCAI ’19: The 17th International Conference on Virtual-Reality Continuum and its Applications in Industry. https://doi.org/10.1145/3359997.3365704
- Faria, A., Duarte, S., Lopes, D. S., & Nicolau, H. (2021). Proximity-aware interactive displays for rehabilitation centres. In Annals of Medicine (Vol. 53, Issue sup1, pp. S84–S85). Informa UK Limited. https://doi.org/10.1080/07853890.2021.1897439
- Gordo, A., Santos Silva, I. dos, Nicolau, H., & Lopes, D. S. (2021). On the potential of virtual reality for locomotion rehabilitation. In Annals of Medicine (Vol. 53, Issue sup1, pp. S147–S148). Informa UK Limited. https://doi.org/10.1080/07853890.2021.1896637
- Matias, N. V., Roupa, I., Gonçalves, S., da Silva, M. T., & Lopes, D. S. (2021). Estimating anatomically plausible segment orientations using a kinect one sensor. In Annals of Medicine (Vol. 53, Issue sup1, pp. S143–S144). Informa UK Limited. https://doi.org/10.1080/07853890.2021.1896569
- Ivo F. Roupa, Sérgio B. Gonçalves, Miguel Tavares da Silva, Inverse Kinematic Analysis of Human Movement using Fully Cartesian Coordinates with Mixed Coordinates, Virtual Physiological Human Conference (VPH 2020), 26-28 August 2020, Paris, France
- Roupa, I., Paulo, S. F., Gonçalves, S. B., Tavares da Silva, M., & Lopes, D. S. (2020). Estimation of lower limb segment orientation using Motion Envelopes: A preliminary study. In Gait & Posture (Vol. 81, pp. 301–302). Elsevier BV. https://doi.org/10.1016/j.gaitpost.2020.08.046
E-Poster
Project Team
Daniel Lopes
STREACKER
Richard Neptune
STREACKER
