Enhancing Stroke Rehabilitation at Home: Evaluating MediaPipe for Human Pose Estimation
Article Sidebar
Main Article Content
Abstract
This research experiment evaluates the accuracy of MediaPipe, a computer vision technology, in estimating human pose compared to motion capture technology. The experiment utilizes poses derived from the Fugl-Meyer Assessment for upper extremity (FMA-UE). Webcams are installed at three positions: front 90º, left 40º, and right 140º of the participant, while participants wear motion capture suits. The experiment begins with simultaneous movements performed on both sides of the body, with data recorded from both systems simultaneously. We calculate angles of body joints and compare the accuracy percentage. The results indicate that MediaPipe's accuracy in estimating pose based on the FMA-UE was not significantly high. Due to limitations in estimating poses in the vertical plane from z-axis and the range resulted in wider angles than reality. However, the usage of normalization for 2D angle calculation presents the better accuracy. Therefore, MediaPipe can still be used for developing interactive applications and assist the doctor evaluate patient, but further research and development are needed for evaluating patients during FMA-UE exercises.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
I/we certify that I/we have participated sufficiently in the intellectual content, conception and design of this work or the analysis and interpretation of the data (when applicable), as well as the writing of the manuscript, to take public responsibility for it and have agreed to have my/our name listed as a contributor. I/we believe the manuscript represents valid work. Neither this manuscript nor one with substantially similar content under my/our authorship has been published or is being considered for publication elsewhere, except as described in the covering letter. I/we certify that all the data collected during the study is presented in this manuscript and no data from the study has been or will be published separately. I/we attest that, if requested by the editors, I/we will provide the data/information or will cooperate fully in obtaining and providing the data/information on which the manuscript is based, for examination by the editors or their assignees. Financial interests, direct or indirect, that exist or may be perceived to exist for individual contributors in connection with the content of this paper have been disclosed in the cover letter. Sources of outside support of the project are named in the cover letter.
I/We hereby transfer(s), assign(s), or otherwise convey(s) all copyright ownership, including any and all rights incidental thereto, exclusively to the Journal, in the event that such work is published by the Journal. The Journal shall own the work, including 1) copyright; 2) the right to grant permission to republish the article in whole or in part, with or without fee; 3) the right to produce preprints or reprints and translate into languages other than English for sale or free distribution; and 4) the right to republish the work in a collection of articles in any other mechanical or electronic format.
We give the rights to the corresponding author to make necessary changes as per the request of the journal, do the rest of the correspondence on our behalf and he/she will act as the guarantor for the manuscript on our behalf.
All persons who have made substantial contributions to the work reported in the manuscript, but who are not contributors, are named in the Acknowledgment and have given me/us their written permission to be named. If I/we do not include an Acknowledgment that means I/we have not received substantial contributions from non-contributors and no contributor has been omitted.
References
สมศักดิ์ เทียมเก่า, “อุบัติการณ์โรคหลอดเลือดสมองประเทศไทย,” วารสารประสาทวิทยาแห่งประเทศไทย, vol. 37, p. 54–60, 2021.
กองโรคไม่ติดต่อหรือสำนักสื่อสารความเสี่ยงฯ กรมควบคุมโรค, “วันอัมพาตโลก 2565 เน้นสร้างความตระหนักรู้เกี่ยวกับสัญญาณเตือนโรคหลอดเลือดสมองให้กับประชาชน,” [ออนไลน์], 2565. เข้าถึงได้: https://pr.moph.go.th/?url=pr/detail/all/02/180623. [เข้าถึงเมื่อ 27 เมษายน 2566].
ตรีนุช อมรภิญโญเกียรติ, “บทบาทของการใช้หุ่นยนต์ฟื้นฟูผู้ป่วยโรคหลอดเลือดสมองในโรงพยาบาล ตากสิน,” วารสารสาธารณสุขและวิทยาศาสตร์สุขภาพ, vol. 5, p. 160–171, 2022.
ปิยธิดา ชูรักษ์, “กรอบแนวคิดการกายภาพบำบัดผู้ป่วยโรคหลอดเลือดสมองในชุมชน,” Thaksin University Online Journal, vol. 1, p. 65, 2022.
J. Adolf et al., “Automatic telerehabilitation system in a home environment using computer vision.,” in pHealth, pp. 142–148, 2020.
G. Bradski and A. Kaehler, Learning OpenCV: Computer vision with the OpenCV library. “O’Reilly Media, Inc.”, 2008.
N. Nakano et al., “Evaluation of 3D markerless motion capture accuracy using OpenPose with multiple video cameras,” Frontiers in sports and active living, vol. 2, art. 50, 2020.
M. Nour, M. Gardoni, J. Renaud and S. Gauthier, “Real-time detection and motivation of eating activity in elderly people with dementia using pose estimation with TensorFlow and OpenCV,” Adv. Soc. Sci. Res. J, vol. 8, p. 28–34, 2021.
D. M. Kishore, S. Bindu and N. K. Manjunath, “Estimation of yoga postures using machine learning techniques,” International Journal of Yoga, vol. 15, no. 2, p. 137 - 143, 2022.
C. El-Habr, X. Garcia, P. Paliyawan and R. Thawonmas, “Runner: A 2D platform game for physical health promotion,” SoftwareX, vol. 10, p. 100329, 2019.
J. W. Burke et al., “Vision based games for upper-limb stroke rehabilitation,” in 2008 International Machine Vision and Image Processing Conference, pp. 159–164, IEEE, 2008.
L. E. Sucar, R. Luis, R. Leder, J. Hernández and I. Sánchez, “Gesture therapy: A vision-based system for upper extremity stroke rehabilitation,” in 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, 2010.
C. Lugaresi et al., "Mediapipe: A framework for building perception pipelines," arXiv preprint arXiv:1906.08172, 2019.
A. Latreche et al., “Reliability and validity analysis of MediaPipe-based measurement system for some human rehabilitation motions,” Measurement, vol. 214, p. 112826, 2023.
R. Sers et al., “Validity of the Perception Neuron inertial motion capture system for upper body motion analysis,” Measurement, vol. 149, p. 107024, 2020.
P. Palani, S. Panigrahi, S. A. Jammi and A. Thondiyath, “Real-time Joint Angle Estimation using Mediapipe Framework and Inertial Sensors,” in 2022 IEEE 22nd International Conference on Bioinformatics and Bioengineering (BIBE), 2022.
A. Fern'ndez-Baena, A. Susin and X. Lligadas, “Biomechanical validation of upper-body and lower-body joint movements of kinect motion capture data for rehabilitation treatments,” in 2012 fourth international conference on intelligent networking and collaborative systems, 2012.
M. Yahya et al., “Motion capture sensing techniques used in human upper limb motion: A review,” Sensor Review, vol. 39, p. 504–511, 2019.
A. R. Fugl-Meyer et al., “A method for evaluation of physical performance,” Scand. J. Rehabil. Med, vol. 7, p. 13–31, 1975.
I. Hen et al., “The dynamics of spatial behavior: how can robust smoothing techniques help?,” Journal of neuroscience methods, vol. 133, p. 161–172, 2004.