Standardized Teaching Aid System for Piano Performance Movements Integrating Posture Estimation and Temporal Feature Extraction
Main Article Content
Abstract
Current piano performance training primarily relies on subjective observation and experience-based assessment, limiting the objective evaluation of posture accuracy and temporal coordination. This study proposes a standardized teaching aid system that integrates an improved posture estimation network with temporal feature extraction to achieve intelligent analysis and real-time feedback for piano performance movements. The system extracts upper-limb and finger keypoints through a lightweight posture estimation framework enhanced by spatial attention mechanisms and constructs spatiotemporal representations using a Temporal Transformer with embedded temporal position encoding. A quantitative evaluation network is further developed to assess spatial coordination, temporal consistency, and fingering stability, generating standardized scores and dynamic feedback signals in a closed-loop teaching framework. Experimental results demonstrate a keypoint localization stability rate of 96.0%, feedback accuracy of 97.2%, and response delays ranging from 0.48 s to 0.77 s across different performance scenarios. The proposed framework exhibits strong robustness and adaptability for complex motion analysis. It is particularly applicable to intelligent sensing environments supported by wireless communication infrastructures and antenna-enabled edge devices, where reliable real-time transmission of visual and motion data is essential for responsive feedback and human–machine interaction. This study provides an effective engineering solution for intelligent motion assessment, standardized skill training, and data-driven interactive learning systems.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
W. B. Verwey, “Chord skill: Learning optimized hand postures and bimanual coordination,” Experimental Brain Research, vol. 241, no. 6, pp. 1643-1659, 2023, doi: 10.1007/s00221-023-06629-2.
K. Ito, T. Watanabe, T. Horinouchi, T. Matsumoto, K. Yunoki, H. Ishida, et al., “Higher synchronization stability with piano experience: Relationship with finger and presentation modality,” Journal of Physiological Anthropology, vol. 42, no. 1, p. 10, 2023, doi: 10.1186/s40101-023-00327-2.
P. Yang, “Integrating intelligent algorithms in music education to analyze and improve posture and motion in instrumental training,” Molecular & Cellular Biomechanics, vol. 22, no. 1, pp. 762-762, 2025, doi: 10.62617/mcb762.
S. Zhang, P. Ni, J. Wen, Q. Han, X. Du, and J. Fu, “Intelligent identification of moving forces based on visual perception,” Mechanical Systems and Signal Processing, vol. 214, no. 1, 111372, 2024, doi: 10.1016/j.ymssp.2024.111372.
C. F. P. Monfredini, D. B. Coelho, A. J. Marcori, and L. A. Teixeira, “Control of interjoint coordination in the performance of manual circular movements can explain lateral specialization,” Human Movement Science, vol. 90, no. 1, 103102, 2023, doi: 10.1016/j.humov.2023.103102.
J. Yuk, N. M. Kitchen, A. Przybyla, R. A. Scheidt, and R. L. Sainburg, “Symmetry and synchrony of bimanual movements are not predicated on interlimb control coupling,” Journal of Neurophysiology, vol. 131, no. 6, pp. 982-996, 2024, doi: 10.1152/jn.00476.2023.
X. Bai, X. Wei, Z. Wang, and M. Zhang, “CONet: Crowd and occlusion-aware network for occluded human pose estimation,” Neural Networks, vol. 172, no. 1, 106109, 2024, doi: 10.1016/j.neunet.2024.106109.
X. Jiang, H. Tao, J. N. Hwang, and Z. Fang, “A multiscale coarse-to-fine human pose estimation network with hard keypoint mining,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 54, no. 3, pp. 1730-1741, 2023, doi: 10.1109/TSMC.2023.3328876.
Q. Cui, H. Sun, Y. Kong, X. Zhang, and Y. Li, “Efficient human motion prediction using temporal convolutional generative adversarial network,” Information Sciences, vol. 545, no. 1, pp. 427-447, 2021, doi: 10.1016/j.ins.2020.08.123.
M. Wu and P. Shi, “Human pose estimation based on a spatial temporal graph convolutional network,” Applied Sciences, vol. 13, no. 5, p. 3286, 2023, doi: 10.3390/app13053286.
D. Mao and S. Liu, “Quantitative Evaluation of Piano Performance Technique and Style Based on Continuous Discretization Method,” Applied Mathematics and Nonlinear Sciences, vol. 9, no. 1, p. 1, 2024, doi: 10.2478/amns.2023.2.00093.
H. Qi and C. She, “Research on Improving Piano Performance Evaluation Method in Piano Assisted Online Education,” International Journal of Advanced Computer Science and Applications, vol. 14, no. 8, pp. 1-12, 2023, doi: 10.14569/IJACSA.2023.0140850.
X. Li, Y. Guo, W. Pan, H. Liu, and B. Xu, “Human pose estimation based on lightweight multi-scale coordinate attention,” Applied Sciences, vol. 13, no. 6, p. 3614, 2023, doi: 10.3390/app13063614.
D. Rong and F. Gang, “Coordinate-Corrected and Graph-Convolution-Based Hand Pose Estimation Method,” Sensors, vol. 24, no. 22, p. 7289, 2024, doi: 10.3390/s24227289.
C. Yang, F. Mei, T. Zang, J. Tu, N. Jiang, and L. Liu, “Human Action Recognition Using Key-Frame Attention-Based LSTM Networks,” Electronics, vol. 12, no. 12, p. 2622, 2023, doi: 10.3390/electronics12122622.
J. Tang, J. Wang, and J. F. Hu, “Predicting human poses via recurrent attention network,” Visual Intelligence, vol. 1, no. 1, p. 18, 2023, doi: 10.1007/s44267-023-00020-z.
H. Wang, Q. Shi, and B. Shan, “Three-dimensional human pose estimation with spatial–temporal interaction enhancement transformer,” Applied Sciences, vol. 13, no. 8, p. 5093, 2023, doi: 10.3390/app13085093.
Z. Chen, W. Huang, H. Liu, Z. Wang, Y. Wen, and S. Wang, “ST-TGR: Spatio-temporal representation learning for skeleton-based teaching gesture recognition,” Sensors, vol. 24, no. 8, p. 2589, 2024, doi: 10.3390/s24082589.
K. Wilson and P. E. Pfeiffer, “Feedback in augmented and virtual reality piano tutoring systems: a mini review,” Frontiers in Virtual Reality, vol. 4, no. 1, 1207397, 2023, doi: 10.3389/frvir.2023.1207397.
W. Luo and B. Ning, “Toward piano teaching evaluation based on neural network,” Scientific Programming, vol. 2022, no. 1, 6328768, 2022, doi: 10.1155/2022/6328768.
Y. Wang, H. Kang, D. Wu, W. Yang, and L. Zhang, “Global and local spatio-temporal encoder for 3D human pose estimation,” IEEE Transactions on Multimedia, vol. 26, no. 1, pp. 4039-4049, 2023, doi: 10.1109/TMM.2023.3321438.
X. Liu and H. Tang, “STRFormer: Spatial–temporal–retemporal transformer for 3D human pose estimation,” Image and Vision Computing, vol. 140, no. 1, 104863, 2023, doi: 10.1016/j.imavis.2023.104863.
S. Yang, D. He, Q. Li, J. Wang, and D. Li, “Hand pose estimation based on improved NSRM network,” EURASIP Journal on Advances in Signal Processing, vol. 2023, no. 1, p. 8, 2023, doi: 10.1186/s13634-023-00970-y.
X. Zou and X. Bi, “LCFFNet: A Lightweight Cross-scale Feature Fusion Network for human pose estimation,” Neural Networks, vol. 183, no. 1, 106959, 2025, doi: 10.1016/j.neunet.2024.106959.
M. Rizwan, S. Ul Haq, N. Gul, M. Asif, S. M. Shah, T. Jan, et al., “Appearance Based Dynamic Hand Gesture Recognition Using 3D Separable Convolutional Neural Network,” Computers, Materials & Continua, vol. 76, no. 1, pp. 1-35, 2023, doi: 10.32604/cmc.2023.038211.
R. Jain, R. K. Karsh, and A. A. Barbhuiya, “Encoded motion image-based dynamic hand gesture recognition,” The Visual Computer, vol. 38, no. 6, pp. 1957-1974, 2022, doi: 10.1007/s00371-021-02259-3.
D. Cao, W. Liu, W. Xing, and X. Wei, “Human pose estimation based on feature enhancement and multi-scale feature fusion,” Signal, Image and Video Processing, vol. 17, no. 3, pp. 643-650, 2023, doi: 10.1007/s11760-022-02271-7.
Y. Ma, Q. Shi, and F. Zhang, “A lightweight Context-aware Feature Transformer Network for human pose estimation,” Electronics, vol. 13, no. 4, p. 716, 2024, doi: 10.3390/electronics13040716.
B. Huang and X. Li, “Human Motion Prediction via Dual-Attention and Multi-Granularity Temporal Convolutional Networks,” Sensors, vol. 23, no. 12, p. 5653, 2023, doi: 10.3390/s23125653.
J. Ma, Y. Zhang, H. Zhou, H. Yang, and X. Wu, “Multi-granularity spatial temporal graph convolution network with consecutive attention for human motion prediction,” Applied Soft Computing, vol. 165, no. 1, 112126, 2024, doi: 10.1016/j.asoc.2024.112126.
K. Y. Lai, C. H. Hsu, Y. C. Lin, C. H. Tsai, C. F. Lin, L. C. Kuo, et al., “Practically Feasible Sensor-Embedded Kinetic Assessment Piano System for Quantifying Striking Force of Digits During Piano Playing,” Journal of Medical and Biological Engineering, vol. 43, no. 6, pp. 749-757, 2023, doi: 10.1007/s40846-023-00835-7.
J. Lin, B. Ding, Z. Song, Z. Li, and S. Li, “A Model of Multi-Finger Coordination in Keystroke Movement,” Sensors, vol. 24, no. 4, p. 1221, 2024, doi: 10.3390/s24041221.
Y. Liu, T. Zhang, Z. Li, and L. Deng, “Deep learning-based standardized evaluation and human pose estimation: A novel approach to motion perception,” Traitement du Signal, vol. 40, no. 5, pp. 2313-2320, 2023, doi: 10.18280/ts.400549.
L. Amadi and G. Agam, “PosturePose: Optimized Posture Analysis for Semi-Supervised Monocular 3D Human Pose Estimation,” Sensors, vol. 23, no. 24, p. 9749, 2023, doi: 10.3390/s23249749.
F. Roggio, B. Trovato, M. Sortino, and G. Musumeci, “A comprehensive analysis of the machine learning pose estimation models used in human movement and posture analyses: A narrative review,” Heliyon, vol. 10, no. 21, e39977, 2024, doi: 10.1016/j.heliyon.2024.e39977.