Identifying Student Body Expression Characteristics in Interactive Music Instruction Using HRNet
Main Article Content
Abstract
This paper addresses the recognition of body expression in interactive music instruction under instrument occlusion, rapid motion, and subtle hand-movement conditions. An improved HRNet-based recognition method is proposed to enhance fine-grained posture estimation and multimodal behavior-feedback analysis. A multimodal dataset was constructed by synchronously collecting video, touch and pressure signals, and sound-field data. In the model, standard convolutions in HRNet are replaced by involution operators to improve spatial feature representation while reducing computational complexity. A multi-scale feature fusion mechanism integrates macroscopic body movements with microscopic hand-motion information, and a Transformer-based hand module further captures detailed features such as knuckle positions and pressure points. Experiments on a self-built dataset show an overall keypoint-recognition accuracy of 92.3%, representing an 8.5% improvement over the original HRNet. Computational complexity is reduced by 44%, and inference speed increases by 69.8%. Ablation analysis verifies the effectiveness of the model in body pose estimation, fine hand-motion recognition, and cross-modal fusion, providing an engineering solution for real-time visual sensing in interactive teaching environments.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
Y. He and N. Suttachitt, “The Development of Music Teaching Strategies for Promoting Music Learning Competency for Elementary School Students Under the Core Literacy of Music Academic Subjects,” Journal of Ecohumanism, vol. 3, no. 8, pp. 3049-3061, 2024, doi: 10.62754/joe.v3i8.4950.
Y. Chen, “Review of Interactive Teaching Methods of Music in Junior Middle Schools,” Asian Journal of Education and Social Studies, vol. 50, no. 2, pp. 49-59, 2024, doi: 10.9734/ajess/2024/v50i21258.
J. Yan and X. Xia, “Interactive audio-visual course teaching of music education based on VR and AI support,” International Journal of Human-Computer Interaction, vol. 40, no. 13, pp. 3552-3559, 2024, doi: 10.1080/10447318.2023.2190668.
A. Sultanov, “Educational Effectiveness of Using Interactive Methods in Music Pedagogy,” Academic Journal of Science, Technology and Education, vol. 1, no. 1, pp. 9-12, 2025.
O. Aliksiichuk, T. Borysova, Z. Kartashova, O. Priadko, M. Kuziv, and S. Chaban-Chaika, “Modern Digital Approaches to Training Music Teachers: Evolution from Classical to Interactive,” International Journal on Culture, History, and Religion, vol. 7, no. SI1, pp. 273-296, 2025, doi: 10.63931/ijchr.v7iSI1.201.
F. G. Rexhepi, R. K. Breznica, and B. R. Rexhepi, “Evaluating the effectiveness of using digital technologies in music education,” Journal of Educational Technology Development and Exchange (JETDE), vol. 17, no. 1, pp. 273-289, 2024, doi: 10.18785/jetde.1701.16.
M. Ouyang, “Employing mobile learning in music education,” Education and Information Technologies, vol. 28, no. 5, pp. 5241-5257, 2023, doi: 10.1007/s10639-022-11353-5.
H. B. Yu, H. S. Hwang, D. H. Kim, H. J. Oh, and H. C. Kim, “Research on the Performance Optimization of HR-Net for Spinal Region Segmentation in Whole Spine X-ray Images,” Journal of Biomedical Engineering Research, vol. 45, no. 4, pp. 139-147, 2024.
K. Ito, H. Nakashima, Y. Kagami, et al., “An appropriate method for predicting the femoral angle on whole-body X-ray images from full-spine X-ray images,” Journal of Orthopaedic Science, vol. 29, no. 2, 2024, doi: 10.1016/j.jos.2023.02.007.
G. Yang, J. Lei, H. Tian, Z. Feng, and R. Liang, “Asymptotic feature pyramid network for labeling pixels and regions,” IEEE Trans actions on Circuits and Systems for Video Technology, vol. 34, no. 9, pp. 7820-7829, 2024, doi: 10.1109/TCSVT.2024.3376773.
D. Chen and S. Zhang, “Deep Learning-Based Involution Feature Extraction for Human Posture Recognition in Martial Arts,” Informatica, vol. 49, no. 12, pp. 77-90, 2025, doi: 10.31449/inf.v49i12.7041.
Q. Ren, Z. Lu, H. Wu, J. Zhang, and Z. Dong, “HR-Net: A landmark based high realistic face reenactment network,” IEEE Trans actions on Circuits and Systems for Video Technology, vol. 33, no. 11, pp. 6347-6359, 2023, doi: 10.1109/TCSVT.2023.3268062.
H. C. Nguyen, T. H. Nguyen, J. Nowak, A. Byrski, A. Siwocha, and V. H. Le, “Combined YOLOv5 and HRNet for high accuracy 2D keypoint and human pose estimation,” Journal of Artificial Intelligence and Soft Computing Research, vol. 12, no. 4, pp. 281-298, 2022, doi: 10.2478/jaiscr-2022-0019.
Y. Wu, X. Chen, F. Dong, L. He, G. Cheng, Y. Zheng, et al., “Performance evaluation of a deep learning-based cascaded HRNet model for automatic measurement of X-ray imaging parameters of lumbar sagittal curvature,” European Spine Journal, vol. 33, no. 11, pp. 4104-4118, 2024, doi: 10.1007/s00586-023-07937-5.
H. Zhang, Y. Dun, Y. Pei, S. Lai, C. Liu, K. Zhang, et al., “HF-HRNet: A simple hardware friendly high-resolution network,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 8, pp. 7699-7711, 2024, doi: 10.1109/TCSVT.2024.3377365.
P. Lisena, A. Meroño-Peñuela, and R. Troncy, “MIDI2vec: Learning MIDI embeddings for reliable prediction of symbolic music metadata,” Semantic Web, vol. 13, no. 3, pp. 357-377, 2022, doi: 10.3233/SW-210446.
S. Shen and K. Wu, “Solfeggio teaching method based on MIDI technology in the background of digital music teaching,” International Journal of Web-Based Learning and Teaching Technologies (IJWLTT), vol. 18, no. 1, pp. 1-18, 2023, doi: 10.4018/IJWLTT.331085.
E. D’Antonio, J. Taborri, I. Mileti, S. Rossi, and F. Patané, “Validation of a 3D markerless system for gait analysis based on Open Pose and two RGB webcams,” IEEE Sensors Journal, vol. 21, no. 15, pp. 17064-17075, 2021, doi: 10.1109/JSEN.2021.3081188.
Z. Cao, G. Hidalgo, T. Simon, S. E. Wei, and Y. Sheikh, “Openpose: Realtime multi-person 2d pose estimation using part affinity fields,” IEEE transactions on pattern analysis and machine intelligence, vol. 43, no. 1, pp. 172-186, 2019, doi: 10.1109/TPAMI.2019.2929257.
A. del Barrio Mártil, “Wacom Intuos Pro 2025 Tablet: A Major Redesign,” FV: Photo-Video News, vol. 38, no. 300, pp. 12-14, 2025, doi: 10.62077/qvoo62.rnf858.
T. D. Nguyen, M. H. Shih, D. Srivastava, S. Tirthapura, and B. Xu, “Stratified random sampling from streaming and stored data,” Distributed and Parallel Databases, vol. 39, no. 3, pp. 665-710, 2021, doi: 10.1007/s10619-020-07315-w.
R. S. Kenett, C. Gotwalt, L. Freeman, and X. Deng, “Self-supervised cross validation using data generation structure,” Applied Stochastic Models in Business and Industry, vol. 38, no. 5, pp. 750-765, 2022, doi: 10.1002/asmb.2701.
Y. Wang, R. Wang, H. Shi, and D. Liu, “MS-HRNet: multi-scale high-resolution network for human pose estimation,” The Journal of Supercomputing, vol. 80, no. 12, pp. 17269-17291, 2024, doi: 10.1007/s11227-024-06125-6.
M. Liu, K. Lin, W. Huo, L. Hu, and Z. He, “Feature enhancement modules applied to a feature pyramid network for object detection,” Pattern Analysis and Applications, vol. 26, no. 2, pp. 617-629, 2023, doi: 10.1007/s10044-023-01152-0.
X. Song, J. Huang, J. Cao, and D. Song, “Feature spatial pyramid network for low-light image enhancement,” The visual computer, vol. 39, no. 1, pp. 489-499, 2023, doi: 10.1007/s00371-021-02343-8.
G. Wang, X. Gan, Q. Cao, and Q. Zhai, “MFANet: multi-scale feature fusion network with attention mechanism,” The Visual Computer, vol. 39, no. 7, pp. 2969-2980, 2023, doi: 10.1007/s00371-022-02503-4.
Z. Li, S. Lu, Y. Dong, and J. Guo, “Msffa: a multi-scale feature fusion and attention mechanism network for crowd counting,” The Visual Computer, vol. 39, no. 3, pp. 1045-1056, 2023, doi: 10.1007/s00371-021-02383-0.
Y. Huang, P. Shi, H. He, H. He, and B. Zhao, “Senet: spatial information enhancement for semantic segmentation neural networks,” The Visual Computer, vol. 40, no. 5, pp. 3427-3440, 2024, doi: 10.1007/s00371-023-03043-1.
C. Zeng, Y. Zhao, Z. Wang, K. Li, X. Wan, and M. Liu, “Squeeze- and-excitation self-attention mechanism enhanced digital audio source recognition based on transfer learning,” Circuits, Systems, and Signal Processing, vol. 44, no. 1, pp. 480-512, 2025, doi: 10.1007/s00034-024-02850-8.
S. Kılıçarslan, K. Adem, and M. Çelik, “An overview of the activation functions used in deep learning algorithms,” Journal of New Results in Science, vol. 10, no. 3, pp. 75-88, 2021, doi: 10.54187/jnrs.1011739.
C. Dewi, R. C. Chen, H. Yu, and X. Jiang, “Robust detection method for improving small traffic sign recognition based on spatial pyramid pooling,” Journal of Ambient Intelligence and Humanized Computing, vol. 14, no. 7, pp. 8135-8152, 2023, doi: 10.1007/s12652-021-03584-0.
J. Chen, F. Qin, F. Lu, L. Guo, C. Li, K. Yan, et al., “CSPP-IQA: a multi-scale spatial pyramid pooling-based approach for blind image quality assessment,” Neural Computing and Applications, vol. 37, no. 11, pp. 7499-7510, 2025, doi: 10.1007/s00521-022-07874-2.
M. Ju, J. Luo, Z. Wang, and H. Luo, “Adaptive feature fusion with attention mechanism for multi-scale target detection,” Neural Computing and Applications, vol. 33, no. 7, pp. 2769-2781, 2021, doi: 10.1007/s00521-020-05150-9.
J. Wang, H. Shao, Y. Peng, and B. Liu, “PSparseFormer: Enhancing fault feature extraction based on parallel sparse selfattention and multiscale broadcast feedforward block,” IEEE Internet of Things Journal, vol. 11, no. 13, pp. 22982-22991, 2024, doi: 10.1109/JIOT.2024.3377674.
A. Elen, S. Ba¸s, and C. Közkurt, “An adaptive Gaussian kernel for support vector machine,” Arabian Journal for Science and Engineering, vol. 47, no. 8, pp. 10579-10588, 2022, doi: 10.1007/s13369-022-06654-3.