Extracting Emotional Feature Changes in College Music Performances via Multi-Channel Convolutional Attention Mechanism
Main Article Content
Abstract
To address the limitations of traditional emotional-feature extraction methods for musical performances, including low accuracy, inadequate temporal-continuity modeling, and weak differentiation of subtle emotional changes, this study proposes a multi-channel convolutional attention-based method for extracting emotional feature variations. A multimodal data acquisition system for university music performances is first constructed to collect audio data, including pitch, rhythm, and dynamics, and visual data, including facial expressions and body movements, from vocal and instrumental performances. A multi-channel convolutional neural network is then designed for multidimensional feature extraction from heterogeneous data. The audio channel uses 1D-CNN to capture time-domain and frequencydomain features, while the visual channel combines 2D-CNN and 3D-CNN to extract spatial and temporal sequence features. A hybrid attention mechanism integrating channel attention and temporal attention is introduced to adaptively weight key emotional features and enhance the capture of dynamic emotional changes. Experiments on the self-built UMPED dataset and the public MusicNet dataset evaluate emotion-feature similarity, classification accuracy, and temporal emotion-change fitting, verifying the effectiveness of the proposed multimodal signal-processing framework.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
W. Yang, Y. Liu, Y. Chen, Y. Cui, C. Guo, G. Shen, et al., “HSMNet: A multi-resolution grayscale image steganalysis method based on hybrid dilated convolution and self-attention multi-channel network,” Information Sciences, vol. 728, 122824, 2026, doi: 10.1016/j.ins.2025.122824.
R. Li, G. Zhang, Y. Niu, K. Rong, W. Liu, and H. Hong, “A multi-channel multi-scale spatiotemporal convolutional cross-attention fusion network for bearing fault diagnosis,” Sensors (Basel, Switzerland), vol. 25, no. 18, 5923, 2025, doi: 10.3390/s25185923.
H. Mulam, V. Rameshkumaar, M. Bachute, and E. Muniyandy, “Dual attention holographic convolutional neural network based ictal and interictal states of automatic seizure detection using multi-channel scalp EEG with Humboldt squid optimization algorithm,” Smart Health, vol. 37, 100599, 2025, doi: 10.1016/j.smhl.2025.100599.
P. Zhou, Z. Qian, D. Fu, and Y. Zhang, “Plunger pump fault diagnosis method based on wavelet convolution and multi-head self-attention mechanism for multi-channel feature fusion,” Journal of Computer and Communications, vol. 13, no. 7, pp. 345-356, 2025, doi: 10.4236/jcc.2025.137018.
M. Yang, R. Jiang, B. Wang, G. Fang, Y. Jia, and F. Fan, “Multi-channel attention mechanism graph convolutional network considering cumulative effect and temporal causality for day-ahead wind power prediction,” Energy, vol. 332, 137023, 2025, doi: 10.1016/j.energy.2025.137023.
Y. Lin, S. Pan, J. Yu, Y. Hong, F. Wang, L. Zheng, et al., “MDCA-DETR: DETR with multi-channel deformable convolution and coordinate attention for mini-LED wafer surface defects detection,” Optics and Lasers in Engineering, vol. 193, 109082, 2025, doi: 10.1016/j.optlaseng.2025.109082.
X. Wu, Y. Fu, Q. Wu, Z. Wen, H. Dong, and D. Zhang, “Gearbox fault diagnosis based on multi-channel orthogonal attention network,” Signal, Image and Video Processing, vol. 19, no. 8, 651, 2025, doi: 10.1007/s11760-025-04132-5.
L. Luo, X. Fan, M. Wang, Z. Liu, and Y. Zhang, “Two-stage deep complex gated convolution attention network for multi-channel noise suppression,” Signal, Image and Video Processing, vol. 19, no. 7, 512, 2025, doi: 10.1007/s11760-025-04158-9.
M. V. Divya, “Multi-channel mantis search spatial attention convolutional neural network based hyperspectral image change detection,” Multimedia Tools and Applications, vol. 84, no. 33, pp. 1-22, 2025, doi: 10.1007/s11042-025-20810-5.
R. Zhao, C. Zhang, and D. Xue, “A multi-scale multi-channel CNN introducing a channel-spatial attention mechanism hyperspectral remote sensing image classification method,” European Journal of Remote Sensing, vol. 57, no. 1, 2353290, 2024, doi: 10.1080/22797254.2024.2353290.
A. G. Kakisim and Z. Turgut, “Multi-channel convolutional neural network with attention mechanism using dualband WiFi signals for indoor positioning systems in smart buildings,” Internet of Things, vol. 29, 101435, 2025, doi: 10.1016/j.iot.2024.101435.
Z. Qu, L. Zhou, X. Yin, and T. Lu, “A crack detection network with multi-channel attention and enhanced information interaction,” Signal, Image and Video Processing, vol. 19, no. 1, 37, 2024, doi: 10.1007/s11760-024-03581-8.
L. Shi, W. Liu, Y. Wu, C. Dai, Z. Ji, and I. Ganchev, “Knowledge graph embedding using a multi-channel interactive convolutional neural network with triple attention,” Mathematics, vol. 12, no. 18, 2821, 2024, doi: 10.3390/math12182821.
J. Liu, Y. Huang, D. Wu, Y. Yang, Y. Chen, L. Chen, et al., “Multi-channel multi-scale convolution attention variational autoencoder (MCA-VAE): An interpretable anomaly detection algorithm based on variational autoencoder,” Sensors, vol. 24, no. 16, 5316, 2024, doi: 10.3390/s24165316.
S. Liu, H. Zhang, A. Zhang, J. Liu, and Y. Liu, “Subsurface temperature estimation of mesoscale eddies in the Northwest Pacific Ocean from satellite observations using a residual muti-channel attention convolution network,” Frontiers in Marine Science, vol. 11, 1397109, 2024, doi: 10.3389/fmars.2024.1397109.
Y. Wang, Y. Zhou, W. Lu, Q. Wu, Q. Li, and R. Zhang, “AC-CfC: An attention-based convolutional closed-form continuous-time neural network for raw multi-channel EEG-based emotion recognition,” Biomedical Signal Processing and Control, vol. 94, 106249, 2024, doi: 10.1016/j.bspc.2024.106249.
M. Zhu and L. Wen, “A multi-channel convolutional neural network based on attention mechanism fusion for facial expression recognition,” Applied Mathematics and Nonlinear Sciences, vol. 9, no. 1, 2, 2024, doi: 10.2478/amns.2023.1.00084.
X. Yang, J. Zhu, and J. Huang, “A multi-channel CNN fault diagnosis method based on squeeze-and-convolution attention for rotating machinery: Papers,” Journal of Advanced Mechanical Design, Systems, and Manufacturing, vol. 18, no. 8, JAMDSM0097, 2024, doi: 10.1299/jamdsm.2024jamdsm0097.
J. He, H. Yang, S. Zhou, J. Chen, and M. Chen, “A dual-attention-mechanism multi-channel convolutional LSTM for short-term wind speed prediction,” Atmosphere, vol. 14, no. 1, 71, 2022, doi: 10.3390/atmos14010071.
V. Mayya, S. Kamath, G. S. Krishnan, and T. Gangavarapu, “Multi-channel, convolutional attention based neural model for automated diagnostic coding of unstructured patient discharge summaries,” Future Generation Computer Systems, vol. 118, pp. 374-391, 2021, doi: 10.1016/j.future.2021.01.013.
Y. Geng, L. Wang, J. Hu, S. Luo, and Q. Zhou, “A dual category-channel attention convolutional network fault diagnosis method for imbalanced multi-sensor data of spacecraft,” Aerospace Science and Technology, vol. 170, 111593, 2026, doi: 10.1016/j.ast.2025.111593.
J. Liao and L. Wang, “MSDANet: A multiscale dual-channel spatial attention network with depthwise separable convolution for hyperspectral image classification,” Canadian Journal of Remote Sensing, vol. 51, no. 1, 2551533, 2025, doi: 10.1080/07038992.2025.2551533.
J. Yin, Z. Huang, Q. Chen, W. Huang, L. Sun, Q. Wu, et al., “Cascaded local–nonlocal pansharpening with adaptive channel-kernel convolution and multi-scale large-kernel attention,” Remote Sensing, vol. 18, no. 1, 97, 2025, doi: 10.3390/rs18010097.
Y. Tang and Z. Luo, “A dual channel fusion method for wind power prediction based on graph attention network and wavelet convolutional neural network,” Electric Power Systems Research, vol. 254, 112659, 2026, doi: 10.1016/j.epsr.2025.112659.
M. Zolfaghari, A. M. Saniee, and H. Sajedi, “Channel-spatial attention modules in convolutional neural networks for image classification,” Scientific Reports, vol. 16, no. 1, 2348, 2025, doi: 10.1038/s41598-025-32055-1.
J. A. Ramirez-Quintana, J. J. Muñoz-Pacheco, G. Ramirez-Alonso, J. A. Medrano-Hermosillo, and A. D. Corral-Saenz, “Lightweight convolutional neural network with efficient channel attention mechanism for real-time facial emotion recognition in embedded systems,” Sensors, vol. 25, no. 23, 7264, 2025, doi: 10.3390/s25237264.
A. Salam, M. Abrar, R. W. Anwer, F. Amin, F. Ullah, I. de la Torre, et al., “Channel-attention DenseNet with dilated convolutions for MRI brain tumor classification,” Computer Modeling in Engineering & Sciences, vol. 145, no. 2, pp. 2457-2479, 2025, doi: 10.32604/cmes.2025.072765.
Y. Lin, M. Zhao, and B. Su, “Lcpa-msdce UNet: A unet variant integrating lightweight channel-pixel attention and multi-scale dilated convolution enhancement modules for medical image segmentation,” The Journal of Supercomputing, vol. 81, no. 17, 1581, 2025, doi: 10.1007/s11227-025-08069-x.
Y. Zhang, B. Qi, H. Yu, Y. Shi, and Z. Huang, “MSDC-CCA: DWT-based infrared image destripper with multi-scale dilated convolution and coordinate-channel attention,” Infrared Physics and Technology, vol. 152, 106242, 2026, doi: 10.1016/j.infrared.2025.106242.
S. Luo, X. Gong, W. Du, L. Wang, K. Feng, and Y. Qian, “Small sample fault diagnosis method based on dual convolutional kernel feature fusion and channel attention weighted temporal convolutional network (DCK-CAM-TCN),” Journal of Vibroengineering, vol. 27, no. 7, pp. 1278-1295, 2025, doi: 10.21595/jve.2025.25034.