Design of Multimodal Behavior Analysis and Evaluation System for Digital Leadership of University Teachers Integrated with Perceiver IO
Main Article Content
Abstract
University ideological and political education faces difficulties in quantitatively evaluating teachers’ digital leadership in complex digital teaching environments. This study proposes a multimodal evaluation framework based on the Perceiver IO architecture, using cross-modal attention and structured queries to model digital leadership behaviors across pedagogical contexts. Multimodal inputs, including speech, visual behavior, instructional content, and interaction logs, are mapped into a shared latent space and decoded into six leadership dimensions. Supervised learning enables structured interpretation of leadership patterns and dimension-specific scoring. Experimental results demonstrate strong model performance, with an F1-score of 0.88 for technology application, 0.80 for value guidance, and an AUC of 0.90 for digital influence. These results indicate that the proposed framework can accurately and robustly assess digital leadership in complex teaching environments. By integrating speech, visual, content, and log-based information through a unified latent representation, the method provides an engineering-oriented approach for multimodal behavior analysis, interpretable teaching evaluation, and cross-modal decision support.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
J. A. Kelder, J. Crawford, I. Al Naabi, and L. To, “Enhancing digital productivity and capability in higher education through authentic leader behaviors: A cross-cultural structural equation model,” Education and Information Technologies, vol. 1, no. 1, pp. 1-17, 2025, doi: 10.1007/s10639-025-13422-x.
H. Antonopoulou, P. Matzavinou, I. Giannoukou, and C. Halkiopoulos, “Teachers’ Digital Leadership and Competencies in Primary Education: A Cross-Sectional Behavioral Study,” Education Sciences, vol. 15, no. 2, pp. 1-16, 2025, doi: 10.3390/educsci15020215.
T. C. Yang, “Assessment of the digital competencies of university instructors through use of the machine learning method,” SN Social Sciences, vol. 3, no. 2, pp. 25-41, 2023, doi: 10.1007/s43545-023-00617-7.
F. Tatari, M. Ebrahimi, and H. Tadayon, “Design, implementation, and evaluation of faculty member evaluation process using electronic 360-degree method,” BMC Medical Education, vol. 25, no. 1, pp. 941-953, 2025, doi: 10.1186/s12909-025-07384-4.
N. H. Hamzah, N. M. Radzi, and I. M. Omar, “Digital leadership competencies through a systematic literature review: bridging educational development and organizational success,” Education, vol. 7, no. 24, pp. 37-64, 2025, doi: 10.35631/ijmoe.724003.
E. Karimi, “Examining The Influence of Transformational Leadership on Digital Integration in Schools: The Role of Digital Competencies,” Journal of Educational Leadership and Management, vol. 3, no. 1, pp. 90-105, 2024.
M. Kamran, “Educational Leadership in a Digital Era: Navigating Challenges and Opportunities,” AL-ĪMĀN Research Journal, vol. 3, no. 01, pp. 188-199, 2025, doi: 10.63283/54wxa275.
M. Jing, Z. Guo, X. Wu, Z. Yang, and X. Wang, “Higher education digital academic leadership: Perceptions and practices from Chinese University Leaders,” Education Sciences, vol. 15, no. 5, pp. 606-633, 2025, doi: 10.3390/educsci15050606.
R. Pu, R. K. Dong, and S. Jiang, “Toward the Education for Sustainable Development (ESD): Digital leadership and knowledge-sharing behavior on the higher education institutional change,” Education and Information Technologies, vol. 3, no. 1, pp. 1-23, 2024.
Z. Cheng, A. Caliskan, N. B. K. Dinh, and C. Zhu, “A systematic review of digital academic leadership in higher education,” International Journal of Higher Education, vol. 13, no. 4, pp. 38-52, 2024, doi: 10.5430/ijhe.v13n4P38.
J. Jameson, N. Rumyantseva, M. Cai, M. Markowski, R. Essex, and I. McNay, “A systematic review and framework for digital leadership research maturity in higher education,” Computers and Education Open, vol. 3, no. 1, pp. 100115-100142, 2022, doi: 10.1016/j.caeo.2022.100115.
T. Karakose, H. Polat, T. Tülüba¸s, and M. Demirkol, “A review of the conceptual structure and evolution of digital leadership research in education,” Education Sciences, vol. 14, no. 11, pp. 1166-1185, 2024, doi: 10.3390/educsci14111166.
T. Karakose and T. Tülüba¸s, “Digital leadership and sustainable school improvement-a conceptual analysis and implications for future research,” Educational Process: International Journal, vol. 12, no. 1, pp. 7-18, 2023, doi: 10.22521/edupij.2023.121.1.
N. M. Ghamrawi and R. Tamim, “A typology for digital leadership in higher education: The case of a large-scale mobile technology initiative (using tablets),” Education and Information Technologies, vol. 28, no. 6, pp. 7089-7110, 2023, doi: 10.1007/s10639-022-11483-w.
T. Xu, W. Deng, S. Zhang, Y. Wei, and Q. Liu, “Research on recognition and analysis of teacher-student behavior based on a blended synchronous classroom,” Applied Sciences, vol. 13, no. 6, pp. 3432-3454, 2023, doi: 10.3390/app13063432.
Z. Xuan, “DRN-LSTM: a deep residual network based on long short-term memory network for students behaviour recognition in education,” Journal of Applied Science and Engineering, vol. 26, no. 2, pp. 245-252, 2022.
J. Mo, R. Zhu, H. Yuan, Z. Shou, and L. Chen, “Student behavior recognition based on multitask learning,” Multimedia tools and applications, vol. 82, no. 12, pp. 19091-19108, 2023, doi: 10.1007/s11042-022-14100-7.
G. Tuerhong, F. Fu, and M. Wushouer, “Adaptive multimodal transformer based on exchanging for multimodal sentiment analysis,” Scientific Reports, vol. 15, no. 1, pp. 27265-27278, 2025, doi: 10.1038/s41598-025-11848-4.
Y. Shi, J. Cai, and L. Liao, “Multi-task learning and mutual information maximization with crossmodal transformer for multimodal sentiment analysis,” Journal of Intelligent Information Systems, vol. 63, no. 1, pp. 1-19, 2025, doi: 10.1007/s10844-024-00858-9.
A. Nagrani, S. Yang, A. Arnab, A. Jansen, C. Schmid, and C. Sun, “Attention bottlenecks for multimodal fusion,” Advances in neural information processing systems, vol. 34, no. 1, pp. 14200-14213, 2021.
S. Nindam, S. H. Na, and H. J. Lee, “MultiFusedNet: A Multi-Feature Fused Network of Pretrained Vision Models via Keyframes for Student Behavior Classification,” Applied Sciences, vol. 14, no. 1, pp. 230-250, 2023, doi: 10.3390/app14010230.
G. Li, F. Liu, Y. Wang, Y. Guo, L. Xiao, and L. Zhu, “A convolutional neural network (CNN) based approach for the recognition and evaluation of classroom teaching behavior,” Scientific Programming, vol. 2021, no. 1, pp. 6336773-6336781, 2021, doi: 10.1155/2021/6336773.
J. Hewstone and R. Araya, “Neural network-based approach to detect and filter misleading audio segments in classroom automatic transcription,” Applied Sciences, vol. 13, no. 24, pp. 13243-13265, 2023, doi: 10.3390/app132413243.
L. Zhou, X. Liu, X. Guan, and Y. Cheng, “CSSA-YOLO: Cross-Scale Spatiotemporal Attention Network for Fine-Grained Behavior Recognition in Classroom Environments,” Sensors, vol. 25, no. 10, pp. 3132-3163, 2025, doi: 10.3390/s25103132.
K. Fatehi, M. T. Torres, and A. Kucukyilmaz, “An overview of high-resource automatic speech recognition methods and their empirical evaluation in low-resource environments,” Speech Communication, vol. 167, no. 1, pp. 103151-103181, 2025, doi: 10.1016/j.specom.2024.103151.
X. Qi, Y. Wen, P. Zhang, and H. Huang, “MFGCN: Multimodal fusion graph convolutional network for speech emotion recognition,” Neurocomputing, vol. 611, no. 1, pp. 128646-128662, 2025, doi: 10.1016/j.neucom.2024.128646.
J. Lei, J. Wang, and Y. Wang, “Multi-level attention fusion network assisted by relative entropy alignment for multimodal speech emotion recognition,” Applied Intelligence, vol. 54, no. 17, pp. 8478-8490, 2024, doi: 10.1007/s10489-024-05630-8.