Classification of Highly-Rated Comments in Rural Short Videos Based on Multimodal Emotion Features

Abstract

The rapid growth of rural short video platforms has generated large volumes of multimodal interaction data, where audiovisual signals and textual comments jointly reflect users’ emotional preferences and engagement patterns. However, semantic inconsistency between heterogeneous modalities and the lack of interpretable emotion modeling limit the accurate classification of highly-rated comments. This study proposes a cross-modal emotion alignment framework that integrates temporal audiovisual features with textual semantic representations for fine-grained comment classification. A dual-stream encoding network combines temporal attention and Wav2Vec2.0 to capture scene evolution, dialect rhythms, and ambient acoustic characteristics, while a domain-adapted text encoder extracts implicit emotional semantics from user comments. A bidirectional cross-attention gating module dynamically aligns multimodal features and suppresses semantic noise caused by irony and metaphor, followed by a multi-head classifier for structured emotion prediction. Experiments on a self-constructed rural short video corpus demonstrate that the proposed model achieves a Macro-F1 score of 89.4% while reducing the confusion rate to 11.9%, outperforming representative baseline methods. Beyond improving multimodal sentiment analysis, the proposed architecture establishes an effective signal-level fusion and feature interaction strategy for heterogeneous audiovisual data, providing useful insights for intelligent information perception, multimodal signal transmission, and adaptive communication systems in electromagnetic sensing and next-generation interactive media applications.