Automatic News Headline Generation and Topic Consistency Assessment Based on SimCSE and Knowledge Graph
Main Article Content
Abstract
To address the problems of insufficient semantic alignment and topic drift in industry news and market report headline generation, this paper proposes a method that fuses SimCSE semantic constraints with knowledge graph entity associations, paying particular attention to key entities related to names of people, organizations, locations, and fields. Considering the growing demand for reliable information processing in intelligent communication systems and electromagnetic information transmission environments, the proposed framework provides a semantic enhancement mechanism that is beneficial for trustworthy content understanding in data-driven applications. Based on SimCSE, the news text is encoded at the sentence level, and contrastive learning is used to improve semantic representation consistency. Named entity recognition is performed and aligned with the Wikidata knowledge graph to construct a local knowledge subgraph containing entity relationships. A graph attention network is used to fuse SimCSE sentence embeddings with knowledge graph entity embeddings to generate knowledge-enhanced representations. The fused representation is applied during the Transformer decoding process to dynamically focus on key entities and core semantics, generating thematically consistent and factually accurate headlines. Experimental results show that the proposed method achieves an average BERTScore of at least 0.87 and an average BLEU score of at least 0.75 across different news domains. In terms of topic consistency, the Topic-F1 score remains at 0.835 ± 0.019 under the highcomplexity condition of four topics, and the topic deviation is as low as 0.165 ± 0.021. In scenarios involving the summarization of complex manufacturing processes or market trend analysis, this study effectively addresses the issues of insufficient semantic alignment and topic drift while providing a reference for semantic information processing in intelligent electromagnetic communication systems.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
K. Singh R, S. Khetarpaul, R. Gorantla, et al., “SHEG: summarization and headline generation of news articles using deep learning,” Neural Computing and Applications, vol. 33, no. 8, pp. 3251-3265, 2021, doi: 10.1007/s00521-020-05188-9.
O. Turner S, L. Buttle, D. Linden S L V, and et al."More" Is More Interesting When Framing Headlines: The Influence of Comparative Language Framing and Consistency With Prior Beliefs on Engagement With Health-Related News, “Applied Cognitive Psychology, 2025, 39(5):e70118,”, doi: 10.1002/acp.70118.
R. Das and D. Singh T, “Assamese news image caption generation using attention mechanism,” Multimedia Tools and Applications, vol. 81, no. 7, pp. 10051-10069, 2022, doi: 10.1007/s11042-022-12042-8.
A. Table, “Generating headlines for Turkish news texts with transformer architecture based deep learning method,” Journal of the Faculty of Engineering and Architecture of Gazi University, vol. 39, no. 1, pp. 485-495, 2024.
N. Hagar, N. Diakopoulos, and B. DeWilde, “Anticipating attention: On the predictability of news headline tests,” Digital Journalism, vol. 10, no. 4, pp. 647-668, 2022, doi: 10.1080/21670811.2021.1984266.
D. Ismayanti, R. Said Y, N. Usman, et al., “The Students Ability in Translating Newspaper Headlines into English A Case Study,” IDEAS: Journal on English Language Teaching and Learning, Linguistics and Literature, vol. 12, no. 1, pp. 108-131, 2024, doi: 10.24256/ideas.v12i1.4767.
H. Gonçalo Oliveira, “Automatic generation of creative text in Portuguese: an overview,” Language Resources and Evaluation, vol. 58, no. 1, pp. 7-41, 2024, doi: 10.1007/s10579-023-09646-3.
T. Shohan F, T. Nayeem M, S. Islam, et al., “XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags,” Findings of the Association for Computational Linguistics ACL 2024, pp. 12991-13024, 2024, doi: 10.18653/v1/2024.findings-acl.771.
D. Vincentio A and S. Hansun, “A Fine-Tuned BART Pre-trained Language Model for the Indonesian Question-Answering Task,” Engineering, Technology & Applied Science Research, vol. 15, no. 2, pp. 21398-21403, 2025, doi: 10.48084/etasr.9828.
R. Goyal, P. Kumar, and P. Singh V, “Automated question and answer generation from texts using text-to-text transformers,” Arabian Journal for Science and Engineering, vol. 49, no. 3, pp. 3027-3041, 2024, doi: 10.1007/s13369-023-07840-7.
N. Mulla and P. Gharpure, “Leveraging well-formedness and cognitive level classifiers for automatic question generation on Java technical passages using T5 transformer,” International Journal of Information Technology, vol. 15, no. 4, pp. 1961-1973, 2023, doi: 10.1007/s41870-023-01262-2.
P. Mishra, C. Diwan, S. Srinivasa, et al., “Automatic title generation for learning resources and pathways with pretrained transformer models,” International Journal of Semantic Computing, vol. 15, no. 04, pp. 487-510, 2021, doi: 10.1142/S1793351X21400134.
B. Juarto and S. Girsang A, “Neural collaborative with sentence BERT for news recommender system,” JOIV: International Journal on Informatics Visualization, vol. 5, no. 4, pp. 448-455, 2021, doi: 10.30630/joiv.5.4.678.
J. Pijeira-Díaz H, S. Subramanya, J. van de Pol, et al., “Evaluating Sentence-BERT-powered learning analytics for auto-mated assessment of students’ causal diagrams,” Journal of Computer Assisted Learning, vol. 40, no. 6, pp. 2667-2680, 2024, doi: 10.1111/jcal.12992.
R. Wang, F. Hou, F. Cahan S, et al., “Fine-grained entity typing with a type taxonomy: a systematic review,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 5, pp. 4794-4812, 2022, doi: 10.1109/TKDE.2022.3148980.
S. Tang, Q. Yang, L. Fan, et al., “Contrastive learning-based semantic communications,” IEEE Transactions on Communications, vol. 72, no. 10, pp. 6328-6343, 2024, doi: 10.1109/TCOMM.2024.3400912.
T. Xiao, S. Liu, S. De Mello, et al., “Learning contrastive representation for semantic correspondence,” International Journal of Computer Vision, vol. 130, no. 5, pp. 1293-1309, 2022, doi: 10.1007/s11263-022-01602-y.
W. He, C. Wu, and S. Zhou, “Study on short text clustering with unsupervised SimCSE,” Computer Science, vol. 50, no. 11, pp. 71-76, 2023.
Y. Ye and S. Ji, “Sparse graph attention networks,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 1, pp. 905-916, 2021, doi: 10.1109/TKDE.2021.3072345.
Y. Liu, S. Yang, Y. Xu, et al., “Contextualized graph attention network for recommendation with item knowledge graph,” IEEE Transactions on knowledge and data engineering, vol. 35, no. 1, pp. 181-195, 2021, doi: 10.1109/TKDE.2021.3082948.
S. Li and Y. Zhang, “Improving entity linking by combining semantic entity embeddings and cross-attention encoder,” Journal of Intelligent & Fuzzy Systems, vol. 46, no. 1, pp. 2899-2910, 2024, doi: 10.3233/JIFS-233124.