Application of AI Large Models in Complex Data Processing: Analysis of Algorithm Optimization and Efficiency Improvement Paths
Main Article Content
Abstract
With the rapid development of the digital economy, complex data characterized by high dimensionality, heterogeneity, sparsity, and dynamics has become increasingly prevalent across industrial and scientific applications, making efficient processing a critical challenge for intelligent systems. In particular, the growing volume of multi-source sensing data generated in electromagnetic wave analysis, antenna systems, and wireless communication networks places higher demands on large-scale intelligent computing frameworks. This paper systematically investigates the major bottlenecks of AI large models in complex data processing and proposes a cross-scenario adaptive optimization framework integrating algorithm optimization and multi-dimensional efficiency enhancement strategies. At the algorithm level, adaptive attention architecture and multi-level regularization are introduced to improve feature representation and generalization capability. At the efficiency level, a collaborative “model– data–engineering” optimization scheme combining structured pruning, mixed-precision quantization, and knowledge distillation is developed. Experimental results demonstrate that the optimized 32B model reduces storage volume by 85% to 4.8 GB, decreases memory usage by 80%, improves inference speed by tenfold, and limits task-specific accuracy loss to less than 3%. Practical validations in medical, industrial, and astronomical scenarios further confirm simultaneous improvements in accuracy and computational efficiency. The proposed framework provides an effective solution for complex data processing while offering valuable technical references for electromagnetic signal intelligence, multisource sensor data fusion, and intelligent wireless information processing.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
P. Rahman, “Statistical analysis on complexly structured data,” Bulletin of the Australian Mathematical Society, pp. 111(1), 2025, doi: 10.1017/S0004972724001060.
D. Wang, K. Yin, and H. Wang, “Intelligent classification of construction quality problems based on unbalanced short text data mining,” Ain Shams Engineering Journal, vol. 15, no. 10, Art. no. 102983, 2024, doi: 10.1016/j.asej.2024.102983.
C. Li, Y. Zhang, Q. Kuang, Liu, and Y, “Deep quantile forests for high-dimensional data,” Statistical Theory and Related Fields, pp. 1-19, 2026, doi: 10.1080/24754269.2026.2616873.
X. Gao, X. Chen, Y. Fang, and H. Cheng, “A Data Governance Model Based on Multi-Agents System,” Proceedings of the 2025 IEEE 7th International Conference on Communications, Information System and Computer Engineering (CISCE); Piscataway, NJ, USA: IEEE; 2025, pp. 339-345, doi: 10.1109/CISCE65916.2025.11065402.
J. Yu, J. Nie, W. Li, and W. Chen, “Dual-track Innovation Driven by AI Large Models: Mechanism and Practical Exploration,” Technology Economics, vol. 43, no. 12, pp. 10-22, 2024, doi: 10.12404/j.issn.1002-980X.J24101816.
S. Almanasra, “Applications of integrating artificial intelligence and big data: A comprehensive analysis,” Journal of Intelligent Systems, pp. 33(1), 2024, doi: 10.1515/jisys-2024-0237.
Q. Wang, C. Yao, D. Li, J. Zhu, and W. Cao, “Research on the Fusion and Analysis Method for the Multi-Source Heterogeneous Elevator Risk Data Based on Deep Learning,” Proceedings of the 2024 International Conference on Electronics and Devices, Computational Science (ICEDCS); Piscataway, NJ, USA: IEEE; 2024, pp. 630-634, doi: 10.1109/icedcs64328.2024.00118.
M. A. Morid, O. R. L. Sheng, and J. Dunbar, “Time Series Prediction using Deep Learning Methods in Healthcare,” ACM Transactions on Management Information Systems, vol. 14, no. 1, pp. 1-29, 2023, doi: 10.1145/3531326.
E. Anderson and A. Cheng, “Portfolio Choices with Many Big Models,” Management Science, vol. 68, no. 1, pp. 690-715, 2022, doi: 10.1287/mnsc.2020.3876.
H. Wu and Z. Wen, “Research on Adaptive Attention Dense Network Structure in Camera Source Recognition Method,” Proceedings of the 2024 IEEE 23rd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom); Piscataway, NJ, USA: IEEE; 2024, pp. 2238-2243, doi: 10.1109/TrustCom63139.2024.00308.
Z. H. Tang, H. Lan, Z. Liu, Q. Wang, L. Zhao, and K. Li, “HiTrain: Heterogeneous Memory Offloading and I/O Optimization for Large Model Training,” Journal of Computer Research and Development, vol. 63, pp. 1-13, 2025, doi: 10.7544/issn1000-1239.202550478.
J. Chen, W. Yao, and J. Li, “Parallel optimization techniques for ultra-large scale refined numerical simulations on Chinese supercomputers,” Cluster Computing, vol. 28, no. 8, pp. 1-24, 2025, doi: 10.1007/s10586-024-05057-3.
L. Liu, G. Liu, B. Qi, X. Deng, D. Xue, and S. Qian, “Survey of Large Model Efficient Inference Technologies in Practical Application Scenarios,” Computer Science, vol. 53, no. 1, pp. 12-28, 2026. Available: https://link.cnki.net/urlid/50.1075.TP.20250703.1601.002.
C. Yao, G. Wang, P. Xiang, and G. Chen, “Opportunities, Challenges and Practices of Medical Large Models and Multi-modal Data Fusion in Smart Healthcare,” Chinese Journal of Health Informatics and Management, vol. 22, no. 02, pp. 171-178, 2025, doi: 10.3969/j.issn.1672-5166.2025.02.02.
Y. Pei, J. Cartlidge, A. Mandal, and D. Gold, “Cross-Modal Temporal Fusion for Financial Market Forecasting,” arXiv preprint arXiv:2504.13522, 2025, doi: 10.3233/FAIA251474.
S. Li, W. Yang, P. Zhang, X. Xiao, D. Cao, Y. Qin, et al., “Climatellm: Efficient Weather Forecasting via Frequency-aware Large Language Models,” arXiv preprint arXiv:2502.11059, 2025, doi: 10.48550/arXiv.2502.11059.
J. Li, L. Shi, M. Ding, Y. Lei, D. Zhao, and L. Chen, “Social Media Text Stance Detection Based on Large Language Models,” Journal of Frontiers of Computer Science and Technology, vol. 19, no. 5, pp. 1302-1312, 2025, doi: 10.3778/j.issn.1673-9418.2408074.
H. Vakharia and X. Du, “Efficient Multi-Resolution Fusion for Remote Sensing Data with Label Uncertainty,” 2024, doi: 10.1109/IGARSS52108.2023.10282851.