Smart Insole Pressure Distribution Prediction and Gait Optimization System Based on Multimodal Data Fusion and ViLT
Main Article Content
Abstract
To improve the accuracy and adaptability of gait analysis in embedded smart insole systems, this study proposes a multimodal pressure distribution prediction and gait optimization framework based on Vision-and-Language Transformer (ViLT) and cross-modal spatiotemporal modeling. The proposed approach addresses the limitations of modality isolation, temporal misalignment, and rigid fusion strategies by integrating plantar pressure sensing, inertial measurement unit (IMU) signals, and biomechanical parameters into a unified end-to-end architecture. A dynamic attention alignment mechanism is developed to establish semantic associations among heterogeneous data streams, while an LSTM-based spatiotemporal coupling model combined with hyperelastic mechanics enables accurate pressure prediction and personalized gait optimization. Considering that wearable sensing systems increasingly rely on electromagnetic signal acquisition and wireless data transmission, the proposed multimodal fusion framework also provides methodological support for intelligent sensing architectures employing antenna-enabled communication and electromagnetic information acquisition. Experimental results demonstrate that the proposed system achieves regional pressure prediction errors below 0.9% for diabetic foot subjects and approximately 0.8% for healthy subjects, while reducing peak plantar pressure by more than 12% and 8%, respectively. The hardware–algorithm co-design and closed-loop feedback strategy significantly enhance gait adaptability and rehabilitation effectiveness, providing a reliable computational framework for intelligent wearable sensing, multimodal electromagnetic data fusion, and nextgeneration smart healthcare applications.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
Y. Xie, L. Lu, F. Gao, S. J. He, H. J. Zhao, Y. Fang, et al., “Integration of artificial intelligence, blockchain, and wearable technology for chronic disease management: A new paradigm in smart healthcare,” Current Medical Science, vol. 41, no. 6, pp. 1123-1133, 2021.
M. T. H. Sarker, I. Ahmed, and M. A. Rahaman, “AI-Based Smart Textile Wearables for Remote Health Surveillance and Critical Emergency Alerts: A Systematic Literature Review,” American Journal of Scholarly Research and Innovation, vol. 2, no. 02, pp. 1-29, 2023.
B. Cheng and P. Wu, “Scalable fabrication of kevlar/Ti3C2T x MXene intelligent wearable fabrics with multiple sensory capabilities,” ACS nano, vol. 15, no. 5, pp. 8676-8685, 2021.
C. Park, R. Mishra, D. Vigano, M. Macagno, S. Rossotti, K. D’Huyvetter, et al., “Smart offloading boot system for remote patient monitoring: toward adherence reinforcement and proper physical activity prescription for diabetic foot ulcer patients,” Journal of Diabetes Science and Technology, vol. 17, no. 1, pp. 42-51, 2023.
S. C. Wu, G. Y. Bin, W. F. Shi, L. Lin, Y. Xu, D. Zhao, et al., “Empowering diabetic foot ulcer prevention: A novel cloud-based plantar pressure monitoring system for enhanced self-care,” Technology and Health Care, vol. 33, no. 2, pp. 701-718, 2025.
K. Xiang, M. Liu, J. Chen, Y. Bao, Z. Wang, K. Xiao, et al., “AI-assisted insole sensing system for multifunctional plantar-healthcare applications,” ACS Applied Materials & Interfaces, vol. 16, no. 25, pp. 32662-32678, 2024.
J. Baik, Y. Choi, K. J. Kim, and H. Lee, “Development of Novel Body Center of Pressure Estimation System based on Tactile Sensor and Deep Learning Models,” The Journal of Korea Robotics Society, vol. 20, no. 1, pp. 69-76, 2025.
R. Bajpai and D. Joshi, “Foot2hip: A deep neural network model for predicting lower limb kinematics from foot measurements,” IEEE/ASME Transactions on Mechatronics, vol. 28, no. 4, pp. 2248-2258, 2023.
L. Yue, L. Zongxing, D. Hui, J. Chao, L. Ziqiang, and L. Zhoujie, “How to achieve human–machine interaction by foot gesture recognition: A review,” IEEE Sensors Journal, vol. 23, no. 15, pp. 16515-16528, 2023.
S. Wei, Y. Zhang, and H. Liu, “A multimodal multilevel converged attention network for hand gesture recognition with hybrid sEMG and A-mode ultrasound sensing,” IEEE Transactions on Cybernetics, vol. 53, no. 12, pp. 7723-7734, 2022.
J. Zhang, Q. Wang, Q. Wang, and Z. Zheng, “Multimodal fusion framework based on statistical attention and contrastive attention for sign language recognition,” IEEE Transactions on Mobile Computing, vol. 23, no. 2, pp. 1431-1443, 2023.
H. Choi, J. P. Yun, B. J. Kim, H. Jang, and S. W. Kim, “Attention-based multimodal image feature fusion module for transmission line detection,” IEEE Transactions on Industrial Informatics, vol. 18, no. 11, pp. 7686-7695, 2022.
G. Li, L. Guo, R. Zhang, J. Qian, and S. Gao, “Transgait: Multimodal-based gait recognition with set transformer,” Applied Intelligence, vol. 53, no. 2, pp. 1535-1547, 2023.
T. Wang, G. Zhou, Y. Pu, R. Moreno, and G. Yang, “Gait recognition with global–local feature fusion based on swin transformer-3DCNN,” Signal, Image and Video Processing, vol. 19, no. 1, pp. 1-9, 2025.
J. Han, D. Wang, Z. Li, N. Dey, R. G. Crespo, and F. Shi, “Plantar pressure image classification employing residual-network model-based conditional generative adversarial networks: A comparison of normal, planus, and talipes equinovarus feet,” Soft Computing, vol. 27, no. 3, pp. 1763-1782, 2023.
P. C. B. S. Negi, S. S. Pandey, S. Sharma, and N. Sharma, “Scalogram-Based Gait Abnormalities Classification Using Deep Convolutional Networks for Neurological and Non-Neurological Disorders,” Journal of Medical and Biological Engineering, vol. 44, no. 3, pp. 375-389, 2024.
S. Jiao, H. Huo, W. Liu, C. Zhao, L. Ma, J. Wang, et al., “Gaitformer: A spatial-temporal attention-enhanced network without softmax for Parkinson’s disease early detection,” Complex & Intelligent Systems, vol. 11, no. 5, pp. 1-17, 2025.
S. U. Yunas and K. B. Ozanyan, “Gait activity classification using multimodality sensor fusion: A deep learning approach,” IEEE Sensors Journal, vol. 21, no. 15, pp. 16870-16879, 2021.
B. Ali, N. Faramarzi, U. Farooq, H. C. Bidsorkhi, A. G. D’Aloia, A. Tamburrano, et al., “Graphene-based smart insole sensor for pedobarometry and gait analysis,” IEEE Sensors Letters, vol. 7, no. 5, pp. 1-4, 2023.
Z. Q. Ling, Y. P. Zhang, G. Z. Cao, J. C. Chen, L. L. Li, D. P. Tan, et al., “AE-CNN-based multisource data fusion for gait motion step length estimation,” IEEE Sensors Journal, vol. 22, no. 21, pp. 20805-20815, 2022.
V. Chhoeum, Y. Kim, and S. D. Min, “A convolution neural network approach to access knee joint angle using foot pressure mapping images: A preliminary investigation,” IEEE Sensors Journal, vol. 21, no. 15, pp. 16937-16944, 2021.
A. Sharifi, M. Ahmadi, M. A. Mehni, S. Jafarzadeh Ghoushchi, and Y. Pourasad, “Experimental and numerical diagnosis of fatigue foot using convolutional neural network,” Computer Methods in Biomechanics and Biomedical Engineering, vol. 24, no. 16, pp. 1828-1840, 2021.
W. Shao, Z. You, L. Liang, X. Hu, C. Li, W. Wang, et al., “A multimodal gait analysis-based detection system of the risk of depression,” IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 10, pp. 4859-4868, 2021.
H. Wu, Y. Liu, H. Yang, Z. Xie, X. Chen, M. Wen, et al., “An Attention-based Temporal Network for Parkinson’s Disease Severity Rating using Gait Signals,” KSII Transactions on Internet and Information Systems (TIIS), vol. 17, no. 10, pp. 2627-2642, 2023.
K. Hu, Z. Wang, K. A. E. Martens, M. Hagenbuchner, M. Bennamoun, A. C. Tsoi, et al., “Graph fusion network-based multimodal learning for freezing of gait detection,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 3, pp. 1588-1600, 2021.
L. Xue and Z. Tao, “GaitRA: Triple-branch multimodal gait recognition with larger effective receptive fields and mixed attention,” Multimedia Tools and Applications, vol. 83, no. 33, pp. 80225-80259, 2024.
E. S. Rahayu, E. M. Yuniarno, I. K. E. Purnama, and M. H. Purnomo, “Adaptive Threshold Filtering to Reduce Noise in Elderly Activity Classification Using Bi-LSTM,” Jurnal Nasional Pendidikan Teknik Informatika: JANAPATI, vol. 13, no. 1, pp. 48-57, 2024.
L. D’arco, G. Mccalmont, H. Wang, and H. Zheng, “Application of smart insoles for recognition of activities of daily living: A systematic review,” ACM Transactions on Computing for Healthcare, vol. 5, no. 1, pp. 1-34, 2024.
P. Zhang, X. Zhang, M. Teng, L. Li, X. Liu, J. Feng, et al., “Leather-Based Shoe Soles for Real-Time Gait Recognition and Automatic Remote Assistance Using Machine Learning,” ACS Applied Materials & Interfaces, vol. 16, no. 45, pp. 62803-62816, 2024.
C. Huang, F. Nihey, K. Fukushi, H. Kajitani, Y. Nozaki, K. Ihara, et al., “Feature selection, construction, and validation of a lightweight model for foot function assessment during gait with in-shoe motion sensors,” IEEE Sensors Journal, vol. 23, no. 8, pp. 8839-8855, 2023.
E. Wang, X. Chen, Y. Li, Z. Fu, and J. Huang, “Lower limb motion intent recognition based on sensor fusion and fuzzy multitask learning,” IEEE Transactions on Fuzzy Systems, vol. 32, no. 5, pp. 2903-2914, 2024.
B. Yang, X. Zhang, J. Zhang, Z. Huang, Q. Lu, J. Zhang, et al., “Knee-Guard: A Calibration-free Wearable Monitoring System for Knee Osteoarthritis Gait Re-training via Effortless Wearing,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 8, no. 4, pp. 1-29, 2024.