Research on Multi-mode Path Planning Algorithm for Nanorobots Based on Improved Deep Reinforcement Learning
Main Article Content
Abstract
To address strong dynamic uncertainty and limited adaptability of traditional algorithms in nanorobot path planning, especially in microscopic textile fiber and porous-material environments, this study proposes a multi-mode path planning algorithm based on an improved deep Q-network (NP-IDQN). The method is applicable to nanorobots driven by fluid, chemical, or electromagnetic micro-actuation fields, where stable navigation must satisfy both trajectory efficiency and physical feasibility. The algorithm is optimized at three levels. At the network-architecture level, residual connections, layer normalization, and Transformer mechanisms are introduced to enhance feature extraction and global correlation modeling for fluid–obstacle interactions. At the training-strategy level, curriculum learning with multigeometry and multi-difficulty scenarios is adopted, covering rectangular, spherical, and mixed obstacles to improve environmental generalization. At the decision-making level, a multi-objective reward function integrates distance optimization, obstacle avoidance, energy efficiency, and microscopic driving-force constraints. Simulation results show that NP-IDQN improves trajectory stability and reduces the risk of local optima compared with conventional reinforcement learning and classical planning methods. The study provides technical support for precise navigation of nanorobots in complex fiber networks, functional material synthesis, environmental purification, and electromagneticactuated micro/nanorobotic systems.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
H. Liu, Z. Li, R. Zhang, Y. Liu, and Y. He, “A Novel Method for Technology Roadmapping: Nanorobots,” Appl Sci, vol. 14, Art. no. 10606, 2024, doi: 10.3390/app142210606.
J. Li, B. Esteban-Fernández de Ávila, W. Gao, et al., “Micro/nanorobots for biomedicine: Delivery, surgery, sensing, and detoxification,” Science robotics, vol. 2, no. 4, pp. eaam6431, 2017, doi: 10.1126/scirobotics.aam6431.
H. Zhou, C. Mayorga-Martinez C, S. Pané, et al., “Magnetically driven micro and nanorobots,” Chemical Reviews, vol. 121, no. 8, pp. 4999-5041, 2021, doi: 10.1021/acs.chemrev.0c01234.
M. Urso, M. Ussia, and M. Pumera, “Smart micro-and nanorobots for water purification,” Nature Reviews Bioengineering, vol. 1, no. 4, pp. 236-251, 2023, doi: 10.1038/s44222-023-00025-9.
X. Kong, P. Gao, J. Wang, et al., “Advances of medical nanorobots for future cancer treatments,” Journal of Hematology & Oncology, vol. 16, no. 1, pp. 74, 2023, doi: 10.1186/s13045-023-01463-z.
Z. Wu, Y. Chen, D. Mukasa, et al., “Medical micro/nanorobots in complex media,” Chemical Society Reviews, vol. 49, no. 22, pp. 8088-8112, 2020, doi: 10.1039/D0CS00309C.
J. Huang, C. Chen, J. Shen, et al., “A self-adaptive neighborhood search A-star algorithm for mobile robots global path planning,” Computers and Electrical Engineering, vol. 123, Art. no. 110018, 2025.
M. Ding, X. Zheng, L. Liu, et al., “Collision-free path planning for cable-driven continuum robot based on improved artificial potential field,” Robotica, vol. 42, no. 5, pp. 1350-1367, 2024, doi: 10.1017/S026357472400016X.
Z. Wu, Z. Meng, W. Zhao, and Z. Wu, “Fast-RRT: A RRT-Based Optimal Path Finding Method,” Appl Sci, vol. 11, Art. no. 11777, 2021, doi: 10.3390/app112411777.
C. Lamini, S. Benhlima, and A. Elbekri, “Genetic algorithm based approach for autonomous mobile robot path planning,” Procedia Computer Science, vol. 127, pp. 180-189, 2018.
M. Morin, I. Abi-Zeid, and G. Quimper C, “Ant colony optimization for path planning in search and rescue operations,” European Journal of Operational Research, vol. 305, no. 1, pp. 53-63, 2023.
G. Che, L. Liu, and Z. Yu, “An improved ant colony optimization algorithm based on particle swarm optimization algorithm for path planning of autonomous underwater vehicle,” Journal of Ambient Intelligence and Humanized Computing, vol. 11, no. 8, pp. 3349-3354, 2020, doi: 10.1007/s12652-019-01531-8.
A. Ahmad H, A. Hussain, and N. Akhtar M, “Enhancing Micro/Nanorobot Navigation for Medical Applications Using Reinforcement Learning,” 2024 19th International Conference on Emerging Technologies (ICET). IEEE, pp. 1-5. https://ieeexplore.ieee.org/abstract/document/10935174, 2024.
M. Xi, J. Yang, J. Wen, et al., “An information-assisted deep reinforcement learning path planning scheme for dynamic and unknown underwater environment,” IEEE Transactions on Neural Networks and Learning Systems, 2023, doi: 10.1109/tnnls.2023.3332172.
Z. Li, N. Shi, L. Zhao, et al., “Deep reinforcement learning path planning and task allocation for multi-robot collaboration,” Alexandria Engineering Journal, vol. 109, pp. 408-423, 2024.
J. Xin, H. Zhao, D. Liu, et al., “Application of deep reinforcement learning in mobile robot path planning,” 2017 Chinese Automation Congress (CAC). IEEE, pp. 7112-7116, 2017, doi: 10.1109/CAC.2017.8244061.
L. Yang, J. Jiang, F. Ji, et al., “Machine learning for micro-and nanorobots,” Nature Machine Intelligence, vol. 6, no. 6, pp. 605-618, 2024, doi: 10.1038/s42256-024-00859-x.
Y. Yang, L. Juntao, and P. Lingling, “Multi-robot path planning based on a deep reinforcement learning DQN algorithm,” CAAI Transactions on Intelligence Technology, vol. 5, no. 3, pp. 177-183, 2020, doi: 10.1049/trit.2020.0024.
J. Gao, W. Ye, J. Guo, and Z. Li, “Deep Reinforcement Learning for Indoor Mobile Robot Path Planning,” Sensors, vol. 20, pp. 5493, 2020, doi: 10.3390/s20195493.
N. Kumaar A A and S. Kochuvila, “Mobile service robot path planning using deep reinforcement learning,” IEEE Access, vol. 11, pp. 100083-100096, 2023, doi: 10.1109/ACCESS.2023.3311519.
Z. Gu, R. Zhu, T. Shen, et al., “Autonomous nanorobots with powerful thrust under dry solid-contact conditions by photothermal shock,” Nature Communications, vol. 14, no. 1, pp. 7663, 2023, doi: 10.1038/s41467-023-43433-6.
Y. Yang, A. Bevan M, and B. Li, “Efficient navigation of colloidal robots in an unknown environment via deep reinforcement learning,” Advanced Intelligent Systems, vol. 2, no. 1, Art. no. 1900106, 2020, doi: 10.1002/aisy.201900106.
L. Yang, J. Jiang, X. Gao, et al., “Autonomous environment-adaptive microrobot swarm navigation enabled by deep learning-based real-time distribution planning,” Nature Machine Intelligence, vol. 4, no. 5, pp. 480-493, 2022, doi: 10.1038/s42256-022-00482-8.
Z. Li, K. Wang, C. Hou, C. Li, F. Zhang, et al., “Self-Sensing Intelligent Microrobots for Noninvasive and Wireless Monitoring Systems,” Microsyst Nanoeng, vol. 9, no. 1, pp. 102, 2023, doi: 10.1038/s41378-023-00574-4.
Q. Wang, Q. Wang, Z. Ning, F. Chan K, J. Jiang, et al., “Tracking and Navigation of a Microswarm under Laser Speckle Contrast Imaging for Targeted Delivery,” Sci Robot, vol. 9, no. 87, 2024, doi: 10.1126/scirobotics.adh1978.
G. Li, C. Shao, Z. Wang, Y. Lu, K. Deng, and D. Gao, “A Dual-Robot Digital Radiographic Inspection System for Rocket Tank Welds,” Appl Syst Innov, vol. 8, pp. 151, 2025, doi: 10.3390/asi8050151.
X. Li, T. Li, Y. Zhang, Y. Zhang, Z. Li, L. Ban, and K. Sun, “A-TEB: An Improved A Algorithm Based on the TEB Strategy for Multi-Robot Motion Planning,” Sensors, vol. 25, pp. 6117, 2025, doi: 10.3390/s25196117.
S. Swinton, E. McGookin, and D. Thomson, “Improving Rover Path Planning in Challenging Terrains: A Comparative Study of RRT-Based Algorithms,” Robotics, vol. 14, pp. 135, 2025, doi: 10.3390/robotics14100135.
P. Zhang, J. Liu, Y. Fu, and J. Sun, “A Planning Framework Based on Semantic Segmentation and Flipper Motions for Articulated Tracked Robot in Obstacle-Crossing Terrain,” Biomimetics, vol. 10, pp. 627, 2025, doi: 10.3390/biomimetics10090627.
S. Lin, A. Liu, J. Wang, and X. Kong, “A Review of Path-Planning Approaches for Multiple Mobile Robots,” Machines, vol. 10, pp. 773, 2022, doi: 10.3390/machines10090773.