Traffic Signal Dynamic Optimization Algorithm Based on Deep Reinforcement Learning
Main Article Content
Abstract
With the acceleration of urbanization, traffic congestion at multiple intersections has become a core bottleneck restricting urban operational efficiency. To address this, this paper proposes a traffic signal dynamic optimization algorithm, the Cross-Attention Mechanism and Dueling Double DQN (CAM-D3QN). This method utilizes a novel crisscross attention module to dynamically model spatial dependencies between intersections and incorporates the Dueling Double DQN architecture for robust Q-value estimation. Validated on CityFlow using Grid-4×4 and Hangzhou-real networks, CAM-D3QN significantly outperforms the state-of-the-art baseline, GPLight, achieving relative improvements of approximately 10.5% in average vehicle delay, 11.3% in average queue length, 3.3% in throughput, alongside notable reductions in stops (12.1%) and fuel consumption (7.2%). Ablation experiments further demonstrate that removing the cross-attention module increases queue length by 30.6% in sudden congestion scenarios. The proposed method achieves superior performance to the baseline across four typical traffic scenarios on the Hangzhou-real road network, demonstrating its generalization capabilities. By leveraging the coordinated optimization of dynamic spatial perception and robust value assessment, this paper provides an effective solution for efficient and robust coordinated traffic signal control. The framework can be combined with traffic states acquired from radar, roadside sensors or wireless communication units in intelligent transportation systems.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
D. R. Aleko and S. Djahel, “An efficient adaptive traffic light control system for urban road traffic congestion re-duction in smart cities,” Information, vol. 11, no. 2, pp. 119-119, 2020, doi: 10.3390/info11020119.
L. F. P. De Oliveira, L. T. Manera, and P. D. G. Da Luz, “Development of a smart traffic light control system with real-time monitoring,” IEEE Internet of Things Journal, vol. 8, no. 5, pp. 3384-3393, 2020, doi: 10.1109/JIOT.2020.3022392.
A. Atta, S. Abbas, M. A. Khan, G. Ahmed, and U. Farooq, “An adaptive approach: Smart traffic congestion control system,” Journal of King Saud University-Computer and Information Sciences, vol. 32, no. 9, pp. 1012-1019, 2020, doi: 10.1016/j.jksuci.2018.10.011.
S. S. S. M. Qadri and E. Oner, “State-of-art review of traffic signal control methods: challenges and opportunities,” European transport research review, vol. 12, no. 1, pp. 55-55, 2020, doi: 10.1186/s12544-020-00439-1.
F. S. P. Borges, A. P. Fonseca, and R. C. Garcia, “Deep reinforcement learning model to mitigate congestion in re-al-time traffic light networks,” Infrastructures, vol. 6, no. 10, pp. 138-138, 2021, doi: 10.3390/infrastructures6100138.
T. Wu, P. Zhou, K. Liu, Y. Yuan, X. Wang, H. Huang, et al., “Multi-agent deep reinforcement learning for urban traffic light control in vehicular networks,” IEEE Transactions on Vehicular Technology, vol. 69, no. 8, pp. 8243-8256, 2020, doi: 10.1109/TVT.2020.2997896.
R. Zhao, H. Hu, Y. Li, Y. Fan, F. Gao, Z. Gao, et al., “Sequence Decision Transformer for Adaptive Traffic Signal Control,” Sensors, vol. 24, no. 19, pp. 6202-6202, 2024, doi: 10.3390/s24196202.
M. Yang, Y. Wang, Y. Yu, and M. Zhou, “MixLight: Mixed-agent cooperative reinforcement learning for traffic light control,” IEEE Transactions on Industrial Informatics, vol. 20, no. 2, pp. 2653-2661, 2023, doi: 10.1109/TII.2023.3296910.
Q. U. Jamil, K. D. Kallu, M. J. Khan, M. Safdar, A. Zafar, M. U. Ali, et al., “Urban traffic signal control optimization through Deep Q Learning and double Deep Q Learning: a novel approach for efficient traffic management,” Multimedia Tools and Applications, vol. 84, no. 22, pp. 24933-24956, 2025, doi: 10.1007/s11042-024-20060-x.
S. M. M. R. Swapno, S. M. N. Nobel, P. Meena, V. P. Meena, A. T. Azar, Z. Haider, et al., “A reinforcement learning ap-proach for reducing traffic congestion using deep Q learning,” Scientific Reports, vol. 14, no. 1, pp. 30452-30452, 2024, doi: 10.1038/s41598-024-75638-0.
M. Eom and B. I. Kim, “The traffic signal control problem for intersections: a review,” European transport research review, vol. 12, no. 1, pp. 50-50, 2020, doi: 10.1186/s12544-020-00440-8.
H. Wei, G. Zheng, V. Gayah, and Z. Li, “Recent advances in reinforcement learning for traffic signal control: A sur-vey of models and evaluation,” ACM SIGKDD explorations newsletter, vol. 22, no. 2, pp. 12-18, 2021, doi: 10.1145/3447556.3447565.
W. H. Lee and C. Y. Chiu, “Design and implementation of a smart traffic signal control system for smart city applica-tions,” Sensors, vol. 20, no. 2, pp. 508-508, 2020, doi: 10.3390/s20020508.
X. Wang, L. Ke, Z. Qiao, and X. Chai, “Large-scale traffic signal control using a novel multiagent reinforcement learning,” IEEE transactions on cybernetics, vol. 51, no. 1, pp. 174-187, 2020, doi: 10.1109/TCYB.2020.3015811.
Z. Fu, J. Zhang, F. Tao, and B. Ji, “Traffic signal phase control at urban isolated intersections: an adaptive strategy utilizing the improved D3QN algorithm,” Measurement Science and Technology, vol. 36, no. 1, pp. 016203-016203, 2024, doi: 10.1088/1361-6501/ad8212.
L. Li-Juan, G. M. Bai, and H. R. Karimi, “Advanced Traffic Signal Control System Using Deep Double Q-Learning with Pedestrian Factors,” Journal of Artificial Intelligence and Soft Computing Research, vol. 15, no. 3, pp. 239-255, 2025, doi: 10.2478/jaiscr-2025-0012.
S. Li, P. Shi, A. Yang, H. Qi, and X. Dong, “Dual-Priority Delayed Deep Double Q-Network (DPD3QN): A Dueling Dou-ble Deep Q-Network with Dual-Priority Experience Replay for Autonomous Driving Behavior Deci-sion-Making,” Algorithms, vol. 18, no. 5, pp. 291-291, 2025, doi: 10.3390/a18050291.
B. Wang, Z. He, J. Sheng, and Y. Chen, “Deep reinforcement learning for traffic light timing optimization,” Processes, vol. 10, no. 11, pp. 2458-2458, 2022, doi: 10.3390/pr10112458.
Y. Dong, H. Huang, G. Zhang, and J. Jin, “Adaptive Transit Signal Priority Control for Traffic Safety and Efficiency Optimization: A Multi-Objective Deep Reinforcement Learning Framework,” Mathematics, vol. 12, no. 24, pp. 3994-3994, 2024, doi: 10.3390/math12243994.
Z. Yang, Y. Kong, and C. H. Hsia, “Derlight: A deep reinforcement learning traffic light control algorithm with dual experience replay,” Journal of Internet Technology, vol. 25, no. 1, pp. 79-86, 2024, doi: 10.53106/160792642024012501007.
N. Triki, M. Karray, and M. Ksantini, “A real-time traffic sign recognition method using a new attention-based deep convolutional neural network for smart vehicles,” Applied Sciences, vol. 13, no. 8, pp. 4793-4793, 2023, doi: 10.3390/app13084793.
Z. Wang, H. Zhu, M. He, Y. Zhou, X. Luo, N. Zhang, et al., “GAN and multi-agent DRL based decentralized traffic light signal control,” IEEE Transactions on Vehicular Technology, vol. 71, no. 2, pp. 1333-1348, 2021, doi: 10.1109/TVT.2021.3134329.
H. Mei, X. Lei, L. Da, B. Shi, and H. Wei, “Libsignal: An open library for traffic signal control,” Machine Learning, vol. 113, no. 8, pp. 5235-5271, 2024, doi: 10.1007/s10994-023-06412-y.
P. Michailidis, I. Michailidis, C. R. Lazaridis, and E. Kosmatopoulos, “Traffic Signal Control via Reinforcement Learning: A Review on Applications and Innovations,” Infrastructures, vol. 10, no. 5, pp. 114-114, 2025, doi: 10.3390/infrastructures10050114.
Q. Wu, J. Wu, J. Shen, Yong, B, and Q. Zhou, “An edge based multi-agent auto communication method for traffic light control,” Sensors, vol. 20, no. 15, pp. 4291-4291, 2020, doi: 10.3390/s20154291.