Coordinated Optimization of Orderly Charging and Grid Interaction at Electric Vehicle Charging Stations Based on Multi-Agent Reinforcement Learning
Main Article Content
Abstract
Uncoordinated charging of large-scale electric vehicles exacerbates peak-valley differences and voltage exceedance risks in the power grid, while existing scheduling methods still have limitations in distributed decision-making, dynamic pricing, and multiobjective balancing. These problems become more significant in charging station clusters where power-electronic converters, communication links, and complex electromagnetic operating environments jointly affect grid interaction stability. In this paper, a collaborative optimization framework based on multi-agent reinforcement learning is proposed for orderly charging at electric vehicle charging stations and coordinated interaction with the power grid. First, each charging station is modeled as an autonomous agent, and distributed environment modeling is realized based on local observation information and Markov decision processes. Second, a proximal policy optimization algorithm is used to generate a dynamic service fee multiplier in a continuous action space, which is combined with a demand elasticity module to form an adaptive pricing mechanism. Finally, a composite reward system integrating grid stability, operational revenue, and user satisfaction is developed, and multi-agent convergence training is achieved through parameter sharing and generalized advantage estimation. The results confirm the overall benefits of joint optimization in load shaping, economic performance, and robustness, providing a technical reference for intelligent charging coordination under grid interaction and electromagnetic compatibility constraints.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
C. B. Saner, A. Trivedi, and D. Srinivasan, “A cooperative hierarchical multi-agent system for EV charging scheduling in presence of multiple charging stations,” IEEE Transactions on Smart Grid, vol. 13, no. 3, pp. 2218-2233, 2022, doi: 10.1109/TSG.2022.3140927.
S. Shao, H. Sartipizadeh, and A. Gupta, “Scheduling EV charging having demand with different reliability constraints,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 10, pp. 11018-11029, 2023, doi: 10.1109/TITS.2023.3279070.
K. Qian, R. Fachrizal, J. Munkhammar, et al., “The impact of considering state-of-charge-dependent maximum charging powers on the optimal electric vehicle charging scheduling,” IEEE Transactions on Transportation Electrification, vol. 9, no. 3, pp. 4517-4530, 2023, doi: 10.1109/TTE.2023.3245332.
K. E. Adetunji, I. W. Hofsajer, A. M. Abu-Mahfouz, et al., “A two-tailed pricing scheme for optimal EV charging scheduling using multiobjective reinforcement learning,” IEEE Transactions on Industrial Informatics, vol. 20, no. 3, pp. 3361-3370, 2023, doi: 10.1109/TII.2023.3305682.
U. Qureshi, A. Ghosh, and B. K. Panigrahi, “Scheduling and routing of mobile charging stations with stochastic travel times to service heterogeneous spatiotemporal electric vehicle charging requests with time windows,” IEEE Transactions on Industry Applications, vol. 58, no. 5, pp. 6546-6556, 2022, doi: 10.1109/TIA.2022.3182323.
U. Qureshi, A. Ghosh, and B. K. Panigrahi, “Multiobjective pareto-optimal intelligent electric vehicle charging schedule in a commercial charging station: A Stochastic convex optimization approach,” IEEE Transactions on Industrial Informatics, vol. 20, no. 11, pp. 12620-12632, 2024, doi: 10.1109/TII.2024.3423373.
K. S. Arikumar, S. B. Prathiba, R. S. Moorthy, et al., “Software defined networking assisted electric vehicle charging: Towards smart charge scheduling and management,” IEEE Transactions on Network Science and Engineering, vol. 11, no. 1, pp. 163-173, 2023, doi: 10.1109/TNSE.2023.3293053.
S. Kiani, K. Sheshyekani, and H. Dagdougui, “ADMM-based hierarchical single-loop framework for EV charging scheduling considering power flow constraints,” IEEE Transactions on Transportation Electrification, vol. 10, no. 1, pp. 1089-1100, 2023, doi: 10.1109/TTE.2023.3269050.
S. P. Sone, J. J. Lehtomaki, Z. Khan, et al., “Robust EV scheduling in charging stations under uncertain demands and deadlines,” IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 12, pp. 21484-21499, 2024, doi: 10.1109/TITS.2024.3466514.
L. Affolabi, M. Shahidehpour, F. Rahimi, et al., “Hierarchical transactive energy scheduling of electric vehicle charging stations in constrained power distribution and transportation networks,” IEEE Transactions on Transportation Electrification, vol. 9, no. 2, pp. 3398-3409, 2022, doi: 10.1109/TTE.2022.3219721.
A. S. Bouhouras, D. Kothona, P. A. Gkaidatzis, et al., “Distribution network energy loss reduction under EV charging schedule,” International journal of energy research, vol. 46, no. 6, pp. 8256-8270, 2022, doi: 10.1002/er.7727.
M. Farhoumandi, S. Bahramirad, M. Shahidehpour, et al., “Blockchain for Peer-to-Peer Energy Trading in Electric Vehicle Charging Stations With Constrained Power Distribution and Urban Transportation Networks,” Energy Internet, vol. 2, no. 1, 2025, doi: 10.1049/ein2.12026.
S. S. Shuvo and Y. Yilmaz, “Demand-side and utility-side management techniques for increasing EV charging load,” IEEE Transactions on Smart Grid, vol. 14, no. 5, pp. 3889-3898, 2023, doi: 10.1109/TSG.2023.3235903.