Dynamic Pricing Strategies for the Electricity Retail Market Using Reinforcement Learning Algorithms
Main Article Content
Abstract
The dynamic pricing mechanism in the electricity retail market plays a pivotal role in achieving optimal resource allocation on both supply and demand sides. However, the complex heterogeneity of consumer behavior at the retail level coupled with frequent price fluctuations in the wholesale market poses significant challenges to traditional optimization approaches. This paper models the dynamic pricing problem of power retailers as a Markov decision process within a continuous action space and proposes a solution framework based on proximal policy optimization algorithms. Centered on maximizing cumulative profits for power retailers, the framework integrates user price response characteristics, procurement cost constraints in wholesale markets, and regulatory boundaries for retail pricing into a unified modeling system, enabling adaptive generation of retail electricity prices across time periods through continuous interaction between agents and their environment. To address common issues with standard PPO algorithms–including premature policy degradation and insufficient exploration efficiency–the study introduces an adaptive exploration coefficient adjustment scheme based on policy entropy, along with a generalized advantage estimation method to reduce gradient estimation variance. Simulation experiments using actual operational data from a provincial electricity market in China during 2023 demonstrated that the proposed algorithm outperforms benchmark algorithms such as the Deep Deterministic Policy Gradient and Dual Delayed Deep Deterministic Policy Gradient in key metrics including average daily retailer profits, peak load reduction rates, and price series stability, validating the feasibility and practical effectiveness of reinforcement learning approaches for dynamic pricing in electricity retail markets.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
References
A. M. H. Shehzad, A. Ussama, F. Umar, et al., “Dynamic Price-Based Demand Response through Linear Regression for Microgrids with Renewable Energy Resources,” Energies, vol. 15, no. 4, pp. 1385–1385, 2022, doi: 10.3390/EN15041385.
Li Bin, Guo Huifang, Xue Li, et al., “Research on Key Technologies for the Development of Power Procurement Agency Services and Auxiliary Decision Support,” Inner Mongolia Electric Power Technology, vol. 41, no. 3, pp. 51–56, 2023, doi: 10.19929/j.cnki.nmgdljs.2023.0039.
Li Yuan, Chi Kun, Wang Zhou, et al., “Pricing strategies for multi-electricity-gas-heating microgrid systems based on reinforcement learning,” Southern Power Grid Technology, vol. 18, no. 1, pp. 94–101, 2024, doi: 10.13648/j.cnki.issn1674-0629.2024.01.010.
Q. Liu, D. Feng, Y. Zhou, et al., “Bounded rational bidding strategy of GenCo in electricity spot market based on prospect theory and distributional reinforcement learning,” Sustainable Energy, Grids and Networks, vol. 44, pp. 102036–102036, 2025, doi: 10.1016/J.SEGAN.2025.102036.
W. Huo, Y. Zhang, H. Zhao, et al., “Price signal forecasting for day-ahead offering strategy of price-maker renewable energy producers considering different risk preferences,” Applied Energy, vol. 401, no. PC, pp. 126819– 126819, 2025, doi: 10.1016/J.APENERGY.2025.126819.
Xu Dan, Hu Xiaojing, Hu Fei, et al., “Electricity market quantity-price combined bidding strategy based on deep reinforcement learning,” Grid Technology, vol. 48, no. 8, pp. 3278–3286, 2024, doi: 10.13335/j.1000-3673.pst.2023.2100.
Liu Feiyu, Wang Jiwen, Wang Zhengfeng, et al., “Research on multi-agent free-cooperative bidding mechanisms based on a two-stage deep reinforcement learning algorithm,” China Journal of Electrical Engineering, vol. 44, no. 12, pp. 4626–4638, 2024, doi: 10.13334/j.0258-8013.pcsee.222935.
L. Ting, Z. Weige, and D. Xiaowei, “Operation Strategy of Electricity Retailers Based on Energy Storage System to Improve Comprehensive Profitability in China’s Electricity Spot Market,” Energies, vol. 14, no. 19, pp. 6424–6424, 2021, doi: 10.3390/EN14196424.
S. Yan, W. Yin, W. Yan, et al., “Optimizing intelligent startup strategy of power system using PPO algorithm,” Intelligent Decision Technologies, vol. 18, no. 4, pp. 3091–3104, 2024, doi: 10.3233/IDT-240122.
W. Lulu, Z. Kaile, L. Jun, et al., “Modified deep learning and reinforcement learning for an incentive-based demand response model,” Energy, vol. 205, 2020, doi: 10.1016/j.energy.2020.118019.
Z. Xu, S. Yuanzhang, Y. Jun, et al., “Day-ahead energy pricing and management method for regional integrated energy systems considering multi-energy demand responses,” Energy, vol. 251, 2022, doi: 10.1016/J.ENERGY.2022.123914.
J. Zhao, Z. Gao, and Z. Chen, “Energy Management of Electric-Hydrogen Coupled Integrated Energy System Based on Improved Proximal Policy Optimization Algorithm,” Energies, vol. 18, no. 15, pp. 3925–3925, 2025, doi: 10.3390/EN18153925.
Song Yonghua, Yu Peipei, and Zhang Hongcai, “Demand response operation control of regional cooling systems based on reinforcement learning with composite two-end sampling under real-time electricity pricing mechanisms,” China Science: Technical Science, vol. 53, no. 10, pp. 1699– 1712, 2023.
A. Natalia and V. Nikolai, “The Optimal Mechanism Design of Retail Prices in the Electricity Market for Several Types of Consumers,” Mathematics, vol. 9, no. 10, pp. 1147–1147, 2021, doi: 10.3390/MATH9101147.
Li Chaoying and Tan Qinliang, “Market trading strategies for thermal power enterprises under the new power system based on agent modeling,” China Electric Power, vol. 57, no. 2, pp. 212–225, 2024, doi: 10.11930/j.issn.1004-9649.202303088.
B. Mandal, K. P. Roy, and C. Paul, “Dynamic Economic Dispatch Problem in Hybrid Renewable Energy Sources Based Power Systems Using Chaotic Hippopotamus Optimization Algorithm,” Iranian Journal of Science and Technology, Transactions of Electrical Engineering, vol. 50, no. 2, pp. 1–28, 2025, doi: 10.1007/S40998-025-00893-4.
“Research Analysis on Price Elasticity and Demand Response Behavior in the Electricity Retail Market,” Beijing: China Electricity Council, 2022, Report No.: CEC-2022-MR-087.