A grid-side energy storage system optimization method based on improved twin deep deterministic policy gradient and adaptive distributed model predictive control

doi:10.19799/j.cnki.2095-4239.2025.0473

Abstract

Abstract:

To address the uncertainties and complexities brought to the grid-side by the grid connection of renewable energy, to solve problems such as capacity allocation, operation cost, and energy accommodation of the grid-side energy storage system, and to improve grid stability and energy accommodation efficiency. Through the integration of the enhanced Deep Deterministic Policy Gradient (DDPG) algorithm and the Adaptive Distributed Model Predictive Control (DMPC) approach, an adaptive collaborative optimization strategy for the grid-side energy storage system is proposed. The enhanced DDPG incorporates a preference experience replay and a noise adjustment mechanism, thereby enhancing learning efficiency and exploration ability. The adaptive DMPC performs parallel computing and local optimization by decomposing large-scale problems. Compared with the traditional DDPG algorithm, this strategy has been shown to have remarkable effects in optimizing the capacity allocation of grid-side energy storage, reducing the system operation cost, and improving the renewable energy consumption rate. This strategy provides an innovative solution for the optimal allocation of the grid-side renewable energy storage system and is of great significance for ensuring the stable operation of the power grid.

Key words: energy storage system, collaborative optimization, renewable energy integration, deep reinforcement learning, model predictive control, adaptive control

CLC Number:

TM 73

Jinlong TAN, Jun CHEN, Qi ZHAO, Dalin CUI, Yongqiang LIU, Lu ZHANG. A grid-side energy storage system optimization method based on improved twin deep deterministic policy gradient and adaptive distributed model predictive control[J]. Energy Storage Science and Technology, 2025, 14(11): 4289-4299.

Figures/Tables 6

Fig. 1

Table 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

References 22

[1]	付诗意, 吕桃林, 闵凡奇, 等. 电动汽车用锂离子电池SOC估算方法综述[J]. 储能科学与技术, 2021, 10(3): 1127-1136. DOI: 10.19799/j.cnki.2095-4239.2021.0013.
	FU S Y, LYU T L, MIN F Q, et al. Review of estimation methods on SOC of lithium-ion batteries in electric vehicles[J]. Energy Storage Science and Technology, 2021, 10(3): 1127-1136. DOI: 10.19799/j.cnki.2095-4239.2021.0013.
[2]	李咸善, 方子健, 李飞, 等. 含多微电网租赁共享储能的配电网博弈优化调度[J]. 中国电机工程学报, 2022, 42(18): 6611-6625. DOI: 10.13334/j.0258-8013.pcsee.211424.
	LI X S, FANG Z J, LI F, et al. Game-based optimal dispatching strategy for distribution network with multiple microgrids leasing shared energy storage[J]. Proceedings of the CSEE, 2022, 42(18): 6611-6625. DOI: 10.13334/j.0258-8013.pcsee.211424.
[3]	陆秋瑜, 杨银国, 谢平平, 等. 风电场集群租赁共享储能两阶段优化运行策略[J]. 电网技术, 2024, 48(3): 1146-1165. DOI: 10.13335/j.1000-3673.pst.2023.0104.
	LU Q Y, YANG Y G, XIE P P, et al. Two-stage optimal operation strategy of wind farm cluster leasing shared energy storage[J]. Power System Technology, 2024, 48(3): 1146-1165. DOI: 10. 13335/j.1000-3673.pst.2023.0104.
[4]	ZHANG T, WU Y N. Collaborative allocation model and balanced interaction strategy of multi flexible resources in the new power system based on Stackelberg game theory[J]. Renewable Energy, 2024, 220: 119714. DOI: 10.1016/j.renene.2023.119714.
[5]	YAN H R, HOU H J, DENG M, et al. Stackelberg game theory based model to guide users' energy use behavior, with the consideration of flexible resources and consumer psychology, for an integrated energy system[J]. Energy, 2024, 288: 129806. DOI: 10.1016/j.energy.2023.129806.
[6]	REHMAN N U, UZAIR M, ALLAUDDIN U. An optical-energy model for optimizing the geometrical layout of solar photovoltaic arrays in a constrained field[J]. Renewable Energy, 2020, 149: 55-65. DOI: 10.1016/j.renene.2019.12.040.
[7]	王书峰, 钟明, 许贤泽, 等. 计及需求侧管理的新能源微电网多目标优化调度方法[J]. 智慧电力, 2022, 50(12): 55-62.
	WANG S F, ZHONG M, XU X Z, et al. Multi-objective optimal scheduling method for renewable energy microgrid considering demand side management[J]. Smart Power, 2022, 50(12): 55-62.
[8]	庄怀东, 吴红斌, 刘海涛, 等. 含电动汽车的微网系统多目标经济调度[J]. 电工技术学报, 2014, 29(S1): 365-373. DOI: 10.19595/j.cnki.1000-6753.tces.2014.s1.051.
	ZHUANG H D, WU H B, LIU H T, et al. Multi-objective economic dispatch of microgrid system with electric vehicle[J]. Transactions of China Electrotechnical Society, 2014, 29(S1): 365-373. DOI: 10.19595/j.cnki.1000-6753.tces.2014.s1.051.
[9]	ZHAO Z L, GUO J T, LUO X, et al. Distributed robust model predictive control-based energy management strategy for islanded multi-microgrids considering uncertainty[J]. IEEE Transactions on Smart Grid, 2022, 13(3): 2107-2120. DOI: 10. 1109/TSG.2022.3147370.
[10]	胡戎, 邱晓燕, 张志荣. 计及灵活性资源的交直流混合配电网双层优化[J]. 电网技术, 2022, 46(6): 2259-2268. DOI: 10.13335/j.1000-3673.pst.2021.1036.
	HU R, QIU X Y, ZHANG Z R. Bi-level optimization of AC/DC hybrid distribution network considering flexible resources[J]. Power System Technology, 2022, 46(6): 2259-2268. DOI: 10.13335/j.1000-3673.pst.2021.1036.
[11]	陈铭, 梅诗妍. 基于风险分析和模糊专家系统的电力工程应急成本估算方法[J]. 沈阳工业大学学报,2025, 47(3): 281-287. DOI: 10.7688/j.issn.1000-1646.2025.03.02.
	CHEN M, MEI S Y. Emergency cost estimation method for power engineering based on risk analysis and fuzzy expert system[J]. Journal of Shenyang University of Technology, 2025, 47(3): 281-287. DOI: 10.7688/j.issn.1000-1646.2025.03.02.
[12]	冉金周, 李华强, 李彦君, 等. 考虑灵活性供需匹配的孤岛微网优化调度策略[J]. 太阳能学报, 2022, 43(5): 36-44. DOI: 10.19912/j.0254-0096.tynxb.2020-0938.
	RAN J Z, LI H Q, LI Y J, et al. Optimal scheduling of isolated microgrid considering flexible power supply and demand[J]. Acta Energiae Solaris Sinica, 2022, 43(5): 36-44. DOI: 10.19912/j.0254-0096.tynxb.2020-0938.
[13]	刘英琪, 谢敏, 韦薇, 等. 高比例风电接入的电力系统灵活性评估与优化[J]. 电力建设, 2019, 40(9): 1-10.
	LIU Y Q, XIE M, WEI W, et al. Assessment and optimization for power system flexibility with high proportion of wind power[J]. Electric Power Construction, 2019, 40(9): 1-10.
[14]	普智勇, 张冬蕊, 尹学卫, 等. 不同储能对多时间尺度风电消纳特性对比研究[J]. 节能技术, 2020, 38(4): 371-374, 378.
	PU Z Y, ZHANG D R, YIN X W, et al. Comparative study of different energy storage systems on multi-timescale wind power smoothing performance[J]. Energy Conservation Technology, 2020, 38(4): 371-374, 378.
[15]	赵冬梅, 徐辰宇, 陶然, 等. 多元分布式储能在新型电力系统配电侧的灵活调控研究综述[J]. 中国电机工程学报, 2023, 43(5): 1776-1799. DOI: 10.13334/j.0258-8013.pcsee.220778.
	ZHAO D M, XU C Y, TAO R, et al. Review on flexible regulation of multiple distributed energy storage in distribution side of new power system[J]. Proceedings of the CSEE, 2023, 43(5): 1776-1799. DOI: 10.13334/j.0258-8013.pcsee.220778.
[16]	胡泽春, 夏睿, 吴林林, 等. 考虑储能参与调频的风储联合运行优化策略[J]. 电网技术, 2016, 40(8): 2251-2257. DOI: 10.13335/j.1000-3673.pst.2016.08.001.
	HU Z C, XIA R, WU L L, et al. Joint operation optimization of wind-storage union with energy storage participating frequency regulation[J]. Power System Technology, 2016, 40(8): 2251-2257. DOI: 10.13335/j.1000-3673.pst.2016.08.001.
[17]	李翠萍, 司文博, 李军徽, 等. 基于集合经验模态分解和多目标遗传算法的火-多储系统调频功率双层优化[J]. 电工技术学报, 2024, 39(7): 2017-2032. DOI: 10.19595/j.cnki.1000-6753.tces.230186.
	LI C P, SI W B, LI J H, et al. Two-layer optimization of frequency modulated power of thermal generation and multi-storage system based on ensemble empirical mode decomposition and multi-objective genetic algorithm[J]. Transactions of China Electrotechnical Society, 2024, 39(7): 2017-2032. DOI: 10.19595/j.cnki.1000-6753.tces.230186.
[18]	付慧敏, 郑刚. 基于数字孪生的配电网智能化故障诊断方法 [J]. 沈阳工业大学学报, 2025, 47 (3): 288-294. DOI: 10.7688/j.issn.1000-1646.2025.03.03.
	FU H M, ZHENG G. Intelligent fault diagnosis method for distribution network based on digital twin [J]. Journal of Shenyang University of Technology, 2025, 47 (3): 288-294. DOI: 10.7688/j.issn.1000-1646.2025.03.03.
[19]	孙辉, 范轩轩, 胡姝博, 等. 虚拟电厂参与日前电力市场的内外协调竞标策略[J]. 电网技术, 2022, 46(4): 1248-1262. DOI: 10.13335/j.1000-3673.pst.2021.1819.
	SUN H, FAN X X, HU S B, et al. Internal and external coordination bidding strategy of virtual power plant participating in day-ahead power market[J]. Power System Technology, 2022, 46(4): 1248-1262. DOI: 10.13335/j.1000-3673.pst.2021.1819.
[20]	买买提.努尔, 张艳丽, 魏文倩, 等. 基于短路比指标的新能源并网容量极限计算[J]. 沈阳工业大学学报, 2024, 46 (6): 742-747. DOI: 10.7688/j.issn.1000-1646.2024.06.02.
	MAIMAITI N, ZHANG Y L, WEI W Q, et al. Calculation of new energy grid-connected capacity limit based on short-circuit ratio index [J]. Journal of Shenyang University of Technology, 2024, 46 (6): 742-747. DOI: 10.7688/j.issn.1000-1646.2024.06.02.
[21]	李建林, 郭兆东, 曾伟, 等. 面向调频的锂电池储能建模及仿真分析[J]. 电力系统保护与控制, 2022, 50(13): 33-42. DOI: 10.19783/j.cnki.pspc.226227.
	LI J L, GUO Z D, ZENG W, et al. Modeling and simulation analysis of lithium battery energy storage oriented to frequency modulation[J]. Power System Protection and Control, 2022, 50(13): 33-42. DOI: 10.19783/j.cnki.pspc.226227.
[22]	谭必蓉, 杜建华, 叶祥虎, 等. 基于模型的锂离子电池SOC估计方法综述[J]. 储能科学与技术, 2023, 12(6): 1995-2010. DOI: 10.19799/j.cnki.2095-4239.2023.0016.
	TAN B R, DU J H, YE X H, et al. Overview of SOC estimation methods for lithium-ion batteries based on model[J]. Energy Storage Science and Technology, 2023, 12(6): 1995-2010. DOI: 10.19799/j.cnki.2095-4239.2023.0016.

内容	参数	数值
储能系统	充放电效率	90%
负荷模型	峰值负荷	300 MW
负荷模型	负荷预测误差	±5%
增强型DDPG	学习率	0.001
增强型DDPG	折现系数	0.95
自适应DMPC	子问题数量	10
自适应DMPC	迭代次数	100