基于改进型双重深度确定性策略梯度与自适应分布式模型预测控制融合的电网侧储能系统协同优化方法

doi:10.19799/j.cnki.2095-4239.2025.0473

摘要/Abstract

摘要：

目的为应对可再生能源并网给电网侧带来的不确定性与复杂性，解决电网侧储能系统在容量分配、运行成本及能源消纳等方面的问题，提升电网稳定性与能源消纳效率。方法融合改进的深度确定性策略梯度（Deep Deterministic Policy Gradient，DDPG）算法与自适应分布式模型预测控制（Distributed Model Predictive Control，DMPC）方法，提出电网侧储能系统自适应协同优化策略。改进 DDPG 引入偏好体验回放和噪声调整机制以增强学习效率与探索能力，自适应 DMPC 通过分解大规模问题实现并行计算与局部优化。结果测试表明，相较于传统 DDPG 算法，该策略在电网侧储能容量分配优化、降低系统运行成本及提高可再生能源消耗率等方面效果显著。结论该策略为电网侧可再生能源储能系统的优化分配提供了创新解决方案，对保障电网稳定运行具有重要意义。

关键词: 储能系统, 协同优化, 可再生能源集成, 深度强化学习, 模型预测控制, 自适应控制

Abstract:

PurposeTo address the uncertainties and complexities brought to the grid - side by the grid connection of renewable energy, solve problems such as capacity allocation, operation cost, and energy accommodation of the grid - side energy storage system, and improve grid stability and energy accommodation efficiency.MethodBy fusing the improved Deep Deterministic Policy Gradient (DDPG) algorithm and the Adaptive Distributed Model Predictive Control (DMPC) method, an adaptive 协同 optimization strategy for the grid - side energy storage system is proposed. The improved DDPG introduces a preference experience replay and a noise adjustment mechanism to enhance learning efficiency and exploration ability. The adaptive DMPC realizes parallel computing and local optimization by decomposing large - scale problems.ResultTests show that, compared with the traditional DDPG algorithm, this strategy has remarkable effects in optimizing the capacity allocation of grid - side energy storage, reducing the system operation cost, and improving the renewable energy consumption rate.ConclusionThis strategy provides an innovative solution for the optimal allocation of the grid - side renewable energy storage system and is of great significance for ensuring the stable operation of the power grid.

Key words: Energy storage system, collaborative optimization, renewable energy integration, deep reinforcement learning, model predictive control, adaptive control

中图分类号:

TM73.TM91

谭金龙, 陈军, 赵启, 崔大林, 刘永强, 张路. 基于改进型双重深度确定性策略梯度与自适应分布式模型预测控制融合的电网侧储能系统协同优化方法[J]. 储能科学与技术, doi: 10.19799/j.cnki.2095-4239.2025.0473.

Jin-long TAN, Jun CHEN, Qi ZHAO, Da-lin CUI, Yong-qiang LIU, Lu ZHANG. A Grid-side Energy Storage System Optimization Method Based on Improved Twin Deep Deterministic Policy Gradient and Adaptive Distributed Model Predictive Control[J]. Energy Storage Science and Technology, doi: 10.19799/j.cnki.2095-4239.2025.0473.

图/表 6

图1

表1

图2

图3

图4

图5

参考文献 22

[1]	付诗意, 吕桃林, 闵凡奇, 等. 电动汽车用锂离子电池 SOC 估算方法综述[J].储能科学与技术,2021,10(03):1127-1136.
	FU S Y, LÜ T L, MIN F Q, et al. Review of SOC estimation methods for lithium-ion batteries used in electric vehicles[J]. Energy Storage Science and Technology, 2021, 10(03): 1127-1136.
[2]	李善成,方子健,李飞,等. 含多微电网租赁共享储能的配电网博弈优化调度 [J]. 中国电机工程学报,2022,42 (18):6611 - 6624.
	LI Xianshan,FANG Zijian,LI Fei,et al. Game - based optimal dispatching strategy for distribution network with multiple microgrids leasing shared energy storage [J]. Proceedings of the CSEE,2022,42 (18):6611 - 6624 (in Chinese).
[3]	陆秋瑜,杨银国,谢平平,等. 风电场集群租赁共享储能两阶段优化运行策略 [J]. 电网技术,2024,48 (3):1146 - 1159.
	LU Qiuyu,YANG Yinguo,XIE Pingping,et al. Two - stage optimal operation strategy of wind farm cluster leasing shared energy storage [J]. Power System Technology,2024,48 (3):1146 - 1159 (in Chinese).
[4]	ZHANG Ting,WU Yunna. Collaborative allocation model and balanced interaction strategy of multi flexible resources in the new power system based on Stackelberg game theory[J]. Renewable Energy,2024,220:119714.
[5]	YAN Haoran,HOU Hongjuan,DENG Min,et al. Stackelberg game theory based model to guide users' energy use behavior,with the consideration of flexible resources and consumer psychology,for an integrated energy system[J]. Energy,2024,288:129806.
[6]	REHMAN U N,UZAIR M,ALLAUDDIN U. An optical - energy model for optimizing the geometrical layout of solar photovoltaic arrays in a constrained field[J]. Renewable Energy,2020,149:55 - 65.
[7]	王书峰,钟明,许贤泽,等. 计及需求侧管理的新能源微电网多目标优化调度方法 [J]. 智慧电力,2022,50 (12):55 - 62.
	WANG Shufeng,ZHONG Ming,XU Xianze,et al. Multi objective optimization scheduling method for new energy microgrids considering demand side management [J]. Smart Power,2022,50 (12):55 - 62.
[8]	庄怀东,吴红斌,刘海涛,等. 含电动汽车的微网系统多目标经济调度 [J]. 电工技术学报,2014,29 (S1):365 - 373.
	ZHUANG Huaidong,WU Hongbin,LIU Haitao,et al. Multi - Objective economic dispatch of microgrid system considering electric vehicles [J]. Transactions of China Electrotechnical Society,2014,29 (S1):365 - 373.
[9]	ZHAO Z,GUO J,LUO X,et al. Distributed robust model predictive control - based energy management strategy for islanded multi - microgrids considering uncertainty[J]. IEEE Transactions on Smart Grid,2022,13(3):2107 - 2120.
[10]	胡戎,邱晓燕,张志荣. 计及灵活性资源的交直流混合配电网双层优化 [J]. 电网技术,2022,46 (6):2259 - 2268.
	HU Rong,QIU Xiaoyan,ZHANG Zhirong,et al. Bi - level optimization of AC/DC hybrid distribution network considering flexible resources [J]. Power System Technology,2022,46 (6):2259 - 2268.
[11]	杨龙杰,李华强,余雪莹,等. 计及灵活性的孤岛型微电网多目标日前优化调度方法 [J]. 电网技术,2018,42 (5):1432 - 1440.
	YANG Longjie,LI Huaqiang,YU Xueying,et al. Multi - objective day - ahead optimal scheduling of isolated microgrid considering flexibility [J]. Power System Technology,2018,42 (5):1432 - 1440.
[12]	冉金周,李华强,李彦君,等. 考虑灵活性供需匹配的孤岛微网优化调度策略 [J]. 太阳能学报,2022,43 (5):36 - 44.
	RAN Jinzhou,LI Huaqiang,LI Yanjun,et al. Optimal scheduling of isolated microgrid considering flexible power supply and demand [J]. Acta Energiae Solaris Sinica,2022,43 (5):36 - 44.
[13]	刘英琪,谢敏,韦薇,等. 高比例风电接入的电力系统灵活性评估与优化 [J]. 电力建设,2019,40 (9):1 - 10.
	LIU Yingqi,XIE Min,WEI Wei,et al. Assessment and optimization of power system flexibility with high proportion of wind power [J]. Electric Power Construction,2019,40 (9):1 - 10.
[14]	普智勇,张冬蕊,尹学卫,等. 不同储能对多时间尺度风电消纳特性对比研究 [J]. 节能技术,2020,38 (4):371 - 374,378.
	PU Zhiyong,ZHANG Dongrui,YI Xuewei,et al. Comparative study on the absorption characteristics of wind power at multiple time scales using different energy storage methods [J]. Energy Conservation Technology,2020,38 (4):371 - 374,378.
[15]	赵冬梅, 徐辰宇, 陶然, 等. 多元分布式储能在新型电力系统配电侧的灵活调控研究综述[J]. 中国电机工程学报, 2023, 43(5): 1776-1799.
	ZHAO D M, XU C Y, TAO R,et al. Review on flexible regulation of multiple distributed energy storage in distribution side of new power system[J]. Proceedings of the CSEE, 2023, 43(5): 1776-1799(in Chinese).
[16]	胡泽春, 夏睿, 吴林林, 等. 考虑储能参与调频的风储联合运行优化策略[J]. 电网技术, 2016, 40(8): 2251-2257.
	Hu Z C, Xia R, Wu L L, et al. Joint operation optimization of wind-storage union with energy storage participating frequency regulation[J]. Power System Technology, 2016, 40(8): 2251-2257.
[17]	李翠萍, 司文博, 李军徽, 等. 基于集合经验模态分解和多目标遗传算法的火-多储系统调频功率双层优化[J]. 电工技术学报, 2024, 39(7): 2017-2032.
	Li C P, Si W B, Li J H, et al. Two-layer optimization of frequency modulated power of thermal generation and multi-storage system based on ensemble empirical mode decomposition and multi-objective genetic algorithm[J]. TRANSACTIONS OF CHINA ELECTROTECHNICAL SOCIETY, 2024, 39(07): 2017-2032.
[18]	杨修宇,刘雪媛,郭琪,等。考虑辅助服务收益的储能与火电机组灵活性改造协调规划方法 [J]. 电网技术,2023,47 (4):1350 - 1359.
	YANG Xiuyu,LIU Xueyuan,GUO Qi,et al. Coordinated planning of energy storage and flexible retrofit of thermal power units considering ancillary service income[J]. Power System Technology,2023,47(4):1350 - 1359(in Chinese).
[19]	孙辉,范轩轩,胡姝博,等。虚拟电厂参与日前电力市场的内外协调竞标策略 [J]. 电网技术,2022,46 (4):1248 - 1258.
	SUN Hui,FAN Xuanxuan,HU Shubo,et al. Internal and external coordination bidding strategy of virtual power plant participating in day-ahead power market[J]. Power System Technology,2022,46(4):1248 - 1258(in Chinese).
[20]	黄邻熹,刘继春,刘阳。与新能源互补和独立参加多级市场的抽蓄电站容量分配策略 [J]. 电网技术,2024,48 (12):4948 - 4957.
	HUANG Linxi,LIU Jichun,LIU Yang. Capacity allocation strategies for pumped storage plants that complement renewable energy sources and participate independently in the multi-level market[J]. Power System Technology,2024,48(12):4948 - 4957(in Chinese).
[21]	李建林,郭兆东,曾伟,等.面向调频的锂电池储能建模及仿真分析[J].电力系统保护与控制,2022,50(13):33-42.
	LI J L, GUO Z D, ZENG W, et al. Modeling and simulation analysis of lithium battery energy storage for frequency regulation[J]. Power System Protection and Control, 2022, 50(13): 33-42.
[22]	谭必蓉,杜建华,叶祥虎,等.基于模型的锂离子电池 SOC 估计方法综述[J].储能科学与技术,2023,12(06):1995-2010.
	TAN B R, DU J H, YE X H, et al. A review of model-based SOC estimation methods for lithium-ion batteries[J]. Energy Storage Science and Technology, 2023, 12(06): 1995-2010.

内容	参数	数值
储能系统	充放电效率	90%
负荷模型	峰值负荷	300 MW
负荷模型	负荷预测误差	±5%
增强型DDPG	学习率	0.001
增强型DDPG	折现系数	0.95
自适应DMPC	子问题数量	10
自适应DMPC	迭代次数	100