基于改进型双重深度确定性策略梯度与自适应分布式模型预测控制融合的电网侧储能系统协同优化方法

doi:10.19799/j.cnki.2095-4239.2025.0473

储能科学与技术 ›› 2025, Vol. 14 ›› Issue (11): 4289-4299.doi: 10.19799/j.cnki.2095-4239.2025.0473

基于改进型双重深度确定性策略梯度与自适应分布式模型预测控制融合的电网侧储能系统协同优化方法

谭金龙¹(), 陈军², 赵启², 崔大林³, 刘永强², 张路²

^1.重庆大学，输配电装备及系统安全与新技术国家重点实验室，重庆 400044
^2.国网新疆电力有限公司电力科学研究院，新疆乌鲁木齐 830011
^3.国网新疆电力有限公司，新疆乌鲁木齐 830002

收稿日期:2025-05-20 修回日期:2025-06-09 出版日期:2025-11-28 发布日期:2025-11-24
通讯作者: 谭金龙 E-mail:473052259@qq.com
作者简介:谭金龙（1989—），男，硕士，高级工程师，主要研究方向为电力系统保护与控制，E-mail：473052259@qq.com。
基金资助:
国网新疆电力有限公司科技项目(5230DK230013)

A grid-side energy storage system optimization method based on improved twin deep deterministic policy gradient and adaptive distributed model predictive control

Jinlong TAN¹(), Jun CHEN², Qi ZHAO², Dalin CUI³, Yongqiang LIU², Lu ZHANG²

^1.State Key Laboratory of Power Transmission and Distribution Equipment and System Safety and New Technology, Chongqing University, Chongqing 400044, China
^2.Electric Power Research Institute, State Grid Xinjiang Electric Power Co. , Ltd. , Urumqi 830011, Xinjiang, China
^3.State Grid Xinjiang Electric Power Co. , Ltd. , Urumqi 830002, Xinjiang, China

Received:2025-05-20 Revised:2025-06-09 Online:2025-11-28 Published:2025-11-24
Contact: Jinlong TAN E-mail:473052259@qq.com

摘要/Abstract

摘要：

为应对可再生能源并网给电网侧带来的不确定性与复杂性，解决电网侧储能系统在容量分配、运行成本及能源消纳等方面的问题，提升电网稳定性与能源消纳效率，本文融合改进的深度确定性策略梯度(deep deterministic policy gradient，DDPG)算法与自适应分布式模型预测控制(distributed model predictive control，DMPC)方法，提出电网侧储能系统自适应协同优化策略。改进DDPG引入偏好体验回放和噪声调整机制以增强学习效率与探索能力，自适应DMPC通过分解大规模问题实现并行计算与局部优化。测试表明，相较于传统 DDPG 算法，该策略在电网侧储能容量分配优化、降低系统运行成本及提高可再生能源消纳率等方面效果显著。该策略为电网侧可再生能源储能系统的优化分配提供了创新解决方案，对保障电网稳定运行具有重要意义。

关键词: 储能系统, 协同优化, 可再生能源集成, 深度强化学习, 模型预测控制, 自适应控制

Abstract:

To address the uncertainties and complexities brought to the grid-side by the grid connection of renewable energy, to solve problems such as capacity allocation, operation cost, and energy accommodation of the grid-side energy storage system, and to improve grid stability and energy accommodation efficiency. Through the integration of the enhanced Deep Deterministic Policy Gradient (DDPG) algorithm and the Adaptive Distributed Model Predictive Control (DMPC) approach, an adaptive collaborative optimization strategy for the grid-side energy storage system is proposed. The enhanced DDPG incorporates a preference experience replay and a noise adjustment mechanism, thereby enhancing learning efficiency and exploration ability. The adaptive DMPC performs parallel computing and local optimization by decomposing large-scale problems. Compared with the traditional DDPG algorithm, this strategy has been shown to have remarkable effects in optimizing the capacity allocation of grid-side energy storage, reducing the system operation cost, and improving the renewable energy consumption rate. This strategy provides an innovative solution for the optimal allocation of the grid-side renewable energy storage system and is of great significance for ensuring the stable operation of the power grid.

Key words: energy storage system, collaborative optimization, renewable energy integration, deep reinforcement learning, model predictive control, adaptive control

中图分类号:

TM 73

谭金龙, 陈军, 赵启, 崔大林, 刘永强, 张路. 基于改进型双重深度确定性策略梯度与自适应分布式模型预测控制融合的电网侧储能系统协同优化方法[J]. 储能科学与技术, 2025, 14(11): 4289-4299.

Jinlong TAN, Jun CHEN, Qi ZHAO, Dalin CUI, Yongqiang LIU, Lu ZHANG. A grid-side energy storage system optimization method based on improved twin deep deterministic policy gradient and adaptive distributed model predictive control[J]. Energy Storage Science and Technology, 2025, 14(11): 4289-4299.

图/表 6

图1

表1

图2

图3

图4

图5

参考文献 22

[1]	付诗意, 吕桃林, 闵凡奇, 等. 电动汽车用锂离子电池SOC估算方法综述[J]. 储能科学与技术, 2021, 10(3): 1127-1136. DOI: 10.19799/j.cnki.2095-4239.2021.0013.
	FU S Y, LYU T L, MIN F Q, et al. Review of estimation methods on SOC of lithium-ion batteries in electric vehicles[J]. Energy Storage Science and Technology, 2021, 10(3): 1127-1136. DOI: 10.19799/j.cnki.2095-4239.2021.0013.
[2]	李咸善, 方子健, 李飞, 等. 含多微电网租赁共享储能的配电网博弈优化调度[J]. 中国电机工程学报, 2022, 42(18): 6611-6625. DOI: 10.13334/j.0258-8013.pcsee.211424.
	LI X S, FANG Z J, LI F, et al. Game-based optimal dispatching strategy for distribution network with multiple microgrids leasing shared energy storage[J]. Proceedings of the CSEE, 2022, 42(18): 6611-6625. DOI: 10.13334/j.0258-8013.pcsee.211424.
[3]	陆秋瑜, 杨银国, 谢平平, 等. 风电场集群租赁共享储能两阶段优化运行策略[J]. 电网技术, 2024, 48(3): 1146-1165. DOI: 10.13335/j.1000-3673.pst.2023.0104.
	LU Q Y, YANG Y G, XIE P P, et al. Two-stage optimal operation strategy of wind farm cluster leasing shared energy storage[J]. Power System Technology, 2024, 48(3): 1146-1165. DOI: 10. 13335/j.1000-3673.pst.2023.0104.
[4]	ZHANG T, WU Y N. Collaborative allocation model and balanced interaction strategy of multi flexible resources in the new power system based on Stackelberg game theory[J]. Renewable Energy, 2024, 220: 119714. DOI: 10.1016/j.renene.2023.119714.
[5]	YAN H R, HOU H J, DENG M, et al. Stackelberg game theory based model to guide users' energy use behavior, with the consideration of flexible resources and consumer psychology, for an integrated energy system[J]. Energy, 2024, 288: 129806. DOI: 10.1016/j.energy.2023.129806.
[6]	REHMAN N U, UZAIR M, ALLAUDDIN U. An optical-energy model for optimizing the geometrical layout of solar photovoltaic arrays in a constrained field[J]. Renewable Energy, 2020, 149: 55-65. DOI: 10.1016/j.renene.2019.12.040.
[7]	王书峰, 钟明, 许贤泽, 等. 计及需求侧管理的新能源微电网多目标优化调度方法[J]. 智慧电力, 2022, 50(12): 55-62.
	WANG S F, ZHONG M, XU X Z, et al. Multi-objective optimal scheduling method for renewable energy microgrid considering demand side management[J]. Smart Power, 2022, 50(12): 55-62.
[8]	庄怀东, 吴红斌, 刘海涛, 等. 含电动汽车的微网系统多目标经济调度[J]. 电工技术学报, 2014, 29(S1): 365-373. DOI: 10.19595/j.cnki.1000-6753.tces.2014.s1.051.
	ZHUANG H D, WU H B, LIU H T, et al. Multi-objective economic dispatch of microgrid system with electric vehicle[J]. Transactions of China Electrotechnical Society, 2014, 29(S1): 365-373. DOI: 10.19595/j.cnki.1000-6753.tces.2014.s1.051.
[9]	ZHAO Z L, GUO J T, LUO X, et al. Distributed robust model predictive control-based energy management strategy for islanded multi-microgrids considering uncertainty[J]. IEEE Transactions on Smart Grid, 2022, 13(3): 2107-2120. DOI: 10. 1109/TSG.2022.3147370.
[10]	胡戎, 邱晓燕, 张志荣. 计及灵活性资源的交直流混合配电网双层优化[J]. 电网技术, 2022, 46(6): 2259-2268. DOI: 10.13335/j.1000-3673.pst.2021.1036.
	HU R, QIU X Y, ZHANG Z R. Bi-level optimization of AC/DC hybrid distribution network considering flexible resources[J]. Power System Technology, 2022, 46(6): 2259-2268. DOI: 10.13335/j.1000-3673.pst.2021.1036.
[11]	陈铭, 梅诗妍. 基于风险分析和模糊专家系统的电力工程应急成本估算方法[J]. 沈阳工业大学学报,2025, 47(3): 281-287. DOI: 10.7688/j.issn.1000-1646.2025.03.02.
	CHEN M, MEI S Y. Emergency cost estimation method for power engineering based on risk analysis and fuzzy expert system[J]. Journal of Shenyang University of Technology, 2025, 47(3): 281-287. DOI: 10.7688/j.issn.1000-1646.2025.03.02.
[12]	冉金周, 李华强, 李彦君, 等. 考虑灵活性供需匹配的孤岛微网优化调度策略[J]. 太阳能学报, 2022, 43(5): 36-44. DOI: 10.19912/j.0254-0096.tynxb.2020-0938.
	RAN J Z, LI H Q, LI Y J, et al. Optimal scheduling of isolated microgrid considering flexible power supply and demand[J]. Acta Energiae Solaris Sinica, 2022, 43(5): 36-44. DOI: 10.19912/j.0254-0096.tynxb.2020-0938.
[13]	刘英琪, 谢敏, 韦薇, 等. 高比例风电接入的电力系统灵活性评估与优化[J]. 电力建设, 2019, 40(9): 1-10.
	LIU Y Q, XIE M, WEI W, et al. Assessment and optimization for power system flexibility with high proportion of wind power[J]. Electric Power Construction, 2019, 40(9): 1-10.
[14]	普智勇, 张冬蕊, 尹学卫, 等. 不同储能对多时间尺度风电消纳特性对比研究[J]. 节能技术, 2020, 38(4): 371-374, 378.
	PU Z Y, ZHANG D R, YIN X W, et al. Comparative study of different energy storage systems on multi-timescale wind power smoothing performance[J]. Energy Conservation Technology, 2020, 38(4): 371-374, 378.
[15]	赵冬梅, 徐辰宇, 陶然, 等. 多元分布式储能在新型电力系统配电侧的灵活调控研究综述[J]. 中国电机工程学报, 2023, 43(5): 1776-1799. DOI: 10.13334/j.0258-8013.pcsee.220778.
	ZHAO D M, XU C Y, TAO R, et al. Review on flexible regulation of multiple distributed energy storage in distribution side of new power system[J]. Proceedings of the CSEE, 2023, 43(5): 1776-1799. DOI: 10.13334/j.0258-8013.pcsee.220778.
[16]	胡泽春, 夏睿, 吴林林, 等. 考虑储能参与调频的风储联合运行优化策略[J]. 电网技术, 2016, 40(8): 2251-2257. DOI: 10.13335/j.1000-3673.pst.2016.08.001.
	HU Z C, XIA R, WU L L, et al. Joint operation optimization of wind-storage union with energy storage participating frequency regulation[J]. Power System Technology, 2016, 40(8): 2251-2257. DOI: 10.13335/j.1000-3673.pst.2016.08.001.
[17]	李翠萍, 司文博, 李军徽, 等. 基于集合经验模态分解和多目标遗传算法的火-多储系统调频功率双层优化[J]. 电工技术学报, 2024, 39(7): 2017-2032. DOI: 10.19595/j.cnki.1000-6753.tces.230186.
	LI C P, SI W B, LI J H, et al. Two-layer optimization of frequency modulated power of thermal generation and multi-storage system based on ensemble empirical mode decomposition and multi-objective genetic algorithm[J]. Transactions of China Electrotechnical Society, 2024, 39(7): 2017-2032. DOI: 10.19595/j.cnki.1000-6753.tces.230186.
[18]	付慧敏, 郑刚. 基于数字孪生的配电网智能化故障诊断方法 [J]. 沈阳工业大学学报, 2025, 47 (3): 288-294. DOI: 10.7688/j.issn.1000-1646.2025.03.03.
	FU H M, ZHENG G. Intelligent fault diagnosis method for distribution network based on digital twin [J]. Journal of Shenyang University of Technology, 2025, 47 (3): 288-294. DOI: 10.7688/j.issn.1000-1646.2025.03.03.
[19]	孙辉, 范轩轩, 胡姝博, 等. 虚拟电厂参与日前电力市场的内外协调竞标策略[J]. 电网技术, 2022, 46(4): 1248-1262. DOI: 10.13335/j.1000-3673.pst.2021.1819.
	SUN H, FAN X X, HU S B, et al. Internal and external coordination bidding strategy of virtual power plant participating in day-ahead power market[J]. Power System Technology, 2022, 46(4): 1248-1262. DOI: 10.13335/j.1000-3673.pst.2021.1819.
[20]	买买提.努尔, 张艳丽, 魏文倩, 等. 基于短路比指标的新能源并网容量极限计算[J]. 沈阳工业大学学报, 2024, 46 (6): 742-747. DOI: 10.7688/j.issn.1000-1646.2024.06.02.
	MAIMAITI N, ZHANG Y L, WEI W Q, et al. Calculation of new energy grid-connected capacity limit based on short-circuit ratio index [J]. Journal of Shenyang University of Technology, 2024, 46 (6): 742-747. DOI: 10.7688/j.issn.1000-1646.2024.06.02.
[21]	李建林, 郭兆东, 曾伟, 等. 面向调频的锂电池储能建模及仿真分析[J]. 电力系统保护与控制, 2022, 50(13): 33-42. DOI: 10.19783/j.cnki.pspc.226227.
	LI J L, GUO Z D, ZENG W, et al. Modeling and simulation analysis of lithium battery energy storage oriented to frequency modulation[J]. Power System Protection and Control, 2022, 50(13): 33-42. DOI: 10.19783/j.cnki.pspc.226227.
[22]	谭必蓉, 杜建华, 叶祥虎, 等. 基于模型的锂离子电池SOC估计方法综述[J]. 储能科学与技术, 2023, 12(6): 1995-2010. DOI: 10.19799/j.cnki.2095-4239.2023.0016.
	TAN B R, DU J H, YE X H, et al. Overview of SOC estimation methods for lithium-ion batteries based on model[J]. Energy Storage Science and Technology, 2023, 12(6): 1995-2010. DOI: 10.19799/j.cnki.2095-4239.2023.0016.

内容	参数	数值
储能系统	充放电效率	90%
负荷模型	峰值负荷	300 MW
负荷模型	负荷预测误差	±5%
增强型DDPG	学习率	0.001
增强型DDPG	折现系数	0.95
自适应DMPC	子问题数量	10
自适应DMPC	迭代次数	100

基于改进型双重深度确定性策略梯度与自适应分布式模型预测控制融合的电网侧储能系统协同优化方法

A grid-side energy storage system optimization method based on improved twin deep deterministic policy gradient and adaptive distributed model predictive control

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 22

相关文章 15

编辑推荐

Metrics

本文评价

[1]	王凯亮, 孙宇军, 钟锦星, 苏向阳, 李俊辉, 刘宗扬, 蔡煜, 陈艺丹. 考虑储能寿命和经验模态分解的区域配电网混合储能配置[J]. 储能科学与技术, 2025, 14(9): 3417-3430.
[2]	杨优军, 马建, 孙杰. 基于大数据与人工智能的光伏储能系统智能运维研究[J]. 储能科学与技术, 2025, 14(9): 3414-3416.
[3]	张磊. 锂离子电池储能电站的运行状态监测与评估[J]. 储能科学与技术, 2025, 14(9): 3538-3540.
[4]	张帅, 张涛, 裴玮, 马腾飞, 肖浩, 施婕, 何传鑫. 基于产消者随机博弈决策的共享储能协同优化[J]. 储能科学与技术, 2025, 14(8): 3216-3228.
[5]	马骞, 肖亮, 程冰, 高琴, 刘春晓, 朱益华, 李成翔. 基于强化学习-模型预测控制（RL-MPC）的分布式储能协同一次调频控制方法[J]. 储能科学与技术, 2025, 14(8): 3138-3148.
[6]	徐鹤勇, 郑铁军, 丁圣权, 蒙飞, 张越, 杨家麒. 基于量子增强混合时空图神经网络的混合储能系统自适应频率调节方法[J]. 储能科学与技术, 2025, 14(8): 3149-3159.
[7]	徐成善, 孙烨, 杨智凯, 赵明强, 李亚伦, 冯旭宁, 王贺武, 卢兰光, 欧阳明高. 储能锂离子电池系统热失控诱发电弧研究进展[J]. 储能科学与技术, 2025, 14(8): 3037-3050.
[8]	苏新凯, 赵璐璐, 陈彦桥, 王础, 陈换军, 金翼. 超级电容产业化研究与应用综述[J]. 储能科学与技术, 2025, 14(8): 2994-3003.
[9]	王子滔, 李浩然. 考虑分布式储能功率支撑的配电网供电恢复频率控制方法[J]. 储能科学与技术, 2025, 14(7): 2738-2751.
[10]	胡力月, 黄威, 周云, 周英强, 邵常政, 王柯. 基于模糊推理的储能系统锂离子电池模组热扩散概率评估方法[J]. 储能科学与技术, 2025, 14(7): 2662-2674.
[11]	李源, 赵明智, 徐玉杰, 蔡杰. 液态二氧化碳储能系统变工况运行特性[J]. 储能科学与技术, 2025, 14(7): 2761-2771.
[12]	张晓慧, 杨瑞赓, 焦松坤. 基于机器学习的储能系统容量规划与需求预测研究[J]. 储能科学与技术, 2025, 14(7): 2881-2883.
[13]	陈勋. 基于深度强化学习的储能系统能量管理与优化调度策略[J]. 储能科学与技术, 2025, 14(6): 2439-2441.
[14]	肖俊阳, 罗金阁, 马伟哲, 程武平, 曾通. 基于改进人工蜂群算法的配电系统储能优化控制策略[J]. 储能科学与技术, 2025, 14(6): 2567-2574.
[15]	许庆祥, 滕伟, 秦润, 宋顺一, 柳亦兵, 梁双印. 电网调频飞轮储能系统并网能量管理与控制策略[J]. 储能科学与技术, 2025, 14(5): 2013-2022.