考虑冷热电互补及储能系统的多园区综合能源系统协调优化调度

doi:10.19799/j.cnki.2095-4239.2021.0631

摘要/Abstract

摘要：

新能源出力和用户负荷的随机性使得综合能源系统的优化调度问题充满挑战，本工作提出一种考虑冷热电互补及储能系统的多园区综合能源协调优化调度方法。首先，引入虚拟电厂技术，将综合能源系统内部资源聚合，考虑园区内的能量储存装置以及园区间的能量传输装置，构建虚拟电厂框架下的多园区综合能源系统结构；其次，建立热电联供单元、储能系统、冷热管道等内部资源模型，以最大化系统收益为目标，建立多虚拟电厂综合能源系统协调调度模型；最后，构建深度强化学习框架，再将Q值迁移方法引入到深度强化学习算法中，基于改进的深度确定性策略梯度算法进行连续状态和动作空间下的优化调度决策。通过算例分析，验证了所提方法的有效性，结果表明基于改进算法的多园区综合能源系统协调优化调度能够有效实现资源的合理分配以及各个园区间的供能互补，降低了系统运行成本。

关键词: 多园区综合能源系统, 储能系统, Q值迁移, 深度强化学习, 优化调度

Abstract:

The randomness of new energy output and user load makes the optimal dispatching of the integrated energy system full of challenges, and a multi-park integrated energy coordinated optimal dispatching method considering the complementary cooling, heating and power and energy storage systems is proposed. First, the virtual power plant technology was introduced to aggregate the internal resources of the integrated energy system, and the energy storage devices in the park and the energy transmission devices between the parks were considered, and a multi-park integrated energy system structure under the framework of the virtual power plant was constructed; secondly, the thermal power system was established. Internal resource models such as co-supply units, energy storage systems, and cooling and heating pipelines, with the goal of maximizing system revenue, established a coordinated scheduling model for a multi-virtual power plant integrated energy system; finally, a deep reinforcement learning framework was built, and the Q value was migrated The method is introduced into the deep reinforcement learning algorithm. Based on the improved deep deterministic policy gradient algorithm, the optimal scheduling decision is made in the continuous state and action space. The effectiveness of the proposed method is verified through the analysis of calculation examples, and the results show that the coordinated and optimized dispatch of the multi-park integrated energy system based on the improved algorithm can effectively realize the reasonable allocation of resources and the complementary energy supply between the parks, and reduce the operating cost of the system.

Key words: multi-park integrated energy system, energy storage system, Q value migration, deep reinforcement learning, optimal scheduling

中图分类号:

TM 73

李昊, 刘畅, 苗博, 张静. 考虑冷热电互补及储能系统的多园区综合能源系统协调优化调度[J]. 储能科学与技术, 2022, 11(5): 1482-1491.

Hao LI, Chang LIU, Bo MIAO, Jing ZHANG. Coordinative optimal dispatch of multi-park integrated energy system considering complementary cooling， heating and power and energy storage systems[J]. Energy Storage Science and Technology, 2022, 11(5): 1482-1491.

图/表 14

图1

图2

图3

图4

图5

图6

图7

表1

表2

表3

图8

表4

图9

表5

参考文献 14

1	YANG W T, LIU W J, CHUNG C Y, et al. Coordinated planning strategy for integrated energy systems in a district energy sector[J]. IEEE Transactions on Sustainable Energy, 2020, 11(3): 1807-1819.
2	韦古强, 胡从川, 刘乙学, 等. 基于液化空气储能的综合能源系统经济性分析[J]. 储能科学与技术, 2021, 10(6): 2403-2410.
	WEI G Q, HU C C, LIU Y X, et al. Economic analysis of integrated energy system based on liquid air energy storage[J]. Energy Storage Science and Technology, 2021, 10(6): 2403-2410.
3	刘涤尘, 马恒瑞, 王波, 等. 含冷热电联供及储能的区域综合能源系统运行优化[J]. 电力系统自动化, 2018, 42(4): 113-120, 141.
	LIU D C, MA H R, WANG B, et al. Operation optimization of regional integrated energy system with CCHP and energy storage system[J]. Automation of Electric Power Systems, 2018, 42(4): 113-120, 141.
4	周亦洲, 孙国强, 黄文进, 等. 多区域虚拟电厂综合能源协调调度优化模型[J]. 中国电机工程学报, 2017, 37(23): 6780-6790, 7069.
	ZHOU Y Z, SUN G Q, HUANG W J, et al. Optimized multi-regional integrated energy coordinated scheduling of a virtual power plant[J]. Proceedings of the CSEE, 2017, 37(23): 6780-6790, 7069.
5	FAN H, YUAN Q Q, CHENG H Z. Multi-objective stochastic optimal operation of a grid-connected microgrid considering an energy storage system[J]. Applied Sciences, 2018, 8(12): 2560.
6	MARTÍNEZ CESEÑA E A, MANCARELLA P. Energy systems integration in smart districts: Robust optimisation of multi-energy flows in integrated electricity, heat and gas networks[J]. IEEE Transactions on Smart Grid, 2018, 10(1): 1122-1131.
7	刘全, 翟建伟, 章宗长, 等. 深度强化学习综述[J]. 计算机学报, 2018, 41(1): 1-27.
	LIU Q, ZHAI J W, ZHANG Zongchang, et al. A survey on deep reinforcement learning[J]. Chinese Journal of Computers, 2018, 41(1): 1-27.
8	蔡新雷, 崔艳林, 董锴, 等. 基于改进K-means和MADDPG算法的风储联合系统日前优化调度方法[J]. 储能科学与技术, 2021, 10(6): 2200-2208.
	CAI X L, CUI Y L, DONG K, et al. Day-ahead optimal scheduling approach of wind-storage joint system based on improved K-means and MADDPG algorithm[J]. Energy Storage Science and Technology, 2021, 10(6): 2200-2208.
9	聂欢欢, 张家琦, 陈颖, 等. 基于双层强化学习方法的多能园区实时经济调度[J]. 电网技术, 2021, 45(4): 1330-1336.
	NIE H H, ZHANG J Q, CHEN Y, et al. Real-time economic dispatch of community integrated energy system based on a double-layer reinforcement learning method[J]. Power System Technology, 2021, 45(4): 1330-1336.
10	刘俊峰, 陈剑龙, 王晓生, 等. 基于深度强化学习的微能源网能量管理与优化策略研究[J]. 电网技术, 2020, 44(10): 3794-3803.
	LIU J F, CHEN J L, WANG X S, et al. Energy management and optimization of multi-energy grid based on deep reinforcement learning[J]. Power System Technology, 2020, 44(10): 3794-3803.
11	WANG Y L, WANG Y D, HUANG Y J, et al. Optimal scheduling of the regional integrated energy system considering economy and environment[J]. IEEE Transactions on Sustainable Energy, 2019, 10(4): 1939-1949.
12	LIN S L, SONG W J, FENG Z P, et al. Energy management strategy and capacity optimization for CCHP system integrated with electric-thermal hybrid energy storage system[J]. International Journal of Energy Research, 2020, 44(2): 1125-1139.
13	杨挺, 赵黎媛, 刘亚闯, 等. 基于深度强化学习的综合能源系统动态经济调度[J]. 电力系统自动化, 2021, 45(5): 39-47.
	YANG T, ZHAO L Y, LIU Y C, et al. Dynamic economic dispatch for integrated energy system based on deep reinforcement learning[J]. Automation of Electric Power Systems, 2021, 45(5): 39-47.
14	彭刘阳, 孙元章, 徐箭, 等. 基于深度强化学习的自适应不确定性经济调度[J]. 电力系统自动化, 2020, 44(9): 33-42.
	PENG L Y, SUN Y Z, XU J, et al. Self-adaptive uncertainty economic dispatch based on deep reinforcement learning[J]. Automation of Electric Power Systems, 2020, 44(9): 33-42.

设备类型	最小功率/kW	最大功率/kW
热电联供	0	600
燃气锅炉	0	800
余热回收装置	0	750
电锅炉	0	900
电制冷空调	0	1650
吸收式制冷机	0	600
蓄电池	-160	160
热储能	-230	230
热管道	0	600
冷管道	0	600

参数	数值	参数	数值
η^MT	0.35	η^ESC	0.95
α	1.20	η^ESD	0.95
η^HR	0.75	η^TSC	0.90
η^GB	0.75	η^TSD	0.90
η^EB	0.90	SOC^ES，MIN	0.20
η^EC	4.00	SOC^ES，MAX	0.80
η^AC	1.20	SOC^TS，MIN	0.00
η^TF	0.15	SOC^TS，MAX	1.00
η^CF	0.05

参数	数值
评论家网络学习率	0.001
演员网络学习率	0.0001
奖励折扣系数	0.9
经验池容量	30000
小批量采样规模	128
软更新系数	0.01

收益与成本/元	场景1			场景2
收益与成本/元	VPP1	VPP2	VPP3	VPP1	VPP2	VPP3
电力市场	-1005	-736	-677	-890	-627	-574
热电联供	1787	1347	1557	1894	1421	1605
燃气锅炉	348	276	328	269	236	256
中断负荷	466	355	401	466	355	401
总成本	3606	2714	2963	3519	2639	2836
总计	9283			8994

成本/元	QT-CDDPG	DDPG	DQN	遗传算法
最大值	10352	10635	11232	11952
最小值	8132	8214	8533	9071
平均值	9013	9201	9662	10210