Day-ahead optimal scheduling approach of wind-storage joint system based on improved K-means and MADDPG algorithm

doi:10.19799/j.cnki.2095-4239.2021.0282

Abstract

Abstract:

The joint operation of wind power and energy storage can effectively deal with the uncertainty of wind power output and improve the competitiveness of wind power. However, optimizing and dispatching the joint operation of energy storage and wind power is a major difficulty. A day-ahead optimal scheduling method of the wind storage joint system based on improved K-means and multi-agent deep deterministic strategy gradient (MADDPG) algorithm is proposed to maximize the benefits of a wind-storage joint operation by ensuring the adjustable capacity of energy storage. First, the improved K-Means clustering algorithm optimized by the firefly algorithm is used to achieve energy storage grouping; then, the wind power and the grouped energy storage equipment are modeled as different agents to form a multi-agent system. The MADDPG algorithm is used to solve the problem, and the MADDPG algorithm's state space, action space, and reward function are designed. Finally, a simulation example is used to validate the algorithm. The results show that, when compared to conventional deep reinforcement learning, the proposed scheduling strategy can better coordinate the operation of wind power and energy storage, effectively smooth the fluctuation of wind power output, and improve the operating income of the wind storage combined system.

Key words: wind power, energy storage system, K-means, MADDPG

CLC Number:

TM 73

Xinlei CAI, Yanlin Cui, Kai DONG, Zijie MENG, Yuan PAN, Zhenfan YU, Jixing WANG, Xiangzhan MENG, Yang YU. Day-ahead optimal scheduling approach of wind-storage joint system based on improved K-means and MADDPG algorithm[J]. Energy Storage Science and Technology, 2021, 10(6): 2200-2208.

Figures/Tables 14

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Table 2

Table 3

Fig. 5

Fig. 6

Fig. 7

Table 4

Fig. 8

Fig. 9

Table 5

References 14

1	孙欣, 方陈, 沈风, 等. 考虑风电出力不确定性的发用电机组组合方法[J]. 电工技术学报, 2017, 32(4): 204-211.
	SUN X, FANG C, SHEN F, et al. An integrated generation-consumption unit commitment model considering the uncertainty of wind power[J]. Transactions of China Electrotechnical Society, 2017, 32(4): 204-211.
2	谢应昭, 卢继平. 含风储混合系统的多目标机组组合优化模型及求解[J]. 电力自动化设备, 2015, 35(3): 18-26.
	XIE Y Z, LU J P. Multi-objective unit commitment optimization model including hybrid wind-storage system and its solution[J]. Electric Power Automation Equipment, 2015, 35(3): 18-26.
3	DING H J, HU Z C, SONG Y H. Stochastic optimization of the daily operation of wind farm and pumpedhydro-storage plant[J]. Renewable Energy, 2012, 48(6): 571-578.
4	熊猛, 高峰, 张海峰, 等. 基于混合储能系统的风电场调度策略[J]. 电网技术, 2014, 38(5): 1242-1249.
	XIONG M, GAO F, ZHANG H F, et al. Hybrid energy storage based generation scheduling of wind farms[J]. Power System Technology, 2014, 38(5): 1242-1249.
5	DUKPA A, DUGGAL I, VENKATESH B, et al. Optimal participation and risk mitigation of wind generators in an electricity market[J]. IET Renewable Power Generation, 2010, 4(2): 165-175.
6	DICORATO M, FORTE G, PISANI M, et al. Planning and operating combined wind-storage system in electricity market[J]. IEEE Transactions on Sustainable Energy, 2012, 3(2): 209-217.
7	余涛, 周斌, 甄卫国. 强化学习理论在电力系统中的应用及展望[J]. 电力系统保护与控制, 2009, 37(14): 122-128.
	YU T, ZHOU B, ZHEN W G. Application and development of reinforcement learning theory in power systems[J]. Power System Protection and Control, 2009, 37(14): 122-128.
8	杨挺, 赵黎媛, 刘亚闯, 等. 基于深度强化学习的综合能源系统动态经济调度[J]. 电力系统自动化, 2021, 45(5): 39-47.
	YANG T, ZHAO L Y, LIU Y C, et al. Dynamic eco-nomic dispatch for integrated energy system based on deep rein-forcement learning[J]. Automation of Electric Power Systems, 2021, 45(5): 39-47.
9	于一潇, 杨佳峻, 杨明, 等. 基于深度强化学习的风电场储能系统预测决策一体化调度[J]. 电力系统自动化, 2021, 45(1): 132-140.
	YU Y X, YANG J J, YANG M, et al. Prediction and decision integrated scheduling of energy storage system in wind farm based on deep reinforcement learning[J]. Automation of Electric Power Systems, 2021, 45(1): 132-140.
10	刘国静, 韩学山, 王尚, 等. 基于强化学习方法的风储合作决策[J]. 电网技术, 2016, 40(9): 2729-2736.
	LIU G J, HAN X S, WANG S, et al. Optimal decision-making in the cooperation of wind power and energy storage based on reinforcement learning algorithm[J]. Power System Technology, 2016, 40(9): 2729-2736.
11	汤文亮, 张平, 汤树芳. 基于精英反向学习的萤火虫K-means改进算法[J]. 计算机工程与设计, 2019, 40(11): 3164-3169.
	TANG W L, ZHANG P, TANG S F, et al. Improved firefly K-means algorithm based on elite opposition-based learning[J]. Computer Engineering and Design, 2019, 40(11): 3164-3169.
12	陈小雪, 尉永清, 任敏, 等. 基于萤火虫优化的加权K-means算法[J]. 计算机应用研究, 2018, 35(2): 466-470.
	CHEN X X, WEI Y Q, REN M, et al. Weighted K-means clustering algorithm based on firefly algorithm[J]. Application Research of Computers, 2018, 35(2): 466-470.
13	KIM B, PARK J, PARK S, et al. Impedance learning for robotic contact tasks using natural actor-critic algorithm[J]. IEEE Transactions on Systems, Man, and Cybernetics Part B, Cybernetics, 2010, 40(2): 433-443.
14	桂熙. 基于MADDPG算法的多个智能体协同控制[D]. 武汉: 武汉纺织大学, 2020.GUI X. Research on multi-agent cooperative control based on MADDPG Algorithm[D]. Wuhan: Wuhan Textile University, 2020.

时段	售电电价/[元/(kW·h)]	出力偏差惩罚/[元/(kW·h)]	储能越限惩罚/[元/(kW·h)]
1—7	0.2	0.3	0.5
8—10	0.5	0.75	0.5
11—15	0.8	1.2	0.5
16—18	0.5	0.75	0.5
19—21	0.8	1.2	0.5
22—24	0.5	0.75	0.5

结构参数	数值
每层神经元个数	128
Critic网络学习率	0.001
Actor网络学习率	0.0001
经验回放池规模	50000
采样规模	256
软更新系数	0.01

时段	ESS1		ESS2		ESS3
时段	电池组数	平均SOC	电池组数	平均SOC	电池组数	平均SOC
1	48	0.21	0	0.00	12	0.43
2	46	0.23	0	0.0	14	0.44
3	46	0.23	0	0.00	14	0.44
4	30	0.25	2	0.74	28	0.53
5	28	0.26	2	0.68	30	0.52
6	26	0.31	4	0.75	30	0.57
7	23	0.35	6	0.78	31	0.55
8	4	0.35	8	0.79	48	0.56
9	0	0.00	12	0.81	48	0.55
10	0	0.00	12	0.80	48	0.57
11	0	0.00	14	0.81	46	0.58
12	0	0.00	13	0.81	47	0.60
13	0	0.00	13	0.75	47	0.62
14	0	0.00	10	0.70	50	0.58
15	4	0.32	7	0.69	49	0.55
16	9	0.30	4	0.68	47	0.54
17	10	0.28	2	0.66	48	0.52
18	10	0.28	0	0.00	50	0.52
19	12	0.29	0	0.00	48	0.49
20	18	0.31	0	0.00	42	0.47
21	45	0.31	0	0.00	15	0.43
22	57	0.32	0	0.00	3	0.42
23	56	0.33	0	0.00	4	0.45
24	51	0.35	0	0.00	9	0.45

方案	总收益/千元	售电收益/千元	出力偏差惩罚/千元	储能越限惩罚/千元
方案1	62.50	84.85	21.50	0.85
方案2	69.01	85.22	15.66	0.55

[1]	Jianmin HAN, Feiyu XUE, Shuangyin LIANG, Tianshu QIAO. Hybrid energy storage system assisted frequency modulation simulation of the coal-fired unit under fuzzy control optimization [J]. Energy Storage Science and Technology, 2022, 11(7): 2188-2196.
[2]	Yuhan GUO, Dan YU, Peng YANG, Ziji WANG, Jintao WANG. Optimal capacity allocation method of a distributed energy storage system based on greedy algorithm [J]. Energy Storage Science and Technology, 2022, 11(7): 2295-2304.
[3]	Tian WU, Mincheng LIN, Hao HAI, Haiyu SUN, Zhaoyin WEN, Fuyuan MA. Development of high-power Ni-MH battery system for primary frequency modulation [J]. Energy Storage Science and Technology, 2022, 11(7): 2213-2221.
[4]	Jianlin LI, Zedong ZHANG, Yaxin LI, Yi ZHOU, Yunli YUE. Research on key technologies of mobile energy storage system under the target of carbon neutrality [J]. Energy Storage Science and Technology, 2022, 11(5): 1523-1536.
[5]	Feng TIAN, Zhijiang CHENG, Handi YANG, Tianxiang YANG. Fault-tolerant control strategy for modular multi-level hybrid converter battery energy storage system [J]. Energy Storage Science and Technology, 2022, 11(5): 1583-1591.
[6]	Hao LI, Chang LIU, Bo MIAO, Jing ZHANG. Coordinative optimal dispatch of multi-park integrated energy system considering complementary cooling， heating and power and energy storage systems [J]. Energy Storage Science and Technology, 2022, 11(5): 1482-1491.
[7]	Fulin FAN, Junhui LI, CAMPOS-GAONA David, Gangui YAN. Energy storage-friendly frequency response service markets： The UK perspective [J]. Energy Storage Science and Technology, 2022, 11(4): 1278-1288.
[8]	Bin GUO, Jie XING, Fei YAO, Xiaomin JING. Optimal configuration of user-side hybrid energy storage based on bi-level programming model [J]. Energy Storage Science and Technology, 2022, 11(2): 615-622.
[9]	Yulong CHEN, Xin WU, Wei TENG, Yibing LIU. Power coordinated control strategy of flywheel energy storage array for wind power smoothing [J]. Energy Storage Science and Technology, 2022, 11(2): 600-608.
[10]	Di LIU, Tiantian ZHANG, Yuwei PENG, Xiaomei TANG, Dan WANG, Chengxiong MAO. Shaft modeling and oscillation analysis for expansion process of compressed air energy storage system [J]. Energy Storage Science and Technology, 2022, 11(2): 563-572.
[11]	Xinlong ZHU, Junyi WANG, Jiashuang PAN, Chuanzhi KANG, Yitao ZOU, Kaijie YANG, Hong SHI. Present situation and development of thermal management system for battery energy storage system [J]. Energy Storage Science and Technology, 2022, 11(1): 107-118.
[12]	Weicheng SHEN, Wenxi ZHEN, Chong SHAO, Qi XIE. Coordinated fault ride through strategy for doubly fed induction generator using a superconducting magnetic energy storage system [J]. Energy Storage Science and Technology, 2022, 11(1): 136-146.
[13]	Liangbo QIAO, Xiaohu ZHANG, Xianzhong SUN, Xiong ZHANG, Yanwei MA. Advances in battery-supercapacitor hybrid energy storage system [J]. Energy Storage Science and Technology, 2022, 11(1): 98-106.
[14]	Shitan ZHANG, Shuai CHU, Weichun GE, Yinxuan LI, Chuang LIU. Evaluation method for the coordinated regulation of large-scale abandoned wind power and heat storage [J]. Energy Storage Science and Technology, 2022, 11(1): 283-290.
[15]	Xiaozhi GAO, Lei WANG, Jin TIAN, Jialu LIU, Qinghua LIU. Research on hybrid energy storage power distribution strategy based on parameter optimization variational mode decomposition [J]. Energy Storage Science and Technology, 2022, 11(1): 147-155.