The application of large language models in energy storage research

doi:10.19799/j.cnki.2095-4239.2024.0176

Abstract

Abstract:

In the pursuit of carbon neutrality, energy storage technology plays an increasingly crucial role in modern society. Addressing future challenges requires innovative methods in energy storage research, given its interdisciplinary and information-intensive nature. With the rapid development of artificial intelligence technology, large language models (LLMs) have achieved significant success in various domains, including text processing, information collection and integration, and picture and video generation. Moreover, the application of LLMs has extended to natural science research, demonstrating promising potential for improving research efficiency. Thus, LLMs are expected to assist in addressing future challenges in energy storage science and technology. This paper first focuses on ChatGPT and reviews AI advancements and LLMs, analyzing their impact on civil use and scientific research, particularly focusing on domestic LLMs. Subsequently, it discusses the basic concepts and fundamentals of LLMs and their application in energy storage, covering information processing, information generation, and system integration. Detailed examples are provided to illustrate the effectiveness of these new methods. Finally, this study outlines the remaining challenges and future development directions of the interdisciplinary nature of LLMs and energy storage.

Key words: large language model, artificial intelligence, energy storage technology, secondary battery

CLC Number:

TP 18

Yuhang YUAN, Yuchen GAO, Jundong ZHANG, Yanbin GAO, Chaolong WANG, Xiang CHEN, Qiang ZHANG. The application of large language models in energy storage research[J]. Energy Storage Science and Technology, 2024, 13(9): 2907-2919.

Figures/Tables 8

Fig. 1

Table 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

References 58

1	SILVER D, HUANG A, MADDISON C J, et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587): 484-489.
2	SILVER D, SCHRITTWIESER J, SIMONYAN K, et al. Mastering the game of Go without human knowledge[J]. Nature, 2017, 550(7676): 354-359.
3	JUMPER J, EVANS R, PRITZEL A, et al. Highly accurate protein structure prediction with AlphaFold[J]. Nature, 2021, 596(7873): 583-589.
4	SENIOR A W, EVANS R, JUMPER J, et al. Improved protein structure prediction using potentials from deep learning[J]. Nature, 2020, 577(7792): 706-710.
5	BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. December 6-12, 2020, Vancouver, BC, Canada. ACM, 2020: 1877-1901.
6	ELOUNDOU T, MANNING S, MISHKIN P, et al. GPTs are GPTs: An early look at the labor market impact potential of large language models[EB/OL]. 2023: arXiv: 2303.10130.[2023-12-05]. http://arxiv.org/abs/2303.10130
7	ZHAO W X, ZHOU K, LI J Y, et al. A survey of large language models[EB/OL]. 2023: arXiv: 2303.18223.[2023-12-05]. http://arxiv.org/abs/2303.18223
8	WU T Y, HE S Z, LIU J P, et al. A brief overview of ChatGPT: The history, status quo and potential future development[J]. IEEE/CAA Journal of Automatica Sinica, 2023, 10(5): 1122-1136.
9	ZHANG D, LIU X, ZHANG X Y, et al. DPA-2: Towards a universal large atomic model for molecular and material simulation[EB/OL]. 2023: arXiv: 2312.15492.[2023-12-05]. http://arxiv.org/abs/2312.15492
10	LIAO C, YU Y M, MEI Y, et al. From words to molecules: A survey of large language models in chemistry[EB/OL]. 2024: arXiv: 2402.01439. http://arxiv.org/abs/2402.01439
11	WHITE A D. The future of chemistry is language[J]. Nature Reviews Chemistry, 2023, 7(7): 457-458.
12	ALAWIDA M, MEJRI S, MEHMOOD A, et al. A comprehensive study of ChatGPT: Advancements, limitations, and ethical considerations in natural language processing and cybersecurity[J]. Information, 2023, 14(8): 462.
13	BISHOP C M. Neural networks and their applications[J]. Review of Scientific Instruments, 1994, 65(6): 1803-1832.
14	GONG S, WANG S, ZHU T S, et al. Multimodal machine learning for materials science: Composition-structure bimodal learning for experimentally measured properties[EB/OL]. 2023: arXiv: 2309.04478. [2023-12-05].http://arxiv.org/abs/2309.04478
15	LIU X Y, PENG H J, LI B Q, et al. Untangling degradation chemistries of lithium-sulfur batteries through interpretable hybrid machine learning[J]. Angewandte Chemie, 2022, 61(48): e202214037.
16	HAN C, GAO Y C, CHEN X, et al. A self-adaptive, data-driven method to predict the cycling life of lithium-ion batteries[J]. InfoMat, 2024: e12521.
17	STUART J. RUSSELL P N. Artificial intelligence: A modern approach, Third Edition [M]. Beijing: Tsinghua University Press, 2013.
18	GOODFELLOW I, BENGIO Y, COURVILLE A. Deep learning [M]. Beijing: Posts & Telecom Press, 2017.
19	CASTRO NASCIMENTO C M, PIMENTEL A S. Do large language models understand chemistry? A conversation with ChatGPT[J]. Journal of Chemical Information and Modeling, 2023, 63(6): 1649-1655.
20	吴思远, 王雪龙, 肖睿娟, 等. 基于大型语言模型的工具对电池研究的机遇与挑战[J]. 储能科学与技术, 2023, 12(3): 992-997.
	WU S Y, WANG X L, XIAO R J, et al. Problem and perspective for battery researcher based on large language model[J]. Energy Storage Science and Technology, 2023, 12(3): 992-997.
21	ZHENG Z L, ZHANG O F, BORGS C, et al. ChatGPT chemistry assistant for text mining and the prediction of MOF synthesis[J]. Journal of the American Chemical Society, 2023, 145(32): 18048-18062.
22	LIU Y X, SHI K J, HE K S, et al. On learning to summarize with large language models as references[EB/OL]. 2023: arXiv: 2305.14239.[2023-12-05]. http://arxiv.org/abs/2305.14239
23	XIE T, WAN Y W, HUANG W, et al. Large language models as master key: Unlocking the secrets of materials science with GPT[EB/OL]. 2023: arXiv: 2304.02213.[2023-12-05]. http://arxiv.org/abs/2304.02213
24	LI J T, LIU Y Q, FAN W Q, et al. Empowering molecule discovery for molecule-caption translation with large language models: A ChatGPT perspective[EB/OL]. 2023: arXiv: 2306.06615.[2023-12-05]. http://arxiv.org/abs/2306.06615
25	KUZMAN T, MOZETIČ I, LJUBEŠIĆ N. Automatic genre identification for robust enrichment of massive text collections: Investigation of classification methods in the era of large language models[J]. Machine Learning and Knowledge Extraction, 2023, 5(3): 1149-1175.
26	ALSHAMI A, ELSAYED M, ALI E, et al. Harnessing the power of ChatGPT for automating systematic review process: Methodology, case study, limitations, and future directions[J]. Systems, 2023, 11(7): 351.
27	THIK J, WANG S W, WANG C H, et al. Realizing the cooking recipe of materials synthesis through large language models[J]. Journal of Materials Chemistry A, 2023, 11(47): 25849-25853.
28	TANG L Y, SUN Z Y, IDNAY B, et al. Evaluating large language models on medical evidence summarization[J]. NPJ Digital Medicine, 2023, 6(1): 158.
29	FARROKHNIA M, BANIHASHEM S K, NOROOZI O, et al. A SWOT analysis of ChatGPT: Implications for educational practice and research[J]. Innovations in Education and Teaching International, 2023: 1-15.
30	HARRER S. Attention is not all you need: The complicated case of ethically using large language models in healthcare and medicine[J]. EBioMedicine, 2023, 90: 104512.
31	BISWAS S, LOGAN N S, DAVIES L N, et al. Assessing the utility of ChatGPT as an artificial intelligence-based large language model for information to answer questions on myopia[J]. Ophthalmic & Physiological Optics, 2023, 43(6): 1562-1570.
32	DWIVEDI Y K, KSHETRI N, HUGHES L, et al. Opinion Paper: "So what if ChatGPT wrote it?" Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy[J]. International Journal of Information Management, 2023, 71: 102642.
33	YU-HAN C, CHUN-CHING C. Investigating the impact of generative artificial intelligence on brainstorming: A preliminary study[C]//2023 International Conference on Consumer Electronics-Taiwan (ICCE-Taiwan). PingTung, Taiwan, China. IEEE, 2023: 193-194.
34	SUMMERS-STAY D, VOSS C R, LUKIN S M. Brainstorm, then select: A generative language model improves its creativity score[C]. In: The AAAI-23 Workshop on Creative AI Across Modalities, F, 2023.
35	ALZAABI A, ALAMRI A, ALBALUSHI H, et al. ChatGPT applications in academic research: A review of benefits, concerns, and recommendations [J]. bioRxiv, 2023, 2023. 08.17.553688.
36	LUND B D, WANG T, MANNURU N R, et al. ChatGPT and a new academic reality: Artificial Intelligence-written research papers and the ethics of the large language models in scholarly publishing[J]. Journal of the Association for Information Science and Technology, 2023, 74(5): 570-581.
37	COTTON D R E, COTTON P A, SHIPWAY J R. Chatting and cheating: Ensuring academic integrity in the era of ChatGPT[J]. Innovations in Education and Teaching International, 2024, 61(2): 228-239.
38	CHEN M, TWOREK J, JUN H, et al. Evaluating large language models trained on code[EB/OL]. 2021: arXiv: 2107.03374.[2023-12-05]. http://arxiv.org/abs/2107.03374
39	FENG Y H, VANAM S, CHERUKUPALLY M, et al. Investigating code generation performance of ChatGPT with crowdsourcing social data[C]//2023 IEEE 47th Annual Computers, Software, and Applications Conference (COMPSAC). Torino, Italy. IEEE, 2023: 876-885.
40	SARSA S, DENNY P, HELLAS A, et al. Automatic generation of programming exercises and code explanations using large language models[C]//Proceedings of the 2022 ACM Conference on International Computing Education Research-Volume 1. August 7-11, 2022, Lugano and Virtual Event, Switzerland. ACM, 2022: 27-43.
41	BURGER B, MAFFETTONE P M, GUSEV V V, et al. A mobile robotic chemist[J]. Nature, 2020, 583(7815): 237-241.
42	ZHU Q, ZHANG F, HUANG Y, et al. An all-round AI-Chemist with a scientific mind[J]. National Science Review, 2022, 9(10): nwac190.
43	SZYMANSKI N J, RENDY B, FEI Y X, et al. An autonomous laboratory for the accelerated synthesis of novel materials[J]. Nature, 2023, 624(7990): 86-91.
44	BOIKO D A, MACKNIGHT R, KLINE B, et al. Autonomous chemical research with large language models[J]. Nature, 2023, 624(7992): 570-578.
45	BRAN A M, COX S, SCHILTER O, et al. ChemCrow: Augmenting large-language models with chemistry tools[EB/OL]. 2023: arXiv: 2304.05376.[2023-12-05]. http://arxiv.org/abs/2304.05376
46	MERCHANT A, BATZNER S, SCHOENHOLZ S S, et al. Scaling deep learning for materials discovery[J]. Nature, 2023, 624(7990): 80-85.
47	RUAN J Q, LIANG G Q, ZHAO H, et al. Applying large language models to power systems: Potential security threats[J]. IEEE Transactions on Smart Grid, 2023, 15: 3333-3336.
48	BONADIA R S, TRINDADE F C L, FREITAS W, et al. On the potential of ChatGPT to generate distribution systems for load flow studies using OpenDSS[J]. IEEE Transactions on Power Systems, 2023, 38(6): 5965-5968.
49	ZENG R, QU L, GAO F, et al. Development status and prospects of the Energy Internet [J]. Scientia Sinica Informationis, 2017, 47(2): 149-170.
50	曹祎, 张莉, 郭静, 等. 基于大语言模型的低碳电力市场发展应用前景[J]. 智慧电力, 2024, 52(2): 8-16.
	CAO Y, ZHANG L, GUO J, et al. Prospects for development of low-carbon electricity markets based on large language models[J]. Smart Power, 2024, 52(2): 8-16.
51	康重庆, 刘静琨, 张宁. 未来电力系统储能的新形态: 云储能[J]. 电力系统自动化, 2017, 41(21): 2-8, 16.
	KANG C Q, LIU J K, ZHANG N. A new form of energy storage in future power system: Cloud energy storage[J]. Automation of Electric Power Systems, 2017, 41(21): 2-8, 16.
52	KIM S C, OYAKHIRE S T, ATHANITIS C, et al. Data-driven electrolyte design for lithium metal anodes[J]. Proceedings of the National Academy of Sciences of the United States of America, 2023, 120(10): e2214357120.
53	赖根明, 焦君宇, 蒋耀, 等. 机器学习原子势在锂金属负极中的应用[J]. 硅酸盐学报, 2023, 51(2): 469-475.
	LAI G M, JIAO J Y, JIANG Y, et al. Analysis of Li metal anode by machine learning potential[J]. Journal of the Chinese Ceramic Society, 2023, 51(2): 469-475.
54	罗啸山, 王振雨, 高朋越, 等. 机器学习加速理论晶体结构预测研究进展[J]. 硅酸盐学报, 2023, 51(2): 552-560.
	LUO X S, WANG Z Y, GAO P Y, et al. Review on machine learning accelerated crystal structure prediction[J]. Journal of the Chinese Ceramic Society, 2023, 51(2): 552-560.
55	YAO N, CHEN X, FU Z H, et al. Applying classical, Ab initio, and machine-learning molecular dynamics simulations to the liquid electrolyte for rechargeable batteries[J]. Chemical Reviews, 2022, 122(12): 10970-11021.
56	FU Z H, CHEN X, YAO N, et al. The chemical origin of temperature-dependent lithium-ion concerted diffusion in sulfide solid electrolyte Li₁₀GeP₂S₁₂[J]. Journal of Energy Chemistry, 2022, 70: 59-66.
57	GAO Y C, YAO N, CHEN X, et al. Data-driven insight into the reductive stability of ion-solvent complexes in lithium battery electrolytes[J]. Journal of the American Chemical Society, 2023, 145(43): 23764-23770.
58	HONG Z J. ChatGPT for computational materials science: A perspective[J]. Energy Material Advances, 2023, 4: 0026.

比较项目	GPT-1	GPT-2	GPT-3	GPT-4
发布时间	2018年6月	2019年2月	2020年5月	2023年3月
参数数量	1.17亿	15亿	1750亿	—
解码层层数	12层	48层	96层	128层
隐藏层层数	768层	1600层	12888层	20480层
上下文窗口	512个令牌	1024个令牌	2049个令牌	8192个令牌
预训练数据大小	约5 GB	约40 GB	约45 TB	未公开
数据来源	BooksCorpus, Wikipedia	WebText	Common Crawl, etc.	未公开
文本总结能力	不支持	不支持	支持	加强
学习目标	无监督学习	多任务学习	上下文学习	多模态学习

[1]	Jiahui HUANG, Zhufang KUANG. The forefront of the integration of artificial intelligence and energy storage technologies [J]. Energy Storage Science and Technology, 2024, 13(9): 3161-3181.
[2]	Junyu JIAO, Quanquan ZHANG, Ningbo CHEN, Jiyu WANG, Qiudi LU, Haohao DING, Peng PENG, Xiaohe SONG, Fan ZHANG, Jiaxin ZHENG. Development and applications of an intelligent big data analysis platform for batteries [J]. Energy Storage Science and Technology, 2024, 13(9): 3198-3213.
[3]	Jing XU, Yuqi WANG, Xiao FU, Qifan YANG, Jingchen LIAN, Liqi WANG, Ruijuan XIAO. Discovery of new battery materials based on a big data approach [J]. Energy Storage Science and Technology, 2024, 13(9): 2920-2932.
[4]	Yi ZHONG, Yan LENG, Sihui CHEN, Peiyi LI, Zhi ZOU, Yang LIU, Jiayu WAN. Accelerating battery research with retrieval-augmented large language models: Present and future [J]. Energy Storage Science and Technology, 2024, 13(9): 3214-3225.
[5]	Ziyu LIU, Zekun JIANG, Wei QIU, Quan XU, Yingchun NIU, Chunming XU, Tianhang ZHOU. Application of artificial intelligence in long-duration redox flow batteries storage systems [J]. Energy Storage Science and Technology, 2024, 13(9): 2871-2883.
[6]	Yingying XIE, Bin DENG, Yuzhi ZHANG, Xiaoxu WANG, Linfeng ZHANG. Intelligent R&D of battery design automation in the era of artificial intelligence [J]. Energy Storage Science and Technology, 2024, 13(9): 3182-3197.
[7]	Wentao ZHU, Yang ZHOU, Yimin XU, Tao SHI. Application and optimization of battery energy storage technology in new energy generation system [J]. Energy Storage Science and Technology, 2024, 13(8): 2737-2739.
[8]	Chu ZHANG, Dongcai CHEN, Xiangping CHEN, Yongxiang CAI. Economic benefit analysis of optimal allocation of energy storage in multiple application scenarios [J]. Energy Storage Science and Technology, 2024, 13(6): 2078-2088.
[9]	Jian SUN, Jianlong TAO, Yunrong HU, Xiaolong CAO, Yongping YANG. Summary of research on power storage technology based on heat pump at home and abroad [J]. Energy Storage Science and Technology, 2024, 13(6): 1963-1976.
[10]	Li ZHOU, Yan LIU. Application and development of alloy materials in energy storage technology [J]. Energy Storage Science and Technology, 2024, 13(6): 1874-1876.
[11]	Zhenxin SUN, Zhiming ZHANG, Fubo MA, Congjin JIANG, Haoyi DU, Huanjun CHEN, Yukui ZHANG. Investigation of energy regulation performance based on entropy theory [J]. Energy Storage Science and Technology, 2024, 13(5): 1584-1591.
[12]	Tianchen LI, Jianzheng YIN, Dawei ZHANG, Xiaoheng LIU. Research on renewable energy grid integration strategy based on hydropower station energy storage technology [J]. Energy Storage Science and Technology, 2024, 13(2): 677-679.
[13]	Su YAN, Fangfang ZHONG, Junwei LIU, Mei DING, Chuankun JIA. Key materials and advanced characterization of high-energy-density flow battery [J]. Energy Storage Science and Technology, 2024, 13(1): 143-156.
[14]	Jiajun ZHANG, Xiaoqiong LI, Zhentao ZHANG, Jiahao HAO, Pingyang ZHENG, Ze YU, Junling YANG, Yanan JING, Yunkai YUE. Research progress of compressed carbon dioxide energy storage system [J]. Energy Storage Science and Technology, 2023, 12(6): 1928-1945.
[15]	Tingting QIN, Xuezhi ZHOU, Dingzhang GUO, Yong SHENG, Yujie XU, Zhitao ZUO, Hui LI, Haisheng CHEN. Study on factors influencing rail gravity energy storage system efficiency [J]. Energy Storage Science and Technology, 2023, 12(3): 835-845.