Research on Intelligent Control Method of Operating Temperature of Reactor Thermal System Based on Deep Reinforcement Learning

Liu Yongchao; Tan Sichao; Li Tong; Cheng Jiahao; Wang Bo; Gao Puzhen; Tian Ruifeng

doi:10.13832/j.jnpe.2024.S2.0197

Volume 45 Issue S2

Jan. 2025

Turn off MathJax

Article Contents

Article Navigation > Nuclear Power Engineering > 2024 > 45(S2): 197-205

Liu Yongchao, Tan Sichao, Li Tong, Cheng Jiahao, Wang Bo, Gao Puzhen, Tian Ruifeng. Research on Intelligent Control Method of Operating Temperature of Reactor Thermal System Based on Deep Reinforcement Learning[J]. Nuclear Power Engineering, 2024, 45(S2): 197-205. doi: 10.13832/j.jnpe.2024.S2.0197

Citation:

Liu Yongchao, Tan Sichao, Li Tong, Cheng Jiahao, Wang Bo, Gao Puzhen, Tian Ruifeng. Research on Intelligent Control Method of Operating Temperature of Reactor Thermal System Based on Deep Reinforcement Learning[J]. Nuclear Power Engineering, 2024, 45(S2): 197-205. doi: 10.13832/j.jnpe.2024.S2.0197

Citation:

Liu Yongchao, Tan Sichao, Li Tong, Cheng Jiahao, Wang Bo, Gao Puzhen, Tian Ruifeng. Research on Intelligent Control Method of Operating Temperature of Reactor Thermal System Based on Deep Reinforcement Learning[J]. Nuclear Power Engineering, 2024, 45(S2): 197-205. doi: 10.13832/j.jnpe.2024.S2.0197

PDF( 13042 KB)

Research on Intelligent Control Method of Operating Temperature of Reactor Thermal System Based on Deep Reinforcement Learning

doi: 10.13832/j.jnpe.2024.S2.0197

Liu Yongchao^{1, 2
,},
Tan Sichao^{1, 2
,
,},
Li Tong^{1, 2},
Cheng Jiahao^{1, 2},
Wang Bo^{1, 2},
Gao Puzhen^{1, 2},
Tian Ruifeng^{1, 2}

1.
Heilongjiang Provincial Key Laboratory of Nuclear Power System & Equipment, Harbin Engineering University, Harbin, 150001, China
2.
Key Laboratory of Nuclear Safety and Advanced Nuclear Energy Technology, Ministry of Industry and InformationTechnology, Harbin Engineering University, Harbin, 150001, China

Received Date: 2024-06-21
Rev Recd Date: 2024-09-11
Publish Date: 2025-01-06

Abstract

Abstract

Traditional proportional-integral-differential (PID) control method is difficult to achieve good and stable control effect. In this paper, an intelligent control method of operating temperature of reactor thermal system based on deep reinforcement learning is proposed. The steps are as follows: RELAP5 model of reactor thermal system is built and extended interactively, so that it can support deep reinforcement learning technology. Secondly, based on the Soft Actor-Critic (SAC) algorithm and coupled with the multivariable Long Short-Term Memory (LSTM) neural network, the time series characteristics of the control history information are effectively extracted. Then, the control model driven by optimization goal can collect data samples by itself, and complete the optimization of control strategy through self-learning mechanism. According to the multivariable state characteristics and time series characteristics, the end-to-end control of operating temperature is realized. Compared with the simulation experiment of PID controller, the proposed method has excellent load tracking ability and disturbance suppression ability, and has good environmental adaptability and control robustness.
- Reactor thermal system,
- Deep reinforcement learning,
- Soft Actor-Critic (SAC),
- Long Short-Term Memory (LSTM),
- Intelligent control

FullText(HTML)

References(10)

References

[1]	DONG Z, CHENG Z H, ZHU Y L, et al. Review on the recent progress in nuclear plant dynamical modeling and control[J]. Energies, 2023, 16(3): 1443. doi: 10.3390/en16031443
[2]	孙奥迪,孙培伟,魏新宇. 小型铅铋冷却快堆堆芯功率控制研究[J]. 核动力工程,2022, 43(6): 155-161.
[3]	DEGRAVE J, FELICI F, BUCHLI J, et al. Magnetic control of tokamak plasmas through deep reinforcement learning[J]. Nature, 2022, 602(7897): 414-419. doi: 10.1038/s41586-021-04301-9
[4]	PARK J K, KIM T K, SEONG S H. Providing support to operators for monitoring safety functions using reinforcement learning[J]. Progress in Nuclear Energy, 2020, 118: 103123. doi: 10.1016/j.pnucene.2019.103123
[5]	SAEED H A, PENG M J, WANG H, et al. Autonomous control model for emergency operation of small modular reactor[J]. Annals of Nuclear Energy, 2023, 190: 109874. doi: 10.1016/j.anucene.2023.109874
[6]	刘永超,李桐,成以恒,等. 基于深度确定性策略梯度算法的自适应核反应堆功率控制器设计[J]. 原子能科学技术,2024, 58(5): 1076-1083.
[7]	HAARNOJA T, ZHOU A, ABBEEL P, et al. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor[C]//Proceedings of the 35th International Conference on Machine Learning. Stockholm: PMLR, 2018: 1861-1870.
[8]	朱少民,夏虹,吕新知,等. 基于ARIMA和LSTM组合模型的核电厂主泵状态预测[J]. 核动力工程,2022, 43(2): 246-253. doi: 10.13832/j.jnpe.2022.02.0246.
[9]	张思原,卢忝余,曾辉,等. 基于LSTM的核电传感器多特征融合多步状态预测[J]. 核动力工程,2021, 42(4): 208-213. doi: 10.13832/j.jnpe.2021.04.0208.
[10]	冀南,杨俊康,赵鹏程,等. 耦合多变量LSTM与优化算法的铅铋反应堆事故参数预测方法研究[J]. 核动力工程,2023, 44(5): 64-70. doi: 10.13832/j.jnpe.2023.05.0064.