一种面向时空城市轨道交通的混合储能系统容量优化与能量管理多任务强化学习方法

Passenger flow fluctuation and delay-induced traffic regulation bring considerable challenges to cost-efficient regenerative braking energy utilization of hybrid electric storage systems (HESSs) in urban rail traction networks. This paper proposes a synergistic HESS sizing and energy management optimization framework based on multi-task reinforcement learning (MTRL) for enhancing the economic operation of HESSs under dynamic spatio-temporal urban rail traffic. The configuration-specific HESS control problem under various spatio-temporal traction load distributions is formulated as a multi-task Markov decision process (MTMDP), and an iterative sizing optimization approach considering daily service patterns is devised to minimize the HESS life cycle cost (LCC). Then, a dynamic traffic model composed of a Copula-based passenger flow generation method and a real-time timetable rescheduling algorithm incorporating a traction energy-passenger-time sensitivity matrix is developed to characterize multi-train traction load uncertainty. Furthermore, an MTRL algorithm based on a dueling double deep Q network with knowledge transfer is proposed to simultaneously learn a generalized control policy from annealing task-specific agents and operation environments for solving the MTMDP effectively. Comparative studies based on a real-world subway have validated the effectiveness of the proposed framework for LCC reduction of HESS operation under urban rail traffic.