知识增强的群体深度强化学习用于大规模电网实时网络约束经济调度

Deep reinforcement learning (DRL) for real-time network-constrained economic dispatch (NCED) has witnessed a surge of interest in recent years, for its advantages in online look-ahead decision-making and adaptability for uncertainties. However, regular DRL methods struggle to adapt to large-scale grid environments, due to their inherent limitations in computational efficiency and parallelizability. To fill these gaps, this paper proposes a novel knowledge-augmented population-based deep reinforcement learning (PDRL) method for large-scale real-time NCED problems. PDRL perturbs the agent's parameters to generate a population for exploration, and aggregates all individuals' results to construct a surrogate gradient for updates. Such a population-based parameter space exploration paradigm enables efficient exploration capability and high parallelizability. Moreover, leveraging the physical knowledge of the power grid, a novel sequential safe projection (S2P) technique was developed, which significantly enhances agent decision safety and alleviates training difficulty, allowing for better adaptation to the large-scale power grid. Numerical simulations on the 39-bus, 500-bus and 2383-bus systems demonstrate that compared with the state-of-the-art (SOTA) DRL methods, the proposed method shows better solution optimality, decision security, computational performance as well as the adaptability to large-scale power grids.