基于序贯决策的异构智能体强化学习方法——面向配网与输网协同的负荷频率控制

With the continuous penetration of large-scale distributed energy resources (DERs) into the distribution network (DN), the DN has acquired the capability to participate in load frequency control (LFC). This paper proposed an order-based heterogeneous-agent soft actor-critic method (OHASAC) to address the coordination challenges among heterogeneous controllable DERs. A neural network is applied to estimate the optimal update sequence of the heterogeneous agents. The optimal LFC problem is formulated as a partially observable Markov game (MG) considering the coordination of DN with sufficient controllable DERs and transmission network (TN). The modeling process considers the coordinated frequency regulation between battery energy storage systems (BESS) and photovoltaics (PV) under variable irradiance conditions. Simulations in the coordinated environment of the distribution network (DN) and transmission network (TN) demonstrate that the proposed method can effectively manage various distributed generation resources while achieving superior generalization and scalability.