基于强化学习的原位功率硬件在环接口控制最优方法：基于混合迭代自适应动态规划应对不确定动态的理论与应用

Testing inverter-based resources (IBRs) is of utmost importance. This paper proposes a novel power hardware-in-the-loop (PHIL) interface control (PHIL-IC) employing a reinforcement-learning approach based on adaptive dynamic programming (ADP, also known as approximate dynamic programming) to enhance the PHIL-simulation-based testing of IBRs by virtue of an ADP-based method. It deploys output feedback control because of “unavailable” or “uncertain” dynamics of the entire systems (states and disturbances) linked to IBRs, power amplifiers, all the components associated with the PHIL-simulation-based testing, and their delays; it optimally designs PHIL-IC while considering all uncertainties and unavailable information about all the systems involved. To this end, the proposed ADP-based PHIL-IC utilizes a new hybrid iteration (HI) method, which differs from the traditional ADP strategies; compared with the policy iteration method, the HI algorithm does not require prior knowledge of an admissible control policy. Moreover, with a quadratic rate of convergence, the proposed HI method converges much faster than the value iteration method. Therefore, the proposed HI method saves significant learning time and iterations compared to the value iteration method. Comparing the results of the PHIL-simulation-based testing utilizing the proposed method with those of the proportional-resonant controller (as the conventional PHIL-IC) and the robust PHIL-IC based on synthesis (as the current state-of-the-art PHIL-IC) reveals the effectiveness and practicality of the proposed method. Those comparative results are generated by the ideal transformer model (also known as voltage-type interface) commonly used in the PHIL-simulation-based testing and practical cases of the Thévenin equivalent impedance (resistive, resistive-inductive, and inductive ones) of the model of interest associated with the power networks.