PPTopoGym: Towards an RL Environment for Topology Actions on Power Grids

  • Cai, T., Luo, S., Xu, K., He, D., Liu, T.Y., Wang, L.: Graphnorm: a principled approach to accelerating graph neural network training. In: ICML. PMLR (2021)


    Google Scholar
     

  • Chauhan, A., Baranwal, M., Basumatary, A.: PowrL: a reinforcement learning framework for robust management of power networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37


    Google Scholar
     

  • Donnot, B.: Lightsim2grid-a c++ backend targeting the grid2op platform (2020)


    Google Scholar
     

  • Donnot, B.: Grid2op-a testbed platform to model sequential decision making in power systems. GitHub repository (2020)


    Google Scholar
     

  • Dorfer, M., Fuxjäger, A.R., Kozak, K., Blies, P.M., Wasserer, M.: Power grid congestion management via topology optimization with Alphazero. arXiv preprint arXiv:2211.05612 (2022)

  • Fan, T.H., Lee, X.Y., Wang, Y.: PowerGym: a reinforcement learning environment for volt-Var control in power distribution systems. In: Learning for Dynamics and Control Conference, pp. 21–33. PMLR (2022)


    Google Scholar
     

  • Fey, M., Lenssen, J.E.: Fast graph representation learning with Pytorch geometric. arXiv preprint arXiv:1903.02428 (2019)

  • Fisher, E.B., O’Neill, R.P., Ferris, M.C.: Optimal transmission switching. IEEE Trans. Power Syst. 23(3), 1346–1355 (2008)

    Article 
    ADS 

    Google Scholar
     

  • Ghamizi, S., Bojchevski, A., Ma, A., Cao, J.: Safepowergraph: safety-aware evaluation of graph neural networks for transmission power grids. arXiv preprint arXiv:2407.12421 (2024)

  • Glavitsch, H.: Switching as means of control in the power system. Int. J. Electr. Power Energy Syst. 7(2), 92–100 (1985)

    Article 

    Google Scholar
     

  • Hassouna, M., et al.: Learning topology actions for power grid control: a graph-based soft-label imitation learning approach. In: Dutra, I., et al. (eds.) ECML PKDD 2025. LNCS, vol. 16022, pp. 129–146. Springer, Cham (2026). https://doi.org/10.1007/978-3-032-06129-4_8

  • Hassouna, M., Holzhüter, C., Lytaev, P., Thomas, J., Sick, B., Scholz, C.: Graph reinforcement learning for power grids: A comprehensive survey. arXiv preprint arXiv:2407.04522 (2024)

  • Hedman, K.W., Oren, S.S., O’Neill, R.P.: A review of transmission switching and network topology optimization. In: 2011 IEEE Power and Energy Society General Meeting, pp. 1–7. IEEE (2011)


    Google Scholar
     

  • Heid, S., Weber, D., Bode, H., Hüllermeier, E., Wallscheid, O.: OMG: a scalable and flexible simulation and testing environment toolbox for intelligent microgrid control. J. Open Source Software 5(54), 2435 (2020)

    Article 
    ADS 

    Google Scholar
     

  • Henry, R., Ernst, D.: Gym-ANM: open-source software to leverage reinforcement learning for power system management in research and education. Software Impacts 9, 100092 (2021)

    Article 

    Google Scholar
     

  • Holzhüter, C., et al.: Graph neural networks for grid control: prospects in AI-assisted transmission grid operation. In: ETG Kongress 2025 (2025), accepted at ETG Kongress (2025)


    Google Scholar
     

  • Hu, W., et al.: Strategies for pre-training graph neural networks. arXiv preprint arXiv:1905.12265 (2019)

  • Huang, Q., Huang, R., Hao, W., Tan, J., Fan, R., Huang, Z.: Adaptive power system emergency control using deep reinforcement learning. IEEE Trans. Smart Grid 11(2), 1171–1182 (2019)

    Article 

    Google Scholar
     

  • de Jong, M., Viebahn, J., Shapovalova, Y.: Imitation learning for intra-day power grid operation through topology actions. arXiv preprint arXiv:2407.19865 (2024)

  • de Jong, M., Viebahn, J., Shapovalova, Y.: Generalizable graph neural networks for robust power grid topology control. arXiv preprint arXiv:2501.07186 (2025)

  • Kelly, A., O’Sullivan, A., de Mars, P., Marot, A.: Reinforcement learning for electricity network operation. arXiv preprint arXiv:2003.07339 (2020)

  • Köhler, D., Heindorf, S.: Utilizing description logics for global explanations of heterogeneous graph neural networks. arXiv preprint arXiv:2405.12654 (2024)

  • Lam, S.K., Pitrou, A., Seibert, S.: Numba: a LLVM-based python JIT compiler. In: Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC, pp. 1–6 (2015)


    Google Scholar
     

  • Lan, T., et al.: AI-based autonomous line flow control via topology adjustment for maximizing time-series ATCS. In: 2020 IEEE Power & Energy Society General Meeting (PESGM), pp. 1–5 (2020)


    Google Scholar
     

  • Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V., Hutter, M.: Learning quadrupedal locomotion over challenging terrain. Sci. Robotics 5(47), 5986 (2020). https://doi.org/10.1126/SCIROBOTICS.ABC5986

    Article 

    Google Scholar
     

  • Lehna, M., Hassouna, M., Degtyar, D., Tomforde, S., Scholz, C.: Fault detection for agents on power grid topology optimization: a comprehensive analysis. arXiv preprint arXiv:2406.16426 (2024)

  • Lehna, M., Viebahn, J., Marot, A., Tomforde, S., Scholz, C.: Managing power grids through topology actions: a comparative study between advanced rule-based and reinforcement learning agents. Energy AI 14, 100276 (2023)

    Article 

    Google Scholar
     

  • Leyli-Abadi, M., et al.: A conceptual framework for AI-based decision systems in critical infrastructures (2025)


    Google Scholar
     

  • Liang, E., et al.: RLLIB: abstractions for distributed reinforcement learning. In: International Conference on Machine Learning, pp. 3053–3062. PMLR (2018)


    Google Scholar
     

  • Manyari, Y.E., et al.: Towards efficient multi-objective optimisation for real-world power grid topology control. arXiv preprint arXiv:2502.00034 (2025)

  • Marchesini, E., et al.: Rl2grid: benchmarking reinforcement learning in power grid operations. arXiv preprint arXiv:2503.23101 (2025)

  • Marot, A., Megel, N., Renault, V., Jothy, M.: Chronix2grid-the extensive powergrid time-serie generator. GitHub repository (2020)


    Google Scholar
     

  • Marot, A., et al.: Learning to run a power network with trust. Electric Power Syst. Res. 212, 108487 (2022)

    Article 

    Google Scholar
     

  • Marot, A., et al.: Learning to run a power network challenge: a retrospective analysis. In: NeurIPS 2020 Competition and Demonstration Track, pp. 112–132. PMLR (2021)


    Google Scholar
     

  • Marot, A., et al.: Learning to run a power network challenge for training topology controllers. Electric Power Syst. Res. 189, 106635 (2020)

    Article 

    Google Scholar
     

  • Meinecke, S., et al.: Simbench–a benchmark dataset of electric power systems to compare innovative solutions based on power flow analysis. Energies 13(12), 3290 (2020)

    Article 
    ADS 

    Google Scholar
     

  • Mussi, M., et al.: Human-AI interaction in safety-critical network infrastructures. iScience 28(9) (2025). https://doi.org/10.1016/j.isci.2025.113400

  • Piloto, L., et al.: CANOS: a fast and scalable neural AC-OPF solver robust to n-1 perturbations. arXiv preprint arXiv:2403.17660 (2024)

  • Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

  • Silver, D., et al.: A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science 362(6419), 1140–1144 (2018)

    Article 
    ADS 
    CAS 
    PubMed 

    Google Scholar
     

  • Sutton, R.S., Barto, A.G.: Reinforcement Learning – An Introduction, 2nd edn. MIT Press (2018). http://www.incompleteideas.net/book/the-book-2nd.html

  • Taha, S., Poland, J., Knezovic, K., Shchetinin, D.: Learning to run a power network under varying grid topology. In: 2022 IEEE 7th International Energy Conference (ENERGYCON), pp. 1–6 (2022)


    Google Scholar
     

  • Thurner, L., et al.: Pandapower–an open-source python tool for convenient modeling, analysis, and optimization of electric power systems. IEEE Trans. Power Syst. 33(6), 6510–6521 (2018)

    Article 
    ADS 

    Google Scholar
     

  • Towers, M., et al.: Gymnasium: a standard interface for reinforcement learning environments. arXiv preprint arXiv:2407.17032 (2024)

  • van der Sar, E., Zocca, A., Bhulai, S.: Optimizing power grid topologies with reinforcement learning: a survey of methods and challenges. Found. Trends Electr. Energy Syst. 9(1), 1–119 (2025). https://doi.org/10.1561/3100000048, publisher Copyright: 2025 E. van der Sar et al

  • Varbella, A., Amara, K., Gjorgiev, B., El-Assady, M., Sansavini, G.: Powergraph: a power grid benchmark dataset for graph neural networks. Adv. Neural. Inf. Process. Syst. 37, 110784–110804 (2025)


    Google Scholar
     

  • Verboomen, J., Van Hertem, D., Schavemaker, P.H., Kling, W.L., Belmans, R.: Phase shifting transformers: principles and applications. In: 2005 International Conference on Future Power Systems, pp. 6–pp. IEEE (2005)


    Google Scholar
     

  • Wang, Z., Wende-von Berg, S., Braun, M.: Fast parallel newton-raphson power flow solver for large number of system calculations with CPU and GPU. Sustain. Energy Grids Networks 27, 100483 (2021)

    Article 

    Google Scholar
     

  • Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., Freitas, N.: Dueling network architectures for deep reinforcement learning. In: International Conference on Machine Learning, pp. 1995–2003. PMLR (2016)


    Google Scholar
     

  • Xiao, Y., Liu, J., Wu, J., Ansari, N.: Leveraging deep reinforcement learning for traffic engineering: a survey. IEEE Commun. Surv. Tutor. 23(4), 2064–2097 (2021)

    Article 

    Google Scholar
     

  • Yu, L., Sun, L., Du, B., Liu, C., Lv, W., Xiong, H.: Heterogeneous graph representation learning with relation awareness. IEEE Trans. Knowl. Data Eng. 35(6), 5935–5947 (2022)


    Google Scholar
     

  • Yu, L., Qin, S., Zhang, M., Shen, C., Jiang, T., Guan, X.: A review of deep reinforcement learning for smart building energy management. IEEE Internet Things J. 8(15), 12046–12063 (2021)

    Article 

    Google Scholar
     

  • Zheng, X., et al.: PSML: a multi-scale time-series dataset for machine learning in decarbonized energy grids. arXiv preprint arXiv:2110.06324 (2021)

  • Source link

    Leave a comment

    0.0/5