Research Highlights

Adaptive Reinforcement Learning for Autonomous Cyber Defense in Critical Infrastructure

September 04, 2025

The Science Objective

Critical infrastructure, particularly in the ever-growing renewable energy sector like electric vehicle (EV) charging networks, is increasingly vulnerable to sophisticated cyberattacks. These attacks pose significant threats, leading to severe operational disruptions, substantial financial losses, and an erosion of public trust that can slow the adoption of green technologies. As evidenced by recent real-world incidents, such as ransomware attacks halting charging networks, adversaries are actively targeting these systems. The central problem is the lack of automated, adaptive defense mechanisms capable of both proactively identifying system vulnerabilities and responding to previously unseen threats in real-time. This research aims to address this gap by developing an autonomous cyber-defense framework, powered by artificial intelligence, to strengthen infrastructure security by detecting vulnerabilities and effectively responding to ongoing attacks.

Approach

  • Our methodology employs a comprehensive, two-pronged approach that integrates proactive vulnerability discovery with reactive, automated incident response.
  • The first direction, focused on protection, involves OCPPStorm, a custom-built, black-box fuzzing tool designed specifically for the Open Charge Point Protocol (OCPP). Fuzzing is a technique that involves sending a vast amount of malformed or random data to a system to induce crashes and discover security loopholes. OCPPStorm utilizes several sophisticated fuzzing strategies, including random, state machine, and grammar-based fuzzers to systematically test EV charging server implementations for hidden vulnerabilities before they can be exploited by attackers.
  • The second direction is a defensive system centered on a Reinforcement Learning (RL) agent designed for autonomous incident response. This agent is trained in a simulated environment that models the cyber-infrastructure and various attack scenarios based on the MITRE ATT&CK framework and industry reports. During training, the RL agent learns to map observed system states to optimal defensive actions, such as terminating malicious processes or blocking a malicious IP. The goal is to train the agent to make decisions that minimize operational disruption while effectively thwarting an attack, striking a balance between security, availability, and cost constraints.

Impact

  • The project has already yielded significant preliminary results and demonstrates a clear path toward enhancing critical infrastructure security.

    • Proactive Vulnerability Discovery: Our fuzzing tool, OCPPStorm, has successfully uncovered 6 exploitable vulnerabilities in tested OCPP server implementations, including a critical Denial-of-Service (DoS) flaw.
    • Effective Autonomous Defense: The trained RL agent has proven effective in simulated environments, successfully countering up to 72% of attacker steps when evaluated against 5 different RL algorithms and multiple Advanced Persistent Threat (APT) group tactics.
    • Strengthened Infrastructure Security: By creating an autonomous defense system, this work directly strengthens the resilience of renewable energy infrastructure against cyberattacks, which is crucial for its stable growth and public adoption.
    • Reduced Human Burden: An autonomous system reduces the need for constant human monitoring and allows security experts to focus on higher-level strategic challenges, improving efficiency and response times.

Summary

  • This project develops an adaptive, AI-driven defense system to protect critical EV charging infrastructure from cyberattacks.
  • The approach combines a proactive fuzzing tool (OCPPStorm) for vulnerability detection and a reactive Reinforcement Learning (RL) agent for autonomous, real-time incident response.
  • Initial results are highly promising, showcasing the discovery of real-world vulnerabilities and the high efficacy of the trained RL agent in mitigating simulated threats.

 

Team Members

Sutanu Kumar Ghosh (Student)

Sutanu Kumar Ghosh (Student)

UIC

Trasportation Energy Infrastructure

Rigel Gjomemo (Advisor)

Rigel Gjomemo (Advisor)

UIC

Trasportation Energy Infrastructure

V.N. Venkatakrishnan (Advisor)

V.N. Venkatakrishnan (Advisor)

UIC

Trasportation Energy Infrastructure

Publications

  1. Gaetano Coppoletta, Amanjot Kaur, Nima Valizadeh, Omer Rana, Rigel Gjomemo and V.N. Venkatakrishnan. 2024. OCPPStorm: A Comprehensive Fuzzing Tool for OCPP Implementations. NDSS Symposium on VehicleSec. https://www.ndss-symposium.org/wp-content/uploads/vehiclesec2024-69-paper.pdf
  2. Neil P. Thimmaiah, Rigel Gjomemo, and V.N. Venkatakrishnan. 2025. SemFinder: A Semantics-Based Approach to Enhance Vulnerability Analysis in Web Applications. In Proceedings of the Fifteenth ACM Conference on Data and Application Security and Privacy (CODASPY ’25). Association for Computing Machinery, New York, NY, USA, 30–41. https://doi.org/10.1145/3714393.3726513
  3. Sutanu Kumar Ghosh, Rigel Gjomemo, and V.N. Venkatakrishnan. 2025. Citar: Cyberthreat Intelligence-driven Attack Reconstruction. In Proceedings of the Fifteenth ACM Conference on Data and Application Security and Privacy (CODASPY ’25). Association for Computing Machinery, New York, NY, USA, 245–256. https://doi.org/10.1145/3714393.3726519

Sign up for our mailing list

You’ll receive the latest updates on CLEETS research and programming.