C-MAPPO-GAT Cuts 100% Downtime Violations to 0.3% in Autonomous Network Defense

Every unconstrained multi-agent reinforcement learning agent in the CAGE Challenge 4 benchmark violates the SOC downtime budget in 100% of episodes, with mean downtime proxy costs of 311-430 against a budget of 50. Reward-only learning is operationally undisciplined. The authors of the new paper introduce a safety-contract graph MARL framework that finally makes autonomous network defense more than a simulation toy.

The Problem with Reward-Only Learning

Standard MARL agents optimize a single reward signal. That’s fine for games, dangerous for network security where you have hard constraints like Mean Time to Recover (MTTR), false-positive response limits, and firewall change-management budgets. The paper replicates MAPPO-GAT, IPPO, and unconstrained variants across three 200-episode seeds. Every unconstrained method blew the downtime budget in every single episode. Mean downtime cost: 355.4 for MAPPO-GAT. Budget: 50. That’s not deployable in any SOC.

Safety Contracts: Separating Budgets from Observations

ACD$^3$-GAT (Adaptive Constrained Counterfactual Decisioning with a Graph Attention Network encoder) is the full proposed architecture. It separates simulator observations from reusable operational budgets, then applies constrained optimization, graph state encoding, and counterfactual action screening. C-MAPPO-GAT is a simpler variant that isolates Lagrangian operational-cost control and budget-aware screening. Both outperform the reward-only baselines by orders of magnitude on constraint satisfaction.

What the Numbers Actually Say

C-MAPPO-GAT reduces downtime violation from 100% to 0.3% and mean downtime cost from 355.4 to 15.5 relative to MAPPO-GAT. ACD$^3$-GAT achieves a mean downtime cost of 48.2 with a 13.8% violation rate. The authors place ACD$^3$-GAT on the safety-contract frontier rather than at the most conservative compliance point - meaning it trades a small increase in violation probability for better overall utility. Topology-seed and coupled adaptive Red-process stress tests confirm that safety-constrained policies degrade far less than reward-only MAPPO-GAT under attack.

The safety-contract framework lets you write down operational budgets as explicit constraints rather than hoping the reward function implicitly learns them. For any SOC team considering autonomous response, this is the difference between a paper exercise and something you could let touch production traffic.

Source: Safety-Contract Graph Multi-Agent Reinforcement Learning for Autonomous Network Security Response
Domain: arxiv.org