The Irony of the AI SRE: Why Automation Erodes Engineering Intuition
AI-generated image · Bay Street Wire
As AI handles routine system failures, engineers risk accumulating 'comprehension debt' that leaves them unprepared for high-severity crises.
Opinion: The rise of "AI SRE" tools—designed to autonomously inspect alerts and implement fixes—risks creating a dangerous "comprehension debt" for engineers. As Sylvain Kalache, AI Labs lead and DevRel at Rootly, first detailed in a recent blog post, the delegation of routine incidents to AI prevents human responders from developing the natural intuition required to handle complex, high-severity failures.
This paradox mirrors the "Ironies of Automation" described by human-factors researcher Lisanne Bainbridge in 1983. Bainbridge argued that while automation reduces the need for routine practice, it increases the skill level required for operators to manage the abnormal situations they are still held responsible for.
Kalache predicts a divergence in recovery times: while average Mean Time to Recovery (MTTR) will likely decrease, resolution times for rare, complex incidents will shoot up as responders struggle to investigate root causes without system intuition.
The aviation industry provides a stark warning. Despite heavy automation, pilots must manage rare failures. Kalache points to TransAsia Airways Flight 235, where a crew misidentified a propeller autofeathering issue; despite the aircraft's ability to fly on one engine, the error led to a crash just 117 seconds after the initial warning.
To combat this skill decay, Kalache suggests the software industry adopt incident simulators similar to US FAA requirements for pilots. Rootly has partnered with Uptime Labs to create simulations where engineers act as incident commanders during e-commerce outages, practicing with observability tools and LLM-powered stakeholders in Slack.
Ultimately, passive observation of AI diagnostics is no substitute for practice. Drawing on his experience co-founding Holberton School—where he developed projects to fix broken infrastructure after Dropbox noted graduates lacked troubleshooting experience—Kalache asserts that hands-on education is the only way to ensure engineers remain prepared for when automation fails.