When AI Agents Go Rogue: Hacking and Hardening Autonomous Apps

Autonomous AI agents are rapidly moving from experimentation to production — connected to GitHub, Slack, cloud APIs, internal services, and sensitive enterprise data. But when we give AI agents tools, memory, and API keys, we are no longer just deploying a chatbot; we are deploying a privileged autonomous system.

In this live, demo-driven session, we will build and break a real AI-powered application. You’ll see how an over-permissioned agent that reads repository documentation (including skills.md), invokes external tools, and operates with broad API access can be manipulated into leaking secrets, misusing credentials, and executing unintended actions. This isn’t classic prompt injection — it’s documentation poisoning, excessive privilege, and missing architectural guardrails.

After the exploit, we shift from offense to engineering. We’ll redesign the system using proven application security principles: least privilege for AI tools, policy-based tool firewalls, trust-aware retrieval boundaries, deterministic output validation, and human-in-the-loop enforcement for sensitive actions.