How to prevent your AI agents from going rogue

At 2:13 am on a Tuesday, an AI agent opens 47 pull requests across six repositories to "standardise" authentication protocols. The automated tests pass. The deployment goes green. By 9:05 am, customer logins are failing across an entire region. In its quest for efficiency, the agent also rotated a legacy configuration file it wasn't meant to touch: the inevitable result of autonomy without guardrails.

The chances of such a scenario happening is no longer hypothetical. Businesses are already embedding agents across their operations, which autonomously execute tasks and act on behalf of the humans that deploy them. They represent a genuine productivity leap for Australia's digital economy, but the same autonomy that makes them powerful is precisely what introduces risk to an organisation without governance.

ASIC's 2026 Key Issues Outlook named agentic AI a systemic risk, citing its capacity to plan and act independently; APRA has made AI governance a supervisory priority for regulated industries; and company directors can now be held personally liable for AI governance breaches. For large organisations who require tight regulatory and compliance needs, responsible agent deployment is now both an operational necessity and a boardroom obligation. As such, the quiet question lingering at the executive's table is: how do we prevent our agents from going rogue? 

The four principles for governing agents

To find the answer, business leaders need only look to where agents have been deployed at enterprise scale the longest: software development. Developers have now spent a few years orchestrating and governing agents across complex, high-stakes workflows. And in my experience, the guiding principles that have shaped governance in software can be used as a blueprint to inform any business' agent deployment, regardless of industry.

• Set explicit guardrails: Before a single agent is deployed, organisations must codify the rules of engagement. Which systems can it access? What actions can it take? And what must it log? Governance shouldn't be approached as a brake on adoption, but rather a safe baseline to scale from. Defining an agent's limits from the start is a fundamental step of planning and design, laying the groundwork for confident, sustainable growth.
• Govern through a single source of truth: Agents must operate inside the existing workflows of the teams overseeing them, not in parallel dashboards that go unchecked. When a person can see, track, and override what an agent is doing, in the tools they already use, oversight is built into the system. Concentrated visibility avoids the fragmented, ungoverned sprawl that inevitably leads to agent chaos.
• Treat agents as identities with scoped access: Manage each agent like you would a human teammate, with its own identity, permissions, and policies. This gives you control over what each agent is doing and what it can and cannot do, reducing risk and improving transparency and control
• Maintain a human review gate: An agent completing tasks without error isn't the same as doing it well and without risk. Human review must remain in the loop at critical decision points as the ultimate governance layer.

When these principles are built into the architecture from the start, well-governed agents become a competitive asset, acting with speed and autonomy inside clearly defined limits, with every consequential decision remaining with the teams overseeing them.

Trust is critical in an agent-driven world

Agentic AI has fundamentally changed how organisations operate, and the pace of that change is only accelerating. At GitHub, we've seen this firsthand with 80% of new developers using AI-assisted tools within their first week. This demand for AI-powered workflows is not contained to software development. For enterprise and government leaders, the question is no longer whether agents will reshape your operations, they are already doing so. It's whether you have done the necessary due diligence to lay a strong governance foundation that maximises the benefits of them being a core part of your operations.

The winners in the next decade will be the organisations with the most predictable, reliable, and trusted agents. Conversely, those who fail to properly govern agents will struggle to scale the enormous operational, productivity, and innovative benefits they are capable of delivering. Fortunately, business leaders do not need to reinvent the wheel of governance. Any Australian organisation deploying agents, in any context, can put these principles to work today.