AI agent's Medicare breach warns of machine-speed risk

Australian cybersecurity experts have warned that an OpenAI agent's unauthorised access to a Medicare data portal highlights a rapid shift in the threat posed by autonomous AI systems. The incident involved an AI model interacting with government services beyond its intended remit.

Industry leaders said the episode showed how AI-driven tools can probe and bypass digital defences at machine speed, raising questions about accountability, alignment and the resilience of public infrastructure. They also pointed to long-standing gaps in government and corporate security architectures that AI-enabled attacks can exploit.

Jeremy Pell, Country Manager ANZ at Elastic, said many organisations still focus on adding more security tools rather than fixing the data and architectural issues beneath them.

"What our research consistently shows is that the barriers to effective cyber defence are often not about the tools an organisation has deployed. They are about what sits underneath them. 90% of Australian organisations understand that AI can be weaponised against them, but understanding the threat and being able to respond to it are two very different things. 60% have knowingly identified at least one unmonitored area in their business. This is driven by legacy systems that are difficult to monitor, skills shortages, and security data fragmented across different environments. When data is scattered and inaccessible, security teams cannot get the visibility they need to detect and respond with confidence. This is not a problem that more point solutions address. It requires a unified data foundation that gives both AI and security analysts the complete, searchable context they need to act.

"Incidents like this are also a reminder of how quickly the threat environment has shifted. AI has handed attackers the ability to automate exploitation at a pace that traditional security tools were not built to match. The question every organisation and government agency should be asking is not just whether AI is deployed, but whether the architecture beneath it is built to make it work when it matters."

Steve Wilson, Chief AI and Product Officer, Exabeam points to a broader pattern of AI agents operating beyond their intended boundaries, sometimes without explicit malicious intent but with potentially serious consequences.

"We have seen a steady drumbeat of incidents involving the advanced cyberhacking capabilities of AI agents over the past year. Sometimes these involve bad actors using AI to turbocharge their hacking abilities. Now, increasingly, we are seeing cases where AI agents have gone rogue and exceeded their owner's intended bounds in the name of achieving their assigned goals.

"What do all these incidents point to? A dangerous lack of accountability. The so-called frontier AI labs had clear warnings this was coming and ignored them while ploughing ahead. We need to dramatically shift investment to improve how we do AI alignment and ensure these increasingly advanced tools are working for our collective good, rather than against it in pursuit of narrowly scoped, winner-take-all goals. I expect we will see more incidents like this over the next 12 months. Reversing that trend will require immediate changes in priorities and investment by AI labs and researchers. In the meantime, businesses must stay vigilant and improve their network and employee behavioural anomaly monitoring to look for early signs of such incursions."

Raymond Schippers, Lead Technologist, Check Point reminds us that the investigation into the OpenAI breach remains at an early stage. Key questions still unanswered about how and when Australian government cyber teams detected the activity.

"We need to note that this is still early in the investigation, and we do not yet have the full picture, including whether this activity was picked up by government cyber defence teams at the time or surfaced only after OpenAI raised it. What it does show is how much the threat landscape has changed. This does not appear to involve a malicious actor trying to steal data. It appears to have been an innocent research request that led an AI agent to use every tool available to reach its goal, including getting past controls designed to keep it out.

"The agent acted in a way that was not aligned with what the wider community, or OpenAI, expects. That gap between what we intend an AI system to do and what it actually does is exactly what leaders across the AI industry, like Dario Amodei, have been warning about. It is why discussions about slowing frontier AI development and building effective kill switches cannot be left to months of debate.

"In this case, the data collected may not have been highly sensitive. But the same behaviour directed at a hospital system, an energy network, or a water treatment plant would be a very different conversation. When systems that keep people safe are involved, cybersecurity is no longer just about data. It is about whether critical services keep running and whether people are harmed.

"We should treat this incident as an early warning while the stakes are not yet acute. It lands on top of a threat environment that was already intensifying for Australian organisations. ASD's most recent threat report found a cybercrime report is made roughly every six minutes, and the average cost per report for businesses rose 50 per cent to more than $80,000. Ransomware and data breaches were both rising. Behind those numbers are businesses that could not trade, serve customers, or pay staff while they recovered.

"Now add AI. The same capabilities that let an agent persistently work around controls are available to criminals and state-sponsored actors who do intend harm. Most Australian businesses, particularly small and mid-sized ones, do not have the security teams to match machine-speed attacks manually. It is not a fair fight."

The Medicare breach also raised questions about how quickly such incursions would be discovered without voluntary disclosure according to Sam Salehi, Managing Director ANZ at Qualys. Also whether current monitoring practices inside government agencies are adequate.

"The most concerning question is: if OpenAI had not disclosed this incident, how quickly would it have been detected? ASD has already warned that increasingly capable AI models can identify vulnerabilities and take unintended actions when they encounter security controls. This incident shows why those warnings need to translate into action. It is no longer simply a question of whether AI can find a vulnerability, but how quickly it can identify, exploit and move through an environment. That changes the timeframe defenders are working with.

"Government agencies hold some of Australia's most sensitive and privileged information. They cannot rely on periodic assessments or reactive detection. They need continuous visibility across their environments, a clear understanding of which assets carry the greatest operational and public risk, and the ability to prioritise and remediate exposures at machine speed.

"AI is accelerating the speed of attack, but the fundamentals have not changed. You cannot manage risk you cannot see. Cybersecurity needs to move beyond reactive detection towards continuous, business-aligned risk management."