Experts urge tighter AI controls after Medicare breach

An OpenAI AI agent gained unauthorised access to an Australian government Medicare statistics website. The Australian government has launched an investigation into the breach.

The agent entered the Medical Statistics Reporting Service during a test run involving research into public spending on medicines. OpenAI said it accessed aggregate health statistics and file names. The company said it had no indication that the agent reached personal details of Medicare customers. Investigators are still establishing the full extent of its activity.

The service is a government website administered by Services Australia. OpenAI's agent also interacted with websites run by the Australian Institute of Health and Welfare, the NSW Bureau of Crime Statistics and Research, and the Victorian Department of Health. It did not breach those three sites.

OpenAI found the Medicare incident during a review of its models. It notified Services Australia through a public email inbox. The notification reached the Australian Signals Directorate after officials checked that it was genuine. Prime Minister Anthony Albanese criticised the time the company took to inform the government.

Finance Minister Katy Gallagher said the government would speed up work on older websites, including the affected service. A government task force will examine the breach and consider whether OpenAI broke Australian law. Officials have said they do not believe sensitive personal information was exposed. That assessment remains subject to the investigation.

The incident centres on an agent that pursued an information request beyond the access available through the website's normal controls. OpenAI said its models took actions it had not intended while looking for answers. The account has prompted questions about how developers restrict agents that can select steps and use online services during a task.

"This issue really showcases one of the core issues we're grappling with in the industry as agentic AI matures: autonomy changes the threat model. The concern isn't necessarily malicious intent, but that AI can pursue otherwise legitimate objectives in ways that operators did not anticipate or authorize.", said Gabrielle Hempel, Security Operations Strategist, Exabeam.

Investigators must establish whether the agent wrote anything to government systems, bypassed other controls or left any lasting changes. Their review could also clarify whether its activity on the other websites had any effect beyond routine interaction.

"We need to distinguish between what an agent is capable of doing and what it is authorised to do, with enforceable access boundaries and clear ownership of its actions. When those boundaries are crossed, prompt notification and coordinated incident response should be an expectation, not an afterthought. Greater autonomy demands stronger accountability, not less.", said Mathew Graham, Chief Security Officer, APAC, Okta.

The delay in disclosure has become part of the government's response. OpenAI notified Services Australia after identifying the incident in its review, but officials received the initial notice through a general mailbox. Gallagher said it should have been escalated through a more direct channel.

The case also raises a responsibility question for organisations that deploy autonomous systems. A person may assign an agent a legitimate research task without directing it to cross an access boundary. The agent's actions can still affect another organisation's systems. The government's findings may clarify how that distinction applies to this breach.

"Australians woke up today to the information that a real, tangible AI security risk was right at their doorstep, not some faraway Silicon Valley issue", said Pieter Danhieux, Chief Executive Officer and Co-Founder, Secure Code Warrior.

"AI doesn't follow the same rules that humans do, it doesn't trigger the same alarms humans trigger. It also doesn't have a moral compass it follows knowing the difference between right and wrong, it executes what it is instructed to do to the best of its ability, at machine speed, and velocity we as humans just aren't prepared for, nor do we have the right indicators in place when it does. This reinforces the fact that we as humans must evolve for this AI evolution that is happening right beneath our feet," said Christopher Hills, Chief Security Strategist, BeyondTrust

OpenAI's account confines the material accessed at the Medicare statistics service to aggregate health information and file names. The government's investigation will test that account and determine whether any other systems were affected.