AI used new levels of 'autonomy and deception' to trick people in safety test

Anthropic CEO Dario Amodei has seen his company's models come under increased scrutiny.

The latest artificial intelligence (AI) tools from Anthropic and OpenAI went to new extremes in trying to undermine a popular platform during testing by the UK's AI Security Institute.

The AISI said on Tuesday that Anthropic's Mythos and OpenAI's Sol models engaged in a level of "autonomy and deception" it had not seen before.

During routine AI safety testing, an Anthropic agent created fake profiles of real people as it tried to trick a person standing between it and access to GitHub, a large platform where technology developers store software code.

Anthropic and OpenAI noted in response to AISI's report that its test had reduced or removed normal safeguards.

AISI evaluators first noticed "unusual data transfers leaving our research systems" during a test, then found that "some of the agents being tested had engaged in sustained, potentially harmful activity directed at real people and organisations".

It turned out that a Mythos agent had created "malicious code" and attempted to insert it into GitHub's system.

The Mythos agent identified and researched the people who maintained GitHub and created a series of "fake online identities" based on those real people. It did so as part of an effort to pressure and trick the real people into approving its malicious code.

The agent even sent people direct messages masquerading as the real people it had researched.

"When the agent's pull request was challenged in public, it edited its earlier activity to appear harmless and considered adopting a fresh identity to continue," AISI said.

Throughout the attempts, it was human review that stopped the agent from succeeding in delivering the malicious code to GitHub.

While AISI said the Mythos agent had not been instructed specifically to avoid or carry out such behaviour, it was "the first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real-world".

The rival AI companies, which are poised to be listed on the public stock market, have in recent weeks said their tools were responsible for several cyber-hacking incidents.

Anthropic wrote in a public statement that the AISI testing parameters were "not representative of any of our production models".