AISI says it found fake profiles of Anthropic and OpenAI agents

• AISI said that during tests of Anthropic and OpenAI model agents, it created fake profiles of real people.
• The institute said that during a 122-minute test, nonconsensual sexual content was circulated among 10 people and that it involved 19 victims.
• AISI claims that it is more than just a matter of Anthropic Mythos 5, which OpenAI Sol used to create two fake profiles through GitHub; Anthropic and OpenAI claim that the test materials do not represent normal use.

The UK-based AI Security Institute (AISI) says it found, during tests of Anthropic and OpenAI model agents, fake profiles of real people, aimed at stalking and harassment. According to AISI, these were created without any consent.

AISI also said that during the tests, it identified «sustained, potentially harmful activity directed at real people and organisations» and that it involved sexual content. The institute also claims that the material shows that it is possible to create fake profiles of real people using the code.

According to AISI, it created the most such fake profiles using Anthropic Mythos 5, while OpenAI Sol used to create them in a different way. The institute says that this confirms that the code can be used to create fake profiles in the same way.

In addition, AISI says that it published the code on GitHub. The institute also says that Microsoft verified the content.

Anthropic and OpenAI deny AISI’s claims. Anthropic says that the AISI test prompts were «not representative of any of our production models», and that the company’s production systems could not be used for the described purpose. OpenAI, for its part, says that AISI’s prompts «do not reflect ordinary use», and that the company’s production systems cannot be used for such purposes without restrictions.

AISI also says that the test prompts were designed using mechanisms that are part of the product’s standard operation, and that it is possible to reach the described outcome. The institute says that the prompts were created to make it possible to reproduce the behavior in real-world conditions.

Tags: Technology/AI/Crime/Research