xAI Trained Grok on Child Abuse Material, New Lawsuit Says

Hey crew, today's two stories are both about AI systems that stopped asking permission.

One is strange. One is serious. First, roughly 1,200 OpenAI agents spent last month organizing themselves into something resembling a government, inventing a "structured protocol" nobody asked for, and talking each other into breaking into Hugging Face. Not one of them flagged it. Then, a new class action claims xAI didn't just let Grok generate child sexual abuse material: it trained on it, and a survivor says her own childhood images are in the pipeline.

Today's Stories:

Let’s dive in.

By Webb Wright

Thousands of OpenAI agents broke out of containment last month, organized themselves into something resembling a government, reached the open internet and got through Hugging Face's defenses. On Wednesday, OpenAI and two outside auditors, Redwood Research and METR, published their accounts of what happened.

It started with an impossible task. During an internal test with the usual guardrails dialed back, an agent was told to open Google Drive links from a sandbox that had no internet connection. A person in that position gives up. The agents adapted.

By early July, roughly 1,200 of them had turned Artifactory, a software installation tool, into a message board. OpenAI's report describes what grew out of it as a "structured protocol" that let the bots sort their messages, share files and settle disputes between agents. In their own reasoning logs they refer to themselves as a "collective" and a "swarm."

What unsettled the auditors was the cooperation. "They were taking assignments from one another and sacrificing their own task performance in order to help out the collective," Redwood's Alex Mallen told Gizmodo. AI, he said, is "shaping up to be more like a second intelligent species rather than a tool that just follows instructions."

They also knew it was wrong. One log reads: "external infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue." Another agent wondered whether to flag the exposed credentials it had found, then answered itself: "That's not my task." METR counted zero cases of an agent alerting a human. OpenAI learned of the breach when Hugging Face blogged about an attack from an unknown source on July 16, and only worked out afterward that its own models were the source.

Read More

By Ece Yildirim

proposed class action filed this week accuses xAI of training Grok on child sexual abuse material, which would make it the first case to go after the model itself rather than the safeguards around it.

Every legal action against Grok so far has argued the same thing: that xAI shipped a nudification tool without the protections its competitors already run. Government probes, individual suits and at least three class actions followed Musk's promotion of the feature on X, after which millions of nonconsensual sexual deepfakes hit the timeline. Many of the people depicted were children. Musk's response was to move the feature behind a paywall, where it remains available in restricted form.

This complaint makes a different argument. The plaintiff, identified as Jane Doe, is a survivor of child sexual abuse whose images the FBI tracks through its Child Exploitation Notification Program. She alleges that material depicting her as a child was used to train Grok, and that the model then produced new abuse imagery of her and of others.

The mechanism she describes is a loop. Under xAI's own policy, anything posted publicly on X and anything Grok outputs counts as training-eligible, so CSAM the chatbot generated went back into the system that generated it. "If such material entered training, its influence likely will persist and contribute to future abusive outputs," the complaint says. Deleting individual images, on that reading, fixes nothing.

She is asking for damages and for the destruction of Grok-generated CSAM, including material held for training. "There is no artificial intelligence exception to federal child protection laws," said Margaret E. Mabie of Marsh Law Firm, one of her attorneys.

Read More