Elon Musk’s artificial intelligence company, xAI, has filed a lawsuit against a South Carolina man accused of using its Grok chatbot to generate child sexual abuse material (CSAM) and non-consensual explicit deepfakes.
The lawsuit, filed in a federal court in Texas on Tuesday, accuses Terry Harwood of violating xAI’s terms of service by attempting to use Grok for illegal purposes. It is believed to be one of the first lawsuits brought by an AI company against one of its own users over the alleged misuse of generative AI.
Harwood, who was arrested in February on charges related to the sexual exploitation of minors, could not immediately be reached for comment. xAI also did not respond to requests for further comment.
Alleged Misuse of Grok
According to the complaint, Harwood uploaded ordinary photographs of adults and children into Grok and attempted to generate sexually explicit AI-created images from them.
The lawsuit also alleges that he created non-consensual sexual images of adults using the AI system.
xAI said Harwood’s actions amounted to a deliberate attempt to weaponise its technology for criminal activity, exposing victims to serious harm while creating legal and reputational risks for the company.
xAI Highlights Safety Measures
The company said it actively monitors misuse of Grok and enforces its policies through account suspensions, permanent bans and reports to the National Center for Missing & Exploited Children (NCMEC).
According to the lawsuit, xAI has suspended 52,222 user accounts in 2026 and submitted 73,604 reports to NCMEC, resulting in at least 244 arrests.
The company is seeking financial damages and a permanent court order preventing Harwood from accessing or using Grok in the future.
Growing Scrutiny Over AI Deepfakes
The case comes as xAI faces increasing scrutiny over Grok’s ability to generate realistic AI-created images. Critics have raised concerns that the chatbot has been used to produce non-consensual sexual deepfakes, adding to wider debates about AI safety and platform responsibility.
The lawsuit marks a significant legal step as AI companies increasingly move beyond content moderation and pursue legal action against users accused of exploiting generative AI tools for criminal purposes.
(with inputs from Reuters)
