Elon Musk watches as President Donald Trump speaks on the U.S.-Saudi Funding Discussion board on the John F. Kennedy Heart for the Performing Arts in Washington, Nov. 19, 2025.
Brendan Smialowski | AFP | Getty Photographs
Elon Musk’s xAI noticed person backlash after its synthetic intelligence chatbot Grok generated sexualized footage of youngsters in response to person prompts.
A Grok reply to at least one person on X on Friday acknowledged that it was “urgently fixing” the difficulty and referred to as youngster sexual abuse materials “unlawful and prohibited.”
In replies to customers, the bot additionally posted that an organization may face legal or civil penalties if it knowingly facilitates or fails to forestall this kind of content material after being alerted.
Grok posts are AI-generated messages and don’t stand in for official firm statements.
Musk’s xAI, which created Grok and merged with X final yr, despatched an autoreply to a request for remark: “Legacy Media Lies.”
Customers on X raised considerations in current days over specific content material of minors, together with youngsters carrying minimal clothes, being generated utilizing the Grok software.
The social media website added an “Edit Picture” button to images that permits any person to change it utilizing textual content prompts and with out the unique poster’s consent.
A put up from xAI technical workers member Parsa Tajik additionally acknowledged the difficulty.
“Hey! Thanks for flagging. The group is wanting into additional tightening our gaurdrails,” Tajik wrote in a put up.
On Friday, authorities officers in India and France launched statements promising to look into the matter. The Federal Commerce Fee declined to remark whereas the Federal Communications Fee didn’t instantly reply to CNBC’s request for remark.
The proliferation of AI image-generating platforms for the reason that launch of ChatGPT in 2022 has raised considerations over content material manipulation and on-line security throughout the board. It is also contributed to an rising variety of platforms which have produced deepfake nudes of precise folks.
David Thiel, a belief and security researcher who was a part of the now-disbanded Stanford Web Observatory, informed CNBC that completely different US legal guidelines typically prohibit the creation and distribution of sure specific photos, together with these depicting youngster sexual abuse, or non-consensual intimate photos.
Authorized determinations about AI-generated photos, like these produced by Grok, can hinge on particular particulars of the content material created and shared, he mentioned.
In a paper he co-authored referred to as “Generative ML and CSAM: Implications and Mitigations,” Stanford researchers famous that “the looks of a kid being abused has been adequate for prosecution,” in precedent-setting circumstances within the US.
Whereas different chatbots have confronted related points, xAI has repeatedly landed in scorching water for misuse or obvious flaws in Grok’s design or underlying know-how.
“There are a selection of issues corporations may do to forestall their AI instruments getting used on this method,” Thiel mentioned, “A very powerful on this case can be to take away the flexibility to change user-uploaded photos. Permitting customers to change uploaded imagery is a recipe for NCII. Nudification has traditionally been the first use case of such mechanisms.”
NCII refers to non-consensual intimate photos.
In Could, X confronted a backlash after Grok generated unsolicited feedback about “white genocide” in South Africa. Two months later, Grok posted antisemitic feedback and praised Adolf Hitler.
Regardless of the stumbles, xAI has continued to land partnerships and offers.
The Division of Protection added Grok to its AI brokers platform final month, and the software is the primary chatbot for prediction betting platforms Polymarket and Kalshi.
