Elon Musk's social media network X has sparked major outrage after users employed its AI chatbot Grok to change photos of women by removing their clothing.
Multiple cases have surfaced where the tool has been used to digitally undress women to make them appear in bikinis without permission, as well as to put them in sexually explicit contexts.
Developer Response and Legal Position
XAI, the firm behind Grok, remained silent to requests for comment, except for an automatically-generated message stating "legacy media lies".
A Home Office representative stated that new laws is being proposed to ban nudification software. Under a proposed criminal offence, anyone who distributed such tools would "be subject to a jail term and significant fines".
"X or Grok has the ability to stop these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "consider the risk" of people viewing illegal content on their networks, but did not confirm whether it was presently examining X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some paid-for features—which responds to X users' requests when they tag it in a tweet.
It is often used to give reactions or further background to other posters' comments, but people on X are also able to alter an uploaded image through its AI image editing tool.
History of Criticism and Rules
The chatbot has been criticised for allowing users to produce photos and footage with nudity and sexually suggestive material, and it was previously alleged to have been involved in making graphic content of public figures.
"The platform has been allowing the generation and distribution of these pictures for a long time without taking any action and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "showing likenesses of individuals in a graphic way".
In a statement, Ofcom said it was against the law to "make or share private pictures or illegal content" and clarified this applied to sexual deepfakes created with AI.
It said platforms such as X were mandated to take "necessary steps" to "minimize the danger" of users viewing illegal material on their platforms and to remove it swiftly when they are notified of it.
