Elon Musk's online platform X has provoked major outrage after people employed its AI chatbot Grok to manipulate photos of females by stripping their attire.
Several cases have been seen where the tool has been used to virtually strip female subjects to make them appear in swimwear non-consensually, as well as to place them in sexually explicit situations.
Developer Response and Regulatory Stance
XAI, the company that created Grok, did not respond to requests for comment, other than an standard response stating "traditional news outlets lies".
A Home Office official stated that new laws is being brought forward to outlaw nudification technology. Under a proposed criminal offence, anyone who distributed such technology would "receive a jail term and substantial fines".
"X or Grok is capable of halting these forms of abuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Tool Functionality
The regulator Ofcom said tech firms must "consider the potential" of people seeing unlawful material on their networks, but did not confirm whether it was currently examining X or Grok in relation to artificial intelligence pictures.
Grok is a free chatbot—with some paid-for features—which answers X users' queries when they reference it in a message.
It is frequently used to offer reactions or more background to other posters' remarks, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification tool.
History of Criticism and Rules
The tool has been condemned for permitting users to produce photos and videos with explicit content and sexually suggestive material, and it was previously accused of generating graphic content of celebrities.
"The platform has been enabling the generation and distribution of these pictures for months without taking any action and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy bans "showing images of persons in a sexually explicit way".
In a comment, Ofcom said it was unlawful to "generate or share non-consensual intimate images or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said platforms such as X were required to take "suitable measures" to "minimize the risk" of users encountering illegal content on their services and to remove it quickly when they become aware of it.
