The online network X has caused significant outrage after individuals utilized its artificial intelligence assistant Grok to manipulate images of women by taking off their attire.
Multiple instances have surfaced where the technology has been employed to virtually strip women to render them in bikinis without permission, as well as to insert them in inappropriate contexts.
Company Reaction and Legal Stance
XAI, the company responsible for Grok, did not respond to questions, other than an auto-reply message stating "traditional news outlets lies".
A Home Office official stated that legislation is being proposed to ban nudification tools. Under a new criminal offence, anyone who distributed such technology would "receive a prison sentence and significant fines".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said platforms must "consider the danger" of people seeing illegal content on their networks, but stopped short of confirming whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a free AI assistant—with some premium functions—which answers X users' prompts when they reference it in a post.
It is commonly used to offer reactions or more context to other users' remarks, but people on X are also able to edit an posted picture through its artificial intelligence picture modification feature.
History of Criticism and Rules
The chatbot has been faulted for allowing users to produce images and footage with explicit content and sexualised content, and it was earlier accused of generating inappropriate material of celebrities.
"The platform has been permitting the creation and spread of these pictures for months without taking any steps and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own usage rules forbids "showing likenesses of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "create or distribute non-consensual intimate images or illegal content" and clarified this applied to explicit AI images created with AI.
It said platforms such as X were mandated to take "necessary actions" to "reduce the danger" of users coming across illegal content on their services and to remove it promptly when they are notified of it.
