The social media network X has sparked major anger after people employed its AI chatbot Grok to manipulate images of women by removing their clothing.
Numerous instances have been seen where the tool has been employed to digitally undress women to render them in bikinis without permission, as well as to place them in sexually explicit scenarios.
Company Reaction and Legal Position
XAI, the firm responsible for Grok, remained silent to requests for comment, other than an standard message stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to outlaw nudification tools. Under a upcoming law, anyone who distributed such tools would "be subject to a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said technology companies must "consider the risk" of people seeing unlawful material on their services, but did not confirm whether it was currently investigating X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' queries when they tag it in a tweet.
It is often used to offer reactions or more background to other users' remarks, but people on X are also capable of modify an uploaded picture through its AI image editing tool.
Background of Controversy and Policy
The tool has been faulted for permitting users to create pictures and videos with nudity and sexualised content, and it was in the past alleged to have been involved in making graphic content of public figures.
"X has been allowing the creation and sharing of these pictures for months without taking any steps and we have yet to see any challenge by regulators," the expert added.
XAI's own acceptable use policy bans "showing likenesses of individuals in a sexually explicit manner".
In a comment, Ofcom said it was against the law to "generate or distribute non-consensual intimate images or illegal content" and stated this applied to explicit AI images created with AI.
It said networks such as X were obligated to take "appropriate actions" to "reduce the risk" of users viewing illegal content on their services and to remove it swiftly when they learn of it.
