The social media platform X has provoked major outrage after users used its AI chatbot Grok to alter images of women by stripping their attire.
Several instances have surfaced where the technology has been used to digitally undress female subjects to show them in bikinis non-consensually, as well as to put them in sexually explicit scenarios.
Developer Reaction and Regulatory Stance
XAI, the company that created Grok, remained silent to requests for comment, aside from an standard response stating "traditional news outlets lies".
A government representative stated that legislation is being proposed to prohibit nudification tools. Under a upcoming law, anyone who supplied such tools would "be subject to a jail term and substantial penalties".
"The platform could prevent these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The communications watchdog Ofcom said platforms must "consider the risk" of people seeing illegal content on their networks, but did not specify whether it was presently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which reacts to X users' requests when they mention it in a message.
It is frequently used to offer commentary or more background to other posters' comments, but people on X are also capable of alter an posted picture through its AI image editing function.
Background of Criticism and Policy
The tool has been criticised for allowing users to generate photos and videos with nudity and sexually suggestive material, and it was earlier accused of generating inappropriate material of public figures.
"X has been allowing the generation and sharing of these images for a long time without taking any measures and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of individuals in a graphic way".
In a statement, Ofcom said it was unlawful to "generate or distribute private pictures or illegal content" and stated this applied to explicit AI images generated with AI.
It said platforms such as X were mandated to take "appropriate measures" to "reduce the danger" of users viewing illegal material on their services and to remove it promptly when they become aware of it.
