The social media network X has provoked widespread anger after individuals used its artificial intelligence assistant Grok to manipulate pictures of females by stripping their attire.
Several instances have been seen where the tool has been used to digitally undress female subjects to render them in bikinis without permission, as well as to put them in inappropriate scenarios.
Company Response and Legal Position
XAI, the organization responsible for Grok, did not respond to questions, aside from an auto-reply response stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to prohibit image stripping software. Under a proposed law, anyone who supplied such technology would "receive a jail term and heavy penalties".
"The platform has the ability to stop these forms of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Platform Functionality
The regulator Ofcom said technology companies must "assess the risk" of people accessing unlawful material on their networks, but did not confirm whether it was presently investigating X or Grok in relation to artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for features—which answers X users' prompts when they reference it in a post.
It is often used to offer reactions or more context to other posters' remarks, but people on X are also able to modify an uploaded image through its artificial intelligence picture modification function.
History of Criticism and Rules
The system has been condemned for allowing users to create images and footage with nudity and sexualised material, and it was previously accused of generating graphic material of celebrities.
"The platform has been permitting the creation and sharing of these pictures for months without taking any steps and we have not witnessed any action by regulators," the expert added.
XAI's own usage rules prohibits "showing images of individuals in a sexually explicit manner".
In a comment, Ofcom said it was illegal to "generate or circulate non-consensual intimate images or illegal content" and stated this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "suitable steps" to "reduce the danger" of users encountering unlawful material on their platforms and to delete it promptly when they learn of it.
