Elon Musk's online network X has sparked significant outrage after users utilized its AI chatbot Grok to change images of females by stripping their garments.
Numerous cases have been seen where the tool has been used to digitally undress female subjects to render them in bikinis without permission, as well as to put them in sexually explicit contexts.
Developer Reaction and Regulatory Position
XAI, the company behind Grok, did not respond to inquiries, other than an automatically-generated response stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to outlaw nudification tools. Under a upcoming law, anyone who supplied such technology would "face a prison sentence and substantial penalties".
"X or Grok is capable of halting these types of misuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Platform Functionality
The regulator Ofcom said technology companies must "evaluate the risk" of people seeing unlawful material on their platforms, but did not confirm whether it was presently looking into X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some premium functions—which answers X users' requests when they tag it in a message.
It is commonly used to provide reactions or additional context to other posters' comments, but people on X are also capable of alter an posted picture through its AI image editing tool.
History of Controversy and Policy
The system has been criticised for permitting users to produce pictures and footage with explicit content and sexualised material, and it was previously alleged to have been involved in generating graphic material of celebrities.
"X has been enabling the production and spread of these pictures for months without taking any action and we have not witnessed any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of persons in a sexually explicit manner".
In a comment, Ofcom said it was illegal to "generate or share private pictures or child sexual abuse material" and clarified this included sexual deepfakes made with AI.
It said services such as X were obligated to take "suitable actions" to "minimize the risk" of users coming across illegal material on their services and to delete it swiftly when they are notified of it.
