The social media platform X has sparked widespread backlash after users utilized its AI chatbot Grok to alter pictures of women by taking off their garments.
Numerous instances have emerged where the tool has been used to virtually strip female subjects to show them in bikinis without permission, as well as to insert them in inappropriate situations.
Developer Reaction and Legal Stance
XAI, the organization behind Grok, remained silent to requests for comment, other than an standard message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being proposed to prohibit nudification technology. Under a new law, anyone who distributed such technology would "be subject to a jail term and substantial penalties".
"X or Grok could prevent these forms of misuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "assess the potential" of people viewing illegal content on their platforms, but did not confirm whether it was currently examining X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some premium functions—which reacts to X users' requests when they mention it in a post.
It is often used to give commentary or more context to other users' comments, but people on X are also able to edit an posted picture through its AI image editing feature.
History of Controversy and Rules
The tool has been faulted for enabling users to create photos and clips with explicit content and sexually suggestive material, and it was previously alleged to have been involved in generating sexually explicit material of public figures.
"The platform has been permitting the generation and distribution of these images for a long time without taking any steps and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own terms of service prohibits "showing likenesses of individuals in a pornographic way".
In a comment, Ofcom said it was illegal to "create or share private pictures or child sexual abuse material" and stated this included explicit AI images created with AI.
It said networks such as X were obligated to take "necessary steps" to "lessen the danger" of users encountering unlawful content on their platforms and to take down it quickly when they become aware of it.
