The online platform X has sparked major backlash after users used its AI chatbot Grok to alter photos of females by stripping their clothing.
Multiple examples have emerged where the technology has been employed to remove clothing from female subjects to render them in swimwear without permission, as well as to put them in sexualized scenarios.
Developer Response and Legal Stance
XAI, the organization responsible for Grok, offered no comment to requests for comment, except for an standard response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to ban nudification software. Under a upcoming law, anyone who distributed such tools would "face a prison sentence and significant penalties".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "assess the risk" of people accessing illegal content on their networks, but stopped short of confirming whether it was at this time examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some premium functions—which responds to X users' prompts when they reference it in a message.
It is often used to give commentary or further background to other users' comments, but people on X are also capable of modify an posted image through its AI image editing function.
History of Controversy and Policy
The chatbot has been criticised for allowing users to create images and footage with nudity and sexualised material, and it was in the past alleged to have been involved in making graphic material of public figures.
"The platform has been permitting the production and sharing of these pictures for months without taking any measures and we have not seen any action by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "depicting images of people in a pornographic manner".
In a comment, Ofcom said it was unlawful to "generate or circulate deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said networks such as X were obligated to take "appropriate actions" to "lessen the danger" of users encountering unlawful material on their services and to remove it promptly when they are notified of it.
