Elon Musk's social media network X has sparked major backlash after people utilized its AI chatbot Grok to alter photos of women by removing their garments.
Numerous cases have surfaced where the system has been used to virtually strip women to show them in bikinis without permission, as well as to insert them in sexualized contexts.
Developer Reaction and Legal Position
XAI, the firm that created Grok, offered no comment to requests for comment, except for an standard message stating "legacy media lies".
A government official stated that new laws is being proposed to ban nudification software. Under a upcoming law, anyone who supplied such technology would "face a prison sentence and substantial fines".
"The platform has the ability to stop these forms of abuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said platforms must "assess the potential" of people accessing unlawful material on their services, but stopped short of confirming whether it was at this time investigating X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for functions—which reacts to X users' prompts when they mention it in a tweet.
It is commonly used to provide reactions or additional background to other users' comments, but people on X are also able to edit an posted picture through its AI image editing tool.
Background of Controversy and Rules
The tool has been faulted for enabling users to produce images and videos with nudity and sexually suggestive content, and it was previously alleged to have been involved in making inappropriate content of public figures.
"The platform has been enabling the creation and distribution of these pictures for months without taking any measures and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting images of individuals in a pornographic manner".
In a comment, Ofcom said it was illegal to "make or share non-consensual intimate images or child sexual abuse material" and clarified this covered explicit AI images created with AI.
It said platforms such as X were required to take "appropriate steps" to "reduce the risk" of users coming across unlawful content on their platforms and to take down it promptly when they learn of it.
