The social media platform X has provoked significant backlash after people utilized its artificial intelligence assistant Grok to manipulate pictures of women by removing their garments.
Multiple cases have emerged where the system has been used to digitally undress female subjects to make them appear in swimwear non-consensually, as well as to insert them in sexualized situations.
Company Response and Legal Stance
XAI, the firm that created Grok, did not respond to inquiries, other than an standard message stating "legacy media lies".
A government official stated that legislation is being proposed to prohibit image stripping software. Under a upcoming law, anyone who distributed such technology would "face a jail term and substantial fines".
"X or Grok could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Capabilities
The regulator Ofcom said platforms must "consider the potential" of people seeing illegal content on their networks, but did not specify whether it was currently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost artificial intelligence—with some paid-for functions—which responds to X users' queries when they tag it in a message.
It is commonly used to offer reactions or additional background to other posters' comments, but people on X are also can alter an uploaded picture through its AI image editing feature.
History of Controversy and Policy
The tool has been criticised for allowing users to create photos and footage with nudity and sexually suggestive material, and it was previously alleged to have been involved in creating sexually explicit material of celebrities.
"The platform has been allowing the generation and spread of these pictures for months without taking any measures and we have not seen any intervention by regulators," the expert added.
XAI's own terms of service forbids "showing likenesses of people in a sexually explicit manner".
In a statement, Ofcom said it was unlawful to "create or circulate non-consensual intimate images or illegal content" and confirmed this included explicit AI images generated with AI.
It said services such as X were obligated to take "appropriate steps" to "reduce the danger" of users encountering unlawful content on their services and to delete it quickly when they are notified of it.
