🔗 Share this article

The social media platform X has caused major backlash after individuals used its artificial intelligence assistant Grok to alter photos of females by stripping their garments.

Numerous examples have been seen where the technology has been used to virtually strip women to make them appear in swimwear non-consensually, as well as to insert them in inappropriate situations.

Developer Response and Regulatory Stance

XAI, the organization responsible for Grok, did not respond to inquiries, aside from an standard message stating "legacy media lies".

A government representative stated that legislation is being brought forward to prohibit nudification software. Under a new criminal offence, anyone who provided such tools would "be subject to a jail term and heavy fines".

"X or Grok has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."

Regulator Position and Platform Functionality

The communications watchdog Ofcom said technology companies must "consider the risk" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in relation to artificial intelligence images.

Grok is a no-cost artificial intelligence—with some additional functions—which responds to X users' requests when they mention it in a message.

It is commonly used to offer commentary or further background to other posters' comments, but people on X are also can modify an posted picture through its AI image editing function.

History of Controversy and Policy

The tool has been condemned for permitting users to generate pictures and videos with nudity and sexually suggestive content, and it was in the past alleged to have been involved in creating sexually explicit material of celebrities.

"X has been allowing the production and sharing of these images for a long time without taking any action and we have not witnessed any challenge by oversight bodies," the expert added.

XAI's own acceptable use policy forbids "depicting images of persons in a graphic manner".

In a statement, Ofcom said it was illegal to "generate or circulate non-consensual intimate images or illegal content" and clarified this included explicit AI images generated with AI.

It said services such as X were required to take "suitable measures" to "reduce the danger" of users viewing illegal material on their services and to take down it quickly when they are notified of it.