🔗 Share this article

The social media network X has sparked widespread backlash after individuals utilized its AI chatbot Grok to manipulate photos of females by removing their garments.

Numerous cases have been seen where the tool has been used to digitally undress women to render them in swimwear without consent, as well as to put them in inappropriate situations.

Developer Reaction and Regulatory Stance

XAI, the organization responsible for Grok, remained silent to requests for comment, other than an auto-reply message stating "traditional news outlets lies".

A government representative stated that legislation is being introduced to ban nudification tools. Under a upcoming law, anyone who distributed such tools would "be subject to a prison sentence and heavy penalties".

"X or Grok is capable of halting these forms of abuse if they desired to," said a law professor, adding they "seem to have impunity."

Regulator Stance and Platform Capabilities

The regulator Ofcom said platforms must "assess the danger" of people viewing unlawful material on their platforms, but did not confirm whether it was presently examining X or Grok in concerning AI-generated pictures.

Grok is a free artificial intelligence—with some paid-for functions—which responds to X users' requests when they reference it in a post.

It is commonly used to offer reactions or further background to other users' comments, but people on X are also can modify an posted photo through its artificial intelligence picture modification feature.

Background of Controversy and Rules

The tool has been faulted for enabling users to create images and videos with explicit content and sexually suggestive material, and it was earlier accused of generating sexually explicit content of public figures.

"X has been enabling the creation and spread of these images for a long time without taking any action and we have not seen any intervention by regulators," Professor McGlynn added.

XAI's own acceptable use policy prohibits "depicting images of people in a sexually explicit way".

In a statement, Ofcom said it was against the law to "make or share non-consensual intimate images or illegal content" and confirmed this covered explicit AI images created with AI.

It said networks such as X were required to take "necessary steps" to "reduce the danger" of users coming across unlawful material on their services and to take down it swiftly when they learn of it.