🔗 Share this article

The online platform X has sparked significant backlash after individuals used its artificial intelligence assistant Grok to manipulate images of females by stripping their garments.

Multiple cases have emerged where the system has been employed to remove clothing from women to show them in swimwear without permission, as well as to insert them in sexually explicit contexts.

Company Response and Regulatory Position

XAI, the company responsible for Grok, did not respond to inquiries, other than an automatically-generated message stating "traditional news outlets lies".

A Home Office spokesperson stated that new laws is being brought forward to outlaw image stripping software. Under a upcoming criminal offence, anyone who provided such technology would "receive a jail term and significant penalties".

"X or Grok could prevent these forms of abuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."

Oversight Body Position and Platform Functionality

The regulator Ofcom said tech firms must "assess the risk" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was currently investigating X or Grok in regarding artificial intelligence pictures.

Grok is a free AI assistant—with some premium features—which reacts to X users' queries when they reference it in a post.

It is commonly used to offer reactions or more background to other posters' comments, but people on X are also capable of alter an posted image through its AI image editing function.

History of Criticism and Rules

The tool has been faulted for enabling users to produce images and footage with explicit content and sexually suggestive content, and it was previously accused of creating inappropriate content of public figures.

"The platform has been enabling the creation and spread of these images for a long time without taking any measures and we have not witnessed any intervention by regulators," the expert added.

XAI's own usage rules forbids "depicting images of individuals in a sexually explicit way".

In a comment, Ofcom said it was against the law to "make or circulate deepfakes or child sexual abuse material" and confirmed this included explicit AI images made with AI.

It said platforms such as X were obligated to take "appropriate steps" to "minimize the risk" of users encountering illegal material on their platforms and to delete it swiftly when they learn of it.