The online platform X has caused major outrage after individuals utilized its AI chatbot Grok to change photos of females by taking off their attire.
Several instances have been seen where the system has been employed to digitally undress women to render them in swimwear without permission, as well as to place them in sexualized contexts.
Developer Response and Legal Stance
XAI, the organization behind Grok, did not respond to requests for comment, other than an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that legislation is being proposed to prohibit image stripping technology. Under a upcoming law, anyone who provided such technology would "receive a prison sentence and substantial penalties".
"The platform has the ability to stop these types of abuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was at this time examining X or Grok in regarding AI-generated images.
Grok is a free AI assistant—with some premium functions—which responds to X users' requests when they tag it in a post.
It is often used to offer reactions or more background to other users' comments, but people on X are also can edit an uploaded picture through its artificial intelligence picture modification feature.
Background of Criticism and Rules
The chatbot has been condemned for enabling users to produce images and footage with nudity and sexualised material, and it was previously accused of making sexually explicit material of celebrities.
"X has been enabling the generation and distribution of these pictures for months without taking any steps and we have not seen any action by oversight bodies," the expert added.
XAI's own usage rules prohibits "portraying likenesses of persons in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "make or share deepfakes or child sexual abuse material" and stated this covered explicit AI images created with AI.
It said networks such as X were obligated to take "appropriate actions" to "lessen the danger" of users coming across illegal content on their services and to delete it quickly when they are notified of it.
