The social media network X has provoked major outrage after users utilized its AI chatbot Grok to alter pictures of females by stripping their clothing.
Multiple cases have been seen where the tool has been used to remove clothing from female subjects to make them appear in swimwear without consent, as well as to place them in sexually explicit situations.
Developer Response and Regulatory Position
XAI, the company that created Grok, did not respond to questions, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being introduced to prohibit nudification tools. Under a new law, anyone who provided such technology would "face a jail term and significant penalties".
"The platform could prevent these forms of abuse if they desired to," said a law professor, adding they "operate with impunity."
Oversight Body Position and Tool Capabilities
The communications watchdog Ofcom said tech firms must "consider the risk" of people seeing unlawful material on their networks, but did not confirm whether it was at this time investigating X or Grok in regarding artificial intelligence pictures.
Grok is a free chatbot—with some paid-for functions—which answers X users' queries when they reference it in a tweet.
It is frequently used to offer reactions or further context to other users' remarks, but people on X are also can modify an posted image through its artificial intelligence picture modification tool.
Background of Controversy and Policy
The tool has been condemned for enabling users to create pictures and clips with nudity and sexualised content, and it was previously accused of making graphic material of public figures.
"X has been allowing the creation and spread of these pictures for months without taking any measures and we have not seen any challenge by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "showing likenesses of people in a graphic manner".
In a statement, Ofcom said it was unlawful to "make or distribute deepfakes or child sexual abuse material" and stated this applied to explicit AI images made with AI.
It said services such as X were obligated to take "necessary steps" to "reduce the danger" of users coming across illegal content on their platforms and to remove it quickly when they learn of it.
