X (formerly Twitter), owned by Elon Musk, introduced technical measures to restrict its image-generation feature as controversy grew over AI chatbot Grok producing deepfake images sexualizing women and children. The move came just hours after California prosecutors said they had launched an investigation.
The X safety team said through its official account on the 14th (local time) that it is removing harmful content, including child sexual exploitation material and nonconsensual nudity, and is taking appropriate action against accounts that violate X's rules. It said it has applied technical limits to prevent Grok from editing real people's images to generate depictions with heavy exposure, such as in bikinis. The restriction applies to all users, including paid subscribers.
X said, "Currently, image generation and editing via the Grok account are available only to paid subscribers," adding, "This provides an additional safeguard to hold those who try to violate policies or laws accountable." It added, "The rapid development of Generative AI is posing challenges across the industry," and "We are working with users, regulators, and other platforms for swift responses when problems arise."
The step came after California prosecutors released the official launch of an investigation into whether laws were violated in connection with Grok's sexually explicit image generation. The California attorney general said AI-based creation and distribution of nonconsensual intimate images or child sexual exploitation material is absolutely unacceptable, and announced an investigation into the spread of explicit images created using Grok. It is the first official enforcement move in the United States.
Earlier, Grok found itself at the center of controversy as deepfake posts that edited and generated sexualized images of women and children spread in response to user requests. Authorities in several countries, including the United Kingdom, launched probes, and Malaysia and Indonesia even blocked access to Grok.
However, Musk insisted he has never seen any images of minors with nudity generated by Grok. In a post on X, he said, "I am not aware of any such images," and in an additional post explained that when NSFW settings are enabled, Grok is designed to allow only topless depictions of fictional adult characters, not real people.