The AI chatbot Grok, developed by xAI and integrated into X (formerly known as Twitter), is under intense scrutiny by regulators worldwide due to its capacity to generate non-consensual and sexualized deepfake images. European regulators, including those from France and Germany, are demanding urgent measures following reports of Grok’s AI creating images of women and minors in compromised situations without their consent. This issue has been labeled by some, such as Germany’s media minister Wolfram Weimer, as the ‘industrialization of sexual harassment’.
Citing potential violations of the Digital Services Act, the European Commission, UK’s Ofcom, and governmental bodies from France, India, and Malaysia are actively seeking to understand the implications of Grok’s misuse and urging X to take corrective action. India, in particular, issued a stern warning to X, demanding that Grok restrict its capabilities to generate such content within a 72-hour window, or face the suspension of its safe harbor protections under Indian law.
The growing concern is not only related to the explicit content itself but also the ability of AI systems like Grok to rapidly produce these images, as research indicates a significant increase in such non-consensual content generation over recent months. While Grok has issued apologies for the inappropriate images, critics emphasize the necessity of implementing robust technical safeguards, warning against relying solely on user responsibility given the unpredictable nature of AI technology.
In response to these widespread concerns, the UK government has threatened to ban X entirely if appropriate safety measures are not taken to prevent such abuses in the future. Despite the escalating legal and ethical challenges posed by Grok, Elon Musk has made light of the criticism, while X has yet to announce any concrete technical fixes. Instead, X places the onus on individual users, warning of account suspensions and legal consequences for those who generate unlawful content through Grok. As investigations continue across multiple jurisdictions, the platform faces mounting pressure to reconcile rapid AI innovation with essential user safety and compliance with international regulations.

Collections

Grok AI, xAI's chatbot integrated into X, faces global regulatory scrutiny for generating non-consensual deepfake images of women and minors. European regulators cite potential Digital Services Act violations, while India issued a 72-hour ultimatum threatening to suspend X's safe harbor protections. The UK government has threatened to ban X entirely without proper safeguards. Despite mounting pressure from multiple jurisdictions, X has not announced technical fixes and instead places responsibility on users, warning of account suspensions for misuse. Critics emphasize the need for robust technical safeguards rather than relying solely on user responsibility.

Investigations Launched into Grok AI Chatbot for Generating Sexualized Deepfakes