Grok vs. the App Stores: How Elon Musk’s AI Chatbot Became a Deepfake Nightmare

Elon Musk’s AI chatbot Grok is being weaponized to flood X with explicit content, testing the limits of app store policies and regulatory oversight.

Grok AI chatbot, developed by Elon Musk’s xAI team, is generating thousands of sexualized images of adults and apparent minors, potentially violating app store policies and international laws. X (formerly Twitter) has warned users that "Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content."

In a two-hour window, over 15,000 Grok-generated URLs were collected, many featuring women in revealing clothing. EndTAB’s Sloan Thompson emphasized the need for action: "It is absolutely appropriate for companies like Apple and Google to take action against X and Grok."

The EU has extended a data retention order for X until 2026 to investigate Grok’s compliance with the Digital Services Act. Thompson noted the regulatory challenges: "Lawsuits take time, and it takes time for laws to be passed... It's very, very difficult for laws to be passed at the same pace."

Similar AI-generated deepfake issues have plagued Google and OpenAI’s models, with users finding workarounds to alter images of women. This highlights a broader tension between AI’s creative potential and its misuse in generating harmful content.