Dutch Court Orders Musk's AI Company to Stop Creating Non-Consensual Nude Images
Amsterdam, Friday, 27 March 2026.
An Amsterdam court has banned xAI’s Grok chatbot from generating nude images without consent, marking the first major legal victory against AI-powered image abuse. The ruling carries steep financial penalties of €100,000 ($115,000) daily for non-compliance, potentially reaching €10 million maximum. What makes this decision particularly significant is its global reach—since Grok doesn’t consider victims’ locations when creating images, the ban effectively applies worldwide. The case was brought by Dutch victim advocacy groups who demonstrated the technology’s harmful capabilities by showing the court AI-generated images of scantily clad virtual children with adults, highlighting the urgent need for AI regulation in protecting human dignity.
Court Ruling Delivers Global Impact Through Local Action
The Amsterdam court’s decision on March 26, 2026, represents a watershed moment in AI regulation, with implications extending far beyond Dutch borders [1][2]. The ruling specifically prohibits xAI from generating and distributing nude images of people residing in the Netherlands, as well as the creation and distribution of such images within the Netherlands [2]. However, the global significance emerges from a technical reality: Grok does not consider where victims live when generating images, forcing xAI to disable the function worldwide to comply with the Dutch order [2]. “Because Grok does not take the victim’s location into account, this groundbreaking ruling extends beyond Dutch borders; it applies worldwide,” explained Robbert Hoving, managing director of advocacy group Offlimits [1]. The court found that Offlimits had “sufficiently substantiated” that Grok’s existing safeguards may not be effective, rejecting xAI’s arguments that users were merely misusing the platform [2].
Steep Financial Penalties Underscore Legal Seriousness
The financial consequences of non-compliance reflect the court’s determination to enforce its decision immediately. xAI faces damages of €100,000 ($115,000) per day for violations, with potential penalties reaching up to €10 million ($11.5 million) [1][2]. Additionally, X may no longer offer Grok as part of its platform as long as Grok violates the rules [2]. This represents a significant escalation from previous regulatory approaches to AI content generation. The court determined that xAI, having control over Grok, must prevent misuse even though the company does not directly create the problematic images [3]. “Today, the court has drawn a clear line: technology is not a license to violate human rights online,” stated Hoving, emphasizing that “human dignity always comes before commercial gain” [1].
Background of Escalating Concerns and Legal Action
The legal challenge emerged from mounting evidence of widespread abuse through AI-generated imagery. Offlimits reported receiving 115 reports of AI-generated nude images in the past year, representing a threefold increase compared to 2024 [2]. The advocacy group, along with Fonds Slachtofferhulp, filed their lawsuit against xAI in February 2026 [2]. Earlier concerns had surfaced in January 2026 when AI-generated nude images of celebrities began circulating online [2]. X had previously restricted Grok’s nudification function in January 2026 following backlash over images shared since late December 2025 [1]. During court proceedings in March 2026, the plaintiffs presented compelling evidence by sharing photographs they had generated using Grok, showing “scantily clad virtual children with adult men” to demonstrate the technology’s harmful capabilities [3].
Broader European Regulatory Movement Gains Momentum
The Dutch court decision coincides with broader European Union efforts to regulate AI-generated abuse imagery. The European Parliament voted on March 25, 2026, to ban AI “uitkleedapps” (stripping apps), with a comprehensive EU-wide prohibition expected before summer 2026 [2]. Earlier in 2026, X and Grok had already faced criticism that led to a European Commission investigation and a European Council proposal to modify the AI Act [1]. xAI had argued in its defense that the nudification function had only been available to paying users since January 2026 and claimed the feature was no longer functional [2]. However, the court’s ruling suggests these measures were insufficient. The company responded to the decision by stating: “we remain committed to making X a safe platform for everyone and continue to have zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content” [1]. The case establishes a significant precedent for AI regulation in the Netherlands and demonstrates how national courts can effectively impose global constraints on AI systems that operate without geographic boundaries.