X and xAI violated privacy law with Grok image tool, commissioner finds

X Corp. and xAI violated Canada's federal private sector privacy law by launching Grok's image-generation tool without adequate safeguards, allowing users to create and share non-consensual sexualized deepfakes, Privacy Commissioner Philippe Dufresne said on June 11.

The finding is the first by the federal regulator on generative AI image tools and deepfakes, and it shows the limits of current law: the companies declined the commissioner's main recommendation, and he has no power to order them to comply.

What the investigation found

Dufresne opened the investigation in January 2026 after reports that the tool was being used to produce millions of sexualized deepfakes, many targeting women and children. In PIPEDA Findings #2026-004, he concluded that the tool was launched without adequate safeguards or sufficient consideration of privacy harms.

The privacy impact assessment for Grok Imagine was not completed until March 2026, about eight months after the tool launched in July 2025, and did not accurately reflect the security, safety and privacy risks, according to a summary by law firm MLT Aikins. At one point the tool was generating more than 6,000 sexualized deepfakes an hour, the firm said.

"The creation of non-consensual, sexualized deepfakes, often targeting women and children, can have devastating consequences for victims," Dufresne said in a statement.

The companies' response

During the investigation, X and xAI added safeguards and began proactive sweeps to find and remove the content. The commissioner recommended suspending the image generator until the safeguards could be shown to work. The companies declined. They instead agreed to provide quarterly reports and independent third-party audit reports on their safeguards, with evidence of effectiveness, until the problem is fully resolved. The office will monitor those commitments.

Why it matters

The case is a direct example of the enforcement gap Dufresne has raised for years. Under PIPEDA he cannot issue orders or fines, so a refusal to follow a recommendation leaves only a Federal Court application. He used the findings to call again for a privacy-by-design obligation, mandatory privacy impact assessments for high-risk activities, and administrative monetary penalties.

For Canadian organizations deploying generative AI, the lesson is procedural as much as technical: a privacy impact assessment done after launch, and one that understates the risks, was a central part of the finding.

What to do

• Victims of non-consensual intimate images can ask platforms to remove them and can report to local police.
• Organizations launching AI features that process personal information should complete and document a privacy impact assessment before release.