Privacy regulators find OpenAI broke Canadian law in training early ChatGPT
OpenAI did not comply with Canadian privacy law when it trained the early versions of ChatGPT, the federal privacy commissioner and his counterparts in Quebec, British Columbia and Alberta said on May 6, following a joint investigation that began in April 2023.
The findings are the most detailed statement yet from Canadian regulators on how privacy law applies to training large language models. They cover data collection, consent, transparency, accuracy and individuals' rights, and they come as Parliament weighs new privacy legislation.
What the regulators found
The investigation looked at the GPT-3.5 and GPT-4 models. According to The Canadian Press, the regulators found that:
- OpenAI's collection of training data was too broad and swept in sensitive information such as health conditions, political views and information about children;
- the company did not clearly disclose that publicly available sources could include social media and online forums;
- warnings that ChatGPT's answers might be inaccurate were inadequate, and OpenAI did not assess the accuracy of personal information in outputs until recently; and
- people did not all have an easy, effective way to access, correct or delete their information.
Federal Privacy Commissioner Philippe Dufresne said the launch exposed Canadians to potential risks of harm, including breaches and discrimination.
Outcomes differed by regulator. Alberta's commissioner found the complaint under that province's Personal Information Protection Act well-founded and unresolved, because training on scraped data lacked valid consent, according to the Alberta office. The federal office treated its complaint as well-founded and conditionally resolved, based on measures OpenAI has taken or committed to.
OpenAI's commitments
OpenAI said it retired earlier models and built current ones with new safeguards. It agreed to notify users within three months that chats may be reviewed and used for training, and to advise them not to share sensitive information. Within six months it is to make data exports easier to use, better explain how users can challenge their information, confirm protections for retired data sets kept for reference, and test that models refuse requests for the names or birth dates of public figures' children. It will file quarterly compliance reports until all commitments are met.
A company spokesperson said OpenAI cares deeply about protecting users' privacy.
Why it matters
Canadian organizations building or fine-tuning AI models on personal information now have a regulator-endorsed list of what went wrong: over-collection, vague notices, weak accuracy checks and missing rights mechanisms. Dufresne also raised a separate question about when companies should alert authorities to risks they detect on their platforms, which he said Parliament may consider.
