OpenAI releases AI classifier to detect child sexual abuse material
OpenAI released a new AI-powered image classifier on November 16, 2026, to detect child sexual abuse material (CSAM), including content generated by artificial intelligence. The technology analyzes visual signs of abuse and assigns scores to images to assist human reviewers in prioritizing cases. OpenAI developed and evaluated the technology with assistance from the National Center for Missing & Exploited Children (NCMEC).
Technical capabilities and testing results
The classifier uses visual signs rather than relying solely on digital fingerprints or hashes, which allows the system to detect novel or previously unseen abusive material. Internal testing showed the classifier can reduce the number of images flagged for review by 48% while maintaining the same detection rate for abusive content. During testing with the platform fal.ai, human reviewers confirmed that 72% of highly suspicious flagged images contained previously unidentified CSAM.
In SafetyKit tests, the technology demonstrated a 90% detection rate for cases involving sexualized descriptions of children in drawings and animations. The SafetyKit tests also showed a maximum false positive rate of 1% for legitimate adult content. North American media reported a specific detection rate of 49.3% for images of infants in specific offline evaluations.
Safety protocols and developer tools
OpenAI is implementing ChatGPT warnings that interrupt attempts to find abusive material. These warnings explain the consequences of such attempts and provide links to specialist support. Violations of ChatGPT safety policies regarding CSAM can result in a 24-hour account suspension, conversation deletion, or potential account closure and reporting to authorities. North American media reported that testing with invideo AI resulted in 1 in 4 cases leading to suspension.
The company will make the tool available to select companies via the OpenAI API at no cost. OpenAI also provided guidelines for developers on CSAM prevention, along with tools and training for law enforcement agencies to assist in investigations. OpenAI stated that the new classifier is the first widely available CSAM classifier with a specific signal for AI-generated content.
Institutional context and safety concerns
The release follows previous scrutiny regarding safety measures for younger users. North American media reported that Common Sense Media conducted an assessment in October 2026 regarding ChatGPT for Teens. That assessment cited unacceptable risks and called for age restrictions to 18+.
Glossary
- child sexual abuse material (CSAM) — This refers to material involving the sexual exploitation of children. It can be obtained through capturing actual exploitation on camera or through voyeurism.
- National Center for Missing & Exploited Children (NCMEC) — This is a charity helpline that receives information regarding child sexual abuse material (CSAM). It is an organization that receives data and hashes related to such content.
- OpenAI API — This is a developer tool used to access OpenAI models. It allows for the use of specific model variants like GPT-5.4 nano.
This article was written with AI assistance from multiple news reports and is reviewed by our editors under our editorial principles. Editorial Standards · Tips & Contact
Comments 0