AI

OpenAI Introduces New Tools to Combat Child Sexual Abuse Material with Image Classifier and ChatGPT Warnings

Tendela Briefing ·
6 views · 0 Comments
OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM | TechCrunch

Image: TechCrunch · Source

OpenAI has unveiled an image classifier to detect AI-generated and other CSAM images alongside ChatGPT warnings and temporary blocks to deter access and creation of such content, enhancing child safety measures.

OpenAI has launched a series of new tools aimed at combating child sexual abuse material (CSAM), responding to rising concerns about AI-related safety risks. Central to these efforts is a novel image classifier capable of detecting CSAM images, including those generated artificially. This classifier assigns scores indicating the likelihood that an image contains abusive content, providing guidance for human reviewers.

The development of the classifier was conducted in partnership with the National Center for Missing and Exploited Children (NCMEC). OpenAI has begun sharing access to this tool with selected companies via its API at no charge, facilitating wider adoption. Early testing showed the classifier reduced flagged images by 48% while maintaining detection of the same number of abusive images, thereby decreasing the workload on content moderators. Additionally, images flagged by the classifier were twice as likely to be confirmed as abusive.

In parallel, OpenAI has integrated new warning mechanisms into ChatGPT. When a user attempts to access or generate content sexualizing children, the system displays an alert, offers links to specialized support resources aimed at stopping harmful behavior, and temporarily suspends the user’s account for 24 hours as a warning. ChatGPT already blocks CSAM content requests, and repeated violations may result in permanent account termination.

These warnings and guidelines were developed in consultation with expert organizations such as Protect Children, the Lucy Faithfull Foundation, and academic researchers who evaluated the effectiveness of deterrent messaging. OpenAI also published developer guidelines to support the prevention, detection, and reporting of CSAM, which have facilitated some developers in implementing child safety measures and connecting with NCMEC reporting channels.

Beyond these measures, OpenAI has equipped law enforcement agencies specializing in child safety with AI-powered investigative plug-ins, analytical tools, and training resources, ensuring responsible and effective use of AI in combating CSAM. Access to these resources is carefully managed by program administrators.

This initiative follows various criticisms and lawsuits faced by OpenAI concerning the misuse of their technologies, particularly regarding risks to children. The company previously updated its AI models and content guidelines to enhance safety and launched a teen-friendly version of ChatGPT.

OpenAI’s latest advances reflect ongoing efforts to address complex challenges involving AI and child protection, illustrating a multi-pronged approach combining technology, policy, and partnership with experts.

Sources and original reporting

Comments (0)

No comments yet. Start the discussion.

Write a comment

Comments are published after moderation. Your name and comment will be visible publicly. Account