OpenAI releases classifier to help detect unknown child abuse material online
The tool gives more organizations access to detection that catches material existing hash matching can miss, while ChatGPT is being set to point users toward local support.
OpenAI has announced new tools, practical guidance and partnerships aimed at preventing, detecting and responding to online child sexual exploitation and abuse. A central piece is a new classifier intended to help organizations find previously unidentified child sexual abuse material, which a partner says can catch material that hash matching misses, including drawings and animations. The company is also deploying interventions in ChatGPT, built with external experts, that aim to redirect users toward local support services. State attorneys general have pressed tech companies to increase their defenses against synthetic child abuse material.
Why it matters
For platforms and the groups that police them, the new classifier adds a way to find child sexual abuse material that has never been catalogued, which hash matching cannot do. In one partner's testing it caught such material while keeping false positives below 1% on legal adult content. For ChatGPT users who raise these topics, the company is building warnings with outside experts that aim to direct them to support services where they live. State attorneys general, who have said companies must work around the clock on synthetic child abuse material, are the audience pressing for these defences to improve.
Matters or hype?
Does this story matter, or is it hype? Vote before you see what everyone else said.
Sources · 1
- OpenAI News ↗ Primary source