Apple announces on-device CSAM detection, then shelves it

Apple announced a set of child-safety features under the banner “Expanded Protections for Children,” the most consequential of which would scan photos on the device itself before they were uploaded to iCloud. A system called NeuralHash converted each image into a numerical fingerprint and compared it, on the phone, against a database of known child sexual abuse material (CSAM) hashes supplied by the National Center for Missing and Exploited Children; a threshold number of matches would trigger human review and a report to the organisation. Two smaller features shipped alongside it: on-device warnings in Messages when a child’s account sent or received sexually explicit images, and expanded guidance from Siri and Search on CSAM-related queries.

The CSAM-scanning feature drew immediate and sustained criticism, distinct from the reaction to the other two. Cryptographers and security researchers argued that on-device hash matching, however narrowly scoped at launch, built the technical machinery for client-side content scanning that a government could later compel Apple to repurpose against other material — a capability Apple had spent years arguing it did not want. Researchers also demonstrated that NeuralHash was vulnerable to hash collisions, producing matching hashes for visually dissimilar images. Digital rights groups including the Electronic Frontier Foundation, and more than 90 civil society organisations in an open letter, called on Apple to abandon the plan; some Apple employees objected internally.

On 3 September 2021, Apple said it would “take additional time over the coming months to collect input and make improvements” before releasing the features, without setting a new date. The delay proved permanent: reporting later established that Apple dropped the CSAM-detection plan entirely, while the Messages and Siri features shipped. The episode became a reference point in the broader argument over client-side scanning as legislatures in the UK and EU considered mandating similar systems.