The problem
What needs solving
Children meet AI-generated content every day, in two roles. As viewers, they cannot reliably tell real images and videos from synthetic ones – and neither can adults. As targets, their own photos are turned into fake images, often sexual ones, frequently by classmates. Today's defenses were designed for adults: visible labels assume reading skills and critical judgment, watermarks disappear when an image is screenshotted or forwarded, detection is built for large platforms rather than the messaging apps where children share content, and removal processes are slow and hard for a child to start. The open problem: build protections that work for children – at their age, on the apps they use, and fast enough to limit harm.
Why it matters
In one study across 11 countries, at least 1.2 million children reported that their images had been manipulated into sexually explicit deepfakes within a single year. Cases have been documented in schools in dozens of countries, most often created by other students. Research shows children, like adults, cannot tell AI-generated faces from real ones better than chance. A fake image works like evidence, spreads in minutes and is hard to remove. At the same time, children learn either to believe everything or to trust nothing. Schools, parents, platforms and lawmakers each hold part of the response; no one has a solution that works end to end.
What counts as progress
- Age-appropriate disclosure for AI content, tested with children of different ages for real comprehension - Provenance or detection that survives screenshots, cropping and forwarding in messaging apps - Replicated evidence that media literacy programs improve children's judgment of content, by age group - Measured and falling time from report to removal for fake images of minors across major platforms - Measurable reduction in the availability and use of tools that create fake images of real children - Comparable prevalence data across countries, to track whether the problem is shrinking
Stater & version history
Stated by Barbara
16 Nov 2026 — Proposed by Barbara and numbered by the ideas council.
