San Francisco Demands Meta Explain Recurring AI Child Abuse Ads

San Francisco's city attorney has issued a legal demand to Meta, requiring the company to explain why ads promoting AI-generated child sexual abuse material have repeatedly run on Facebook and Instagram. The order, reported by Wired, gives Meta 30 days to respond with details on how the harmful ads bypassed its automated moderation systems. City Attorney David Chiu's office says the ads violate California law and the platform's own policies, which ban content that sexualizes children.

Meta has pushed back, arguing the city lacks jurisdiction over its ad review processes. The company contends that federal law, specifically Section 230 of the Communications Decency Act, shields it from liability for third-party content. Legal experts say that argument is shaky, as Section 230 does not protect platforms from state enforcement actions targeting illegal content, particularly when the platform has been notified and fails to act.

The ads in question reportedly used AI tools to generate realistic but fake images of minors in sexualized poses, a growing problem across the industry. Researchers have flagged that such synthetic content is becoming harder to detect as generation quality improves. Meta's own transparency reports show it removed over 30 million pieces of child exploitation content in 2023, but the company has not disclosed how many AI-generated ads slipped through or how quickly they were taken down after user reports.

This is not the first time San Francisco has taken on Meta over harmful content. In 2022, the city sued Meta for allegedly failing to protect young users from drug dealers on its platforms. That case is still pending, and this new demand could add another legal front. If Meta fails to respond adequately, the city attorney could escalate to a subpoena or file a lawsuit, potentially forcing the company to overhaul its ad screening processes for AI-generated content.

The broader implication is that tech platforms are now accountable not just for user posts, but for algorithmically generated media that mimics real people. Unlike traditional CSAM, which involves actual victims, AI-generated content creates victims out of thin air, yet it still normalizes exploitation and can desensitize viewers. Advocates argue that platforms need to invest in provenance detection and watermarking standards, but Meta has been slow to adopt such measures industry-wide.

What remains unclear is whether Meta can prove it has taken reasonable steps to prevent these ads, or whether the city will force a public reckoning over the limits of automated moderation. The company has not yet announced any new tools or policy changes in response to the demand. With the 30-day clock running, the next move could set a precedent for how cities regulate AI-generated harmful content on social media.