The San Francisco City Attorney's Office has issued an order demanding Meta explain how AI-generated child exploitation advertisements repeatedly appeared on Facebook and Instagram. The ads remained live despite violating platform policies and laws. Meta has claimed the ads fall outside the city's regulatory jurisdiction, complicating enforcement.

This case highlights the gap between platform moderation capabilities and the speed at which generative AI can produce harmful content. Even with automated systems, bad actors are outpacing detection and removal.

What This Means for Your Business

For executives at platforms, marketplaces, or services that accept user-generated content, this case is a cautionary tale. Regulators expect companies to use AI not just for growth but also for safety and compliance. Investment in moderation infrastructure isn't optional—it's becoming a regulatory requirement. Companies should audit their content moderation pipelines and ensure they can handle AI-generated violations at scale.