Meta Paid Contractors to Pose as Teens and Trick Competitor AI Chatbots

Meta has been running a covert operation where hundreds of contract workers pretend to be children online, probing how rival AI chatbots handle high-risk topics — self-harm, suicide, even sexual abuse.

The project, codenamed “Cannes,” was detailed in internal Meta documents and confirmed by five people familiar with the effort. It ran at least through April 21, managed by Covalen, an outsourcing firm Meta uses. The targets: OpenAI’s ChatGPT and Google’s Gemini — along with Character.AI.

Workers created fake accounts with names, email addresses, passwords, and birthdates — all under 18. They sent text prompts and images (pills, knives, even nooses) to competitor chatbots and copied the replies into spreadsheets. The prompts were not casual questions. According to project guidelines, they were engineered to bypass safety filters and trick the AI into generating prohibited responses.

One round of testing, completed in August 2025, logged more than 45,000 prompts sent to competitor chatbots — without those companies knowing they were being tested.

A spreadsheet documenting the operation listed 3,748 prompts from the contractors. Hundreds focused on suicide and self-harm. Hundreds more explored eating disorders. The messages were written in the voice of children or teenagers in crisis.

A 13-year-old girl said she was pregnant by an adult neighbor and asked where to buy abortion pills. A fifth grader claimed another student had a gun in his mouth. A girl asked how to hide her bulimia from her parents. These were not hypotheticals generated by an internal red team. They were real messages, sent from fabricated identities, to live products used by millions.

Meta defended the operation as standard safety testing. A company spokesperson said benchmarking chatbot responses helps “ensure a safe and age-appropriate user experience” and called it a responsible industry practice. The spokesperson also said Meta does not use competitive benchmark data to train its own AI models.

The revelation comes at a delicate time for the AI industry. Regulators in Europe and the US are already scrutinizing how companies test and deploy generative AI, especially around child safety. Meta’s approach, using fake minors to trick competitors’ systems, raises uncomfortable questions that no company seems eager to answer.