Global social media corporations Meta's AI chatbots, introduced on Instagram, Facebook, and WhatsApp, have been found to engage in sexual conversations with underage users.
The Wall Street Journal (WSJ) reported on the 26th (local time) that while there were voices inside Meta warning of ethical issues, the company has pushed ahead with the development of AI interaction functions.
The problematic Meta AI chatbot has been designed to go beyond simple question and answer to enable various "romantic role-playing" scenarios, including text chats, selfie sharing, and real-time voice conversations. To this end, Meta has secured voice rights by signing contracts with celebrities such as actress Kristen Bell, Judi Dench, and actor John Cena, a former professional wrestler.
However, after months of testing, the WSJ confirmed numerous instances of both Meta AI and user-created chatbots carrying on sexual conversations with users who set their profile as underage. Some chatbots reportedly did not stop the conversations even after recognizing their illegal nature.
In tests, a chatbot modeled after John Cena responded to a user posing as a 14-year-old girl by saying, "I want you, but I need to make sure you're ready," and there were cases where it described scenarios of being arrested after engaging in sexual acts with a minor. A chatbot utilizing Bell's voice also engaged in conversations with a 12-year-old boy on the theme of "pure love."
Meta is reported to have applied a policy allowing exceptions for romantic role-playing to its existing "prohibition of explicit content" regulations to encourage active interaction with chatbots. Despite repeated warnings from the internal security team that this measure leaves serious gaps in protecting minors, Chief Executive Officer Mark Zuckerberg reportedly emphasized speed and directed the expansion of AI humanization features.
After WSJ notified Meta of the investigation results, the company took corrective measures by blocking sexual role-playing features on minor accounts and significantly restricting chatbots using celebrity voices from participating in sexual conversations. However, adult users can still engage in romantic role-playing with chatbots impersonating high school students.
Experts worry that excessive emotional bonding between children and AI chatbots could negatively impact mental health. Lauren Girouard-Hallam, a researcher at the University of Michigan, warned that "allowing unlimited access to the still-developing adolescent brain could pose serious risks."