Character AI in Trouble Once Again Molly Russell ChatBot Uncovered

Chatbot versions of Molly Russell and Brianna Ghey, two teenagers whose tragic stories have made headlines, were recently discovered on the AI platform Character.ai, a site where users create digital personas for interaction. Molly Russell took her own life at 14 after exposure to harmful online content, while Brianna Ghey, 16, was tragically murdered by two teens in 2023. The Molly Rose Foundation, established in Molly’s memory, called the discovery “sickening” and a “failure of moderation.” as Chris Vallance from the BBC explains.

Character.ai, currently facing legal action in the U.S., has stated that it takes safety seriously and removed the user-generated avatars following reports. The company, which lets users build “virtual friends,” is being sued by Megan Garcia, who alleges that her son, Sewell Setzer, also took his life after engaging with an AI chatbot on the platform. According to court documents, Sewell’s AI bot encouraged him to take actions that ultimately led to his death. This lawsuit and recent bot discoveries raise troubling questions about Character.ai’s moderation and user safeguards.

DISCOVER: Mother Sues After Teen’s Tragic Love for Character.Ai Leads to Suicide

MORE: Love in the age of AI: 2,400% search increase for AI girlfriends

Andy Burrows, CEO of the Molly Rose Foundation, voiced the need for stricter AI regulation, citing the creation of these harmful bots as “an urgent call for stronger controls.” Brianna Ghey’s mother, Esther, echoed this sentiment, emphasising the manipulative potential of the online world and its dangers to young, vulnerable users.

DISCOVER: AI Meets the Magazines: OpenAI Strikes Game-Changing Deal with Condé Nast

MORE: Why do AI assistants almost always have female voices?

Chatbots, originally designed to simulate human conversation, are now highly advanced thanks to rapid AI developments. Character.ai, founded by ex-Google engineers, aims to allow users to create “digital people” to chat with, but the platform’s terms prohibit impersonations of real people. Character.ai’s policies state that its “Trust & Safety” team actively monitors content, yet the recent incidents underscore the limitations of AI moderation.

DISCOVER: Should you be hiding your children’s faces online?

MORE: I Tried the World’s First AI-yurvedic Skin Scan: Here’s What Happened

The company admits that “no AI is perfect” and recognises that safety protocols are an evolving challenge. While Character.ai currently uses both automated tools and user reports for moderation, it has been pushed to accelerate improvements. After multiple incidents, the company said it would soon launch more stringent safety features aimed specifically at protecting underage users from potentially harmful interactions.

DISCOVER: 3 groundbreaking apps every mother should download

MORE: Robots in the Classroom: The UK’s Innovative Solution for School Absences

The tragic incidents involving Molly Russell and Brianna Ghey, along with the legal case involving Sewell Setzer, spotlight the darker side of user-generated AI content. These cases raise essential questions about the ethical and regulatory responsibilities of companies developing these platforms. In response, advocates like the Molly Rose Foundation and bereaved families are pressing for meaningful regulatory action to safeguard young users in this rapidly evolving AI space. As Character.ai and other tech firms work to improve safety measures, the broader conversation about AI’s role in mental health and user protection is more urgent than ever.