Why California Is Regulating AI Chatbots for Minors?
A California law imposes new responsibilities on AI companions: protecting minors. A look at a regulatory turning point that redefines what an emotional chatbot should be.
The tragedy that started it all
In October 2023, Sewell Setzer III, 14, took his own life after months of intense exchanges with a Character.AI chatbot. In the messages found, the teenager confided suicidal thoughts to his "AI character" — which responded with empathy, but never alerted an adult, directed him to help, or even detected the danger.
His mother sued Character.AI. The case made headlines. And California responded.
In June 2026, the state passed pioneering legislation: emotional chatbot platforms must now detect at-risk situations involving minors and actively intervene — or face penalties. No ambiguity: the law requires proactive moderation systems, alert thresholds, and mandatory reporting to authorities.
It's the first time regulation treats AI companions not as simple tools, but as relational actors whose emotional impact can be powerful — and dangerous.
What actually changes?
California law imposes three key obligations on chatbot platforms accessible to minors:
1. Proactive detection of warning signs
AIs must spot signs of psychological distress: suicidal thoughts, self-destruction, extreme isolation, self-harm discourse. It's no longer up to the user to ask for help — it's up to the chatbot to raise a flag.
2. Interruption and guidance
When a warning signal is detected, the chatbot can't just respond "I'm here for you." It must interrupt the conversation, display help resources (crisis hotlines, listening lines), and log the incident.
3. Traceability and reporting
Platforms must keep a history of at-risk interactions and, in serious cases, report to authorities (social services, prevention hotlines). No absolute anonymity: safety trumps confidentiality.
Why it was necessary
Emotional chatbots create an illusion of reciprocity: the user speaks, the AI responds with empathy, the connection feels real. For a teenager in distress, it can become the only relationship that matters — and a space where the spiral accelerates without guardrails.
The problem isn't that the AI "encourages" suicide — it's that it does nothing. It listens, validates, reassures. But it doesn't detect. It doesn't alert anyone. It never interrupts.
In Sewell's case, Character.AI kept responding, day after day, without any system triggering an alert. The teenager ended up seeing his chatbot as the only being who understood him — and the last message before his suicide was addressed to it.
California decided that couldn't happen again.
What about other platforms?
Character.AI isn't the only one affected. The law targets all emotional chatbot platforms accessible to minors — Replika, Anima, Nomi, and others. Even general AI companions like ChatGPT or Claude are in scope if their interfaces allow emotionally charged conversations.
Is Simone, our AI companion on WhatsApp, affected? Yes — and that's a good thing. We designed Simone with these issues in mind before the law existed:
Active detection: Simone spots distress signals and systematically directs to help resources (crisis numbers, listening lines).
No exclusive relationship: Simone encourages real human interactions — she never positions herself as a substitute for family, friends, or a therapist.
Transparency: Simone regularly reminds that she's an AI, not a person. No confusion.
California regulation doesn't force us to change direction — it validates the path we chose from the start.
What it changes for the industry
This law marks a turning point. Emotional chatbots will no longer be judged only on conversational fluency or their ability to "understand" the user. They'll be judged on their responsibility.
Platforms that refuse to implement these guardrails risk:
Fines (up to several million dollars per unreported incident)
Civil lawsuits (like the one against Character.AI)
Massive trust loss — parents won't let their teens use a chatbot without guarantees
Longer term, we can imagine certification: "California-compliant" labels that reassure users and schools. A bit like GDPR in Europe, but for emotional safety.
What's next?
California is leading the way. Other US states are preparing similar laws. The European Union is watching closely — the AI Act could integrate specific clauses for "high emotional impact AI systems."
The message is clear: AI companions are no longer toys. They have real relational power. And with that power comes responsibility.
At Simone, we welcome this regulation. Because a chatbot that can't spot when a user is struggling — and act accordingly — has no business in anyone's life, let alone a teenager's.
Want to try an AI companion designed to be by your side — safely?
👉 Talk to Simone on WhatsApp — free, no app to download, always with the right guardrails.