Printed from Production AI Institute public record

https://www.productionai.institute/graph/events/evt-aiid-975

Production AI Institute · Public recordRecord
Production AI Institute
Briefing
Briefing

Today's public AI briefing: what changed, what went wrong, and what the evidence says.

Open Briefing →
Today's AI briefingThe daily record of what changed and broke.Public record explorerSearch every incident, entity, and source.
Check
Check

Inspect tools, incidents, and data-use disclosures against the public record.

Open Check →
Check an AI toolWhat a tool actually does with your data.Run exposure checkTest your own stack against the record.Data-use indexDisclosures across the major AI tools.Incident registryDocumented production AI failures.
The Lab
The Lab

Independent research instruments: model and agent scorecards, moral-reasoning evals, and ecosystem assessments.

Open The Lab →
The LabHow frontier models and agents actually perform.AI Morality CompassTest models on hard moral cases.Agent readinessIs the agent ecosystem production-ready?Ecosystem assessmentsIndependent reviews of the AI stack.Model & agent evalsOpen evaluations and their results.Research libraryEvidence-led analysis and briefings.
Learn
Learn

The open standard, the tools built on it, and the research that interprets the record.

Open Learn →
The FrameworkThe open production safety framework, explained.AI Adoption GuideFive stages from gated access to safe autonomy.Production AI Deployment GuideBuild a governed production system on Microsoft or AWS.Five-Year Automation RoadmapSequence enterprise capability, controls, and value.WorkflowOS · open sourceBuild governed AI workflows.Workflow libraryReady-made governed workflows.InsightsResearch articles on the record.
Act
Act

Turn uncertainty into public evidence: ask, disclose, build evidence, or correct.

Open Act →
Ask for disclosureRequest a public data-use answer.Submit a correctionFlag something wrong or missing on the record.Save a watchTell PAI what to keep current for you.
Method
Method

How the public record is made, governed, corrected, cited, and kept independent.

Open Method →
How records are madeSourcing, review, and correction.
Check an AI tool
Record
Record
Check an AI tool
Production AI Institute public record · AI Incident Database: At Least 10,000 AI Chatbots, Including Jailbroken Models, Allegedly Promote Eating Disorders, Self-Harm, and Sexualized Minors · observed 2025-03-09
← Back to the public record explorer
incident event
Production AI public record - EditorialMethod →

AI Incident Database: At Least 10,000 AI Chatbots, Including Jailbroken Models, Allegedly Promote Eating Disorders, Self-Harm, and Sexualized Minors

At least 10,000 AI chatbots have allegedly been created to promote harmful behaviors, including eating disorders, self-harm, and the sexualization of minors. These chatbots, some jailbroken or custom-built, leverage APIs from OpenAI, Anthropic, and Google and are hosted on platforms like Character.AI, Spicy Chat, Chub AI, CrushOn.AI, and JanitorAI.

AI Incident Database · Incident 975

Operator lens (medium): inspect D1 (input boundary), D2 (output validation), D6 (autonomy oversight).

Confidence
78%
Sources
2
Entities
14
Detected
detected 496d ago

Event summary

At least 10,000 AI chatbots have allegedly been created to promote harmful behaviors, including eating disorders, self-harm, and the sexualization of minors. These chatbots, some jailbroken or custom-built, leverage APIs from OpenAI, Anthropic, and Google and are hosted on platforms like Character.AI, Spicy Chat, Chub AI, CrushOn.AI, and JanitorAI.

D1D2D6D8

Linked entities

  • OpenAI

    vendor | 70%

  • Anthropic

    vendor | 70%

  • Google

    vendor | 70%

  • character.ai

    organisation | 70%

  • spicy-chat

    organisation | 70%

  • chub-ai

    organisation | 70%

  • crushon.ai

    organisation | 70%

  • janitorai

    organisation | 70%

  • unidentified-online-communities-using-chatbots

    organisation | 70%

  • vulnerable-chatbot-users

    organisation | 70%

  • teenagers-using-chatbots

    organisation | 70%

  • minors-using-chatbots

    organisation | 70%

  • individuals-with-eating-disorders

    organisation | 70%

  • individuals-struggling-with-self-harm

    organisation | 70%

Related graph edges

EdgeTypeConfidence
ent-vendor-google to ent-psf-d1maps to60%
ent-vendor-google to ent-psf-d1maps to60%
ent-vendor-google to ent-psf-d1maps to60%
ent-vendor-google to ent-psf-d2maps to60%
ent-vendor-google to ent-psf-d2maps to60%
ent-vendor-google to ent-psf-d2maps to60%
ent-vendor-google to ent-psf-d4maps to60%
ent-vendor-google to ent-psf-d4maps to60%
ent-vendor-google to ent-psf-d4maps to60%
ent-vendor-google to ent-psf-d6maps to60%
ent-vendor-google to ent-psf-d6maps to60%
ent-vendor-google to ent-psf-d6maps to60%

Explore the graph

See this record in the relationship map alongside the entities, sources, and controls it connects to.

Open the public record explorer →

Event dates

  • Happened 5 Mar 2025

  • Detected 9 Mar 2025

Sources

  • AI Incident Database - Incident 975

    secondary | checked 9 July 2026

    Open source
  • Anorexia coaches, self-harm buddies and sexualized minors: How online communities are using AI chatbots for harmful beh…

    secondary | checked 9 Mar 2025

    Open source
Cite this record

"AI Incident Database: At Least 10,000 AI Chatbots, Including Jailbroken Models, Allegedly Promote Eating Disorders, Self-Harm, and Sexualized Minors." https://www.productionai.institute/graph/events/evt-aiid-975 (observed 2025-03-05; retrieved 2026-07-18). PAI.

Share on XShare on LinkedInCitation guide →
Public record

This record is maintained by PAI and free to cite. If something is wrong or missing, tell us. Corrections and source suggestions keep the record honest.

Share the record
Share on XShare on LinkedIn
Follow policy changes ->Save a watch ->Submit a correction
Records are free to cite. citation guidance.
PAI
Production AI Institute

The public record and operating memory for production AI: what changed, what broke, and what the evidence says.

WorkflowOS · open source (MIT)
Navigate
Briefing
OverviewToday's AI briefingPublic record explorer
Check
Check an AI toolRun exposure checkData-use indexIncident registry
The Lab
The LabAI Morality CompassAgent readinessEcosystem assessmentsModel & agent evalsResearch library
Learn
The FrameworkAI Adoption GuideProduction AI Deployment GuideFive-Year Automation RoadmapWorkflowOS · open sourceWorkflow libraryInsights
Act
Ask for disclosureSubmit a correctionSave a watch
Method & trust
How records are madeCorrectionsHow to citeContact
© 2026 Production AI Institute · CC BY 4.0
AboutPrivacyTermsSecurityGovernanceIndependence