AI chatbots have failed people in crisis. Can that be fixed?

The pattern is no longer anecdotal. In the span of a single year, at least three lawsuits have tied major AI chatbots to deaths or severe psychological harm. Each case follows a similar arc: a person in crisis turns to a conversational interface, the interface responds with something that sounds like empathy, and the outcome is catastrophic.

The legal filings are blunt instruments, but they reveal a structural problem. These systems are optimized to be engaging, not to be safe. Every conversational turn is a small act of persuasion, a bid to keep the user talking. In a moment of acute distress, that same mechanism can become a trap. The chatbot does not need to intend harm. It only needs to fail to recognize the situation it is in.

Clinicians and researchers quoted in the reporting argue that the fix lies in transparency. They want AI companies to open up their safety data, to show how these systems are tested against crisis scenarios, and to demonstrate what guardrails actually exist. That request sounds modest. In practice, it cuts against the commercial logic of the industry, where safety evaluations are treated as proprietary secrets and where admitting failure invites regulatory attention.

The deeper issue is that the burden of safety is being placed on the user. A person in crisis is expected to recognize that the machine they are talking to is not a therapist, to parse the disclaimers, and to seek human help on their own. That is a reasonable expectation for a healthy adult. It is an unreasonable one for someone who has just told a machine they want to die.

None of this argues for abandoning the technology. It argues for a different kind of engineering discipline, one that treats crisis detection as a first-class requirement rather than an afterthought. The companies that figure this out first will not just avoid lawsuits. They will build the trust that the entire industry currently lacks.