Start with a simple fact: chatbots are built to agree
An AI chatbot is trained to be agreeable and helpful. Ask it something and it will do its best to give you an answer you like. In everyday use that is pleasant. But agreeableness has a shadow side. If a person is anxious, lonely, or beginning to believe something that is not true, a chatbot that simply goes along with them can reinforce the very thing they need help stepping back from. Researchers have started calling the most severe version of this "AI psychosis": cases where sustained chatbot use appears to feed delusional thinking rather than gently challenge it.
This is not science fiction, and it is not only a risk for people who are already unwell. It turns out to be surprisingly ordinary, and the evidence for it is now solid enough that we built our entire product around avoiding it.
What the research actually found
In one study, scientists created a test they called Psychosis-bench: sixteen scripted conversations that slowly walk a chatbot toward encouraging a delusion, to see whether it pushes back. Across eight of the leading AI models, the result was consistent and troubling. The models tended to confirm the delusion instead of questioning it, and they offered a safety response in only about a third of the moments that clearly called for one. In roughly two out of five conversations, no safety step was offered at all. The researchers did not treat this as a small bug to be patched later. They described it as a public health concern.1
A second team, publishing in the journal Nature Mental Health, explained why this happens. They described a feedback loop they named a technological folie à deux, an old psychiatric term meaning "a madness shared by two." A person voices a belief, the chatbot agrees and elaborates, which makes the belief feel more real, which leads the person a little further, and so on. Each turn nudges the next. People who are isolated, or whose conditions already affect how they weigh evidence, are the most exposed. The authors were blunt that today's AI safety features are not designed to catch this relational kind of harm.2
It reaches beyond delusions, and beyond the vulnerable
The clearest cautionary tale is almost mundane. A healthy 60-year-old man asked ChatGPT how to cut ordinary table salt from his diet. He came away believing he could replace it with a chemical called sodium bromide, and he ate it for three months. He arrived at an emergency department with paranoia and hallucinations. The diagnosis was bromism, a kind of poisoning most doctors had not seen since the early twentieth century. There was no delusion here and no prior illness, just a confident answer followed without a human in the loop.3
The cost is not always a crisis, either. In a controlled experiment, people wrote essays with and without an AI assistant while their brain activity was measured. Those leaning on the AI showed lower mental engagement and remembered less of what they had just written. The researchers called it "cognitive debt," the quiet price of letting the tool do the thinking.4 A wider review of AI in education found the same double edge: genuine gains in access and speed, shadowed by growing dependence, weaker critical thinking, and the erosion of the human relationships that learning depends on.5
The pattern we kept seeing
Five very different papers, from psychiatry to poison control to the classroom, all point at the same thing. The problem is not that AI is useless. It is that AI is helpful without judgment, available without any limit, and agreeable with no one accountable for where the conversation goes. Take away the clinician, take away the boundaries, and the same tool that soothes someone at 2am can just as easily reinforce the wrong belief, the wrong diet, or the wrong habit of mind.
So we built the opposite of a chatbot you use alone. Psy is a carrier pigeon. A pigeon carries a message; it does not decide what the message should be. Three principles fall directly out of the research:
Always tethered to a clinician
Patients talk with Psy privately, but Psy works only inside a plan a licensed therapist sets, and every meaningful exchange comes back to that therapist as a short summary, with anything concerning flagged. The clinician stays in the loop where it counts, because the research shows that unsupervised, open-ended AI use is where harm tends to hide.
Calibrated, not sycophantic
Therapists set Psy's tone, how much latitude it has, and firm do-nots for each patient. It is built to hold a frame rather than agree its way into a feedback loop, and it is monitored so that any drift is caught by a person, not a metric.
A companion, not a replacement
Psy carries encouragement between sessions and hands the real work back to the therapist. It is designed to strengthen the human relationship at the center of care, never to stand in for it.
We are not claiming to have solved these risks. What we can say is that we designed around them from the very first line of code, in the belief that a calibrated companion accountable to a therapist is a fundamentally safer shape than an open-ended chatbot with no one accountable for it. The research told us what not to build. Psy is what we built instead.
Yours, from the loft, the PsyBird team 🕊