A study published in late July found AI chatbots have gotten meaningfully better at one job: catching suicide risk. Real progress. But a recent APA survey found close to nine in ten psychologists still worry chatbots may inadvertently encourage self-harm, and more than a third are now seeing patients treat AI as a supplemental provider.
Here's what the headline misses. Picture an intake form asking someone to rate their mood, one to ten. They fill it out an hour after a bad moment, feeling like a two. Last night, alone, it was a seven. The form gives them one box. They write a five. That's not a lie. It's the honest answer to the wrong question.
I specialize in eating disorders. Almost nobody opens with it. The people who lead with "I haven't eaten in a week" are usually too acute for outpatient care to begin with. The clients I actually see come in for anxiety, for "life is a lot right now." My intake has a full eating disorder section. Most don't answer it honestly the first time, not because they're hiding it, but because in that first hour it doesn't feel like the thing that matters. Two or three months in, once there's trust, it comes out almost as an aside. It was on the form the whole time. It just wasn't answerable yet.
A relationship gets months to earn that disclosure. A chatbot gets one conversation.
If your product's safety story stops at "we screen for suicide risk," you've solved the part that was already easiest to solve. The quieter failure doesn't make headlines. It's the person who answers every question the way the form wanted and still isn't okay. Nothing flags that case. It just moves through the system labeled fine.
Building or reviewing patient-facing clinical tools and want a second set of eyes on where those gaps sit? DM me.
Jayme Scarfo is a Licensed Professional Counselor and Certified Eating Disorder Specialist, CAMS-Trained, who consults with digital health and behavioral health companies on clinical content review, curriculum, and patient-facing program design. She sees clients privately through Empower Counseling & Consulting, LLC, based in Arizona.