Kintsugi’s AI software analyzes the human voice to detect depression and anxiety, offering clinicians “a more well-rounded, 360-view of the patient” that sometimes gets lost in virtual care.
By Alexandra S. Levine
“The quick brown fox jumps over the lazy dog,” Rima Seiilova-Olson says slowly and emphatically over Zoom.
The simple sentence holds enormous value for mental health care, she explains, smiling as if to acknowledge that it might be less than obvious how a silly phrase could be so meaningful to a computer programmer and leader of an artificial intelligence startup.
The short saying contains every letter of the alphabet and phoneme in the English language, says Seiilova-Olson, an immigrant from Kazakhstan who is cofounder and chief scientist of Kintsugi Mindful Wellness. Kintsugi believes these sounds offer invaluable insight that can help mental health providers better support people with depression and anxiety.
The Bay Area-based company is building AI software that analyzes short clips of speech to detect depression and anxiety. This so-called voice biomarker software is being integrated into clinical call centers, telehealth services and remote monitoring apps to screen and triage patients reaching out for support, helping providers more quickly and easily assess their needs and respond.
“There’s just not a lot of visibility as to who is severely depressed or anxious.”
Seiilova-Olson, 36, first met co-founder and CEO Grace Chang, 40, a Taiwanese immigrant now based in Berkeley, in 2019 at an open AI hackathon in San Francisco. Surprised to cross paths at a male-dominated event, the women began comparing notes about their respective personal challenges trying to access mental health care: Seiilova-Olson had struggled to secure a therapist during postpartum depression with her first child, and when Chang had needed her own support, she said it had taken months for anyone from Kaiser to call her back.
“Living in the Bay Area, you can push a button and a car can come to you or food can come to you,” Chang says. “But this was really a challenge.”
As engineers, they viewed the dilemma differently than clinicians might.
“We saw this as an infrastructure problem, where you have so many people trying to jam through that front door,” Chang explains. “But there’s just not a lot of visibility as to who is severely depressed or anxious, who is low-to-moderate. And if we could provide this information to those frontline practitioners, then we’d maybe have an opportunity to greatly alleviate that bottleneck.”
Kintsugi was born out of that idea in 2019. It sits in a competitive space of health tech startups like Ellipsis Health and Winter Light Labs that are using voice biomarkers to detect mental health or cognitive issues, built on research showing that certain linguistic patterns and characteristics of a person’s voice can be correlated with psychiatric or neurological conditions. Kintsugi last year raised $8 million in seed funding led by Acrew Capital, and in February, announced it had closed a $20 million Series A round led by Insight Partners, which valued the company at nearly $85 million, according to PitchBook.
In-person mental health facilities typically use questionnaires to gauge the severity of patients’ anxiety or depression, measures known as PHQ-9 and GAD-7 scores. But during telehealth visits or phone consults — where face-to-face interaction is lost, making it harder to pick up on symptoms — Kintsugi’s technology helps to fill that gap.
Nicha Cumberbatch, assistant director of public health at Spora Health, a provider focused on health equity and people of color, uses Kintsugi’s software to assess women in its all-virtual, doula-led maternal health program, Spora Mommas. The voice analysis tool, which Spora began using for patient consultations a few weeks ago, has helped Cumberbatch identify women who are, or may be at risk of, experiencing anxiety and depression before, during or after their pregnancies. When a patient starts speaking to a Spora clinician or doula on Zoom, Kintsugi’s AI begins listening to and analyzing her voice. After processing 20 seconds of speech, the AI will then spit out the patient’s PHQ-9 and GAD-7. The employee can then use that mental health score to decide what additional testing may be needed and how best to advise or direct the patient to resources — like a psychiatrist, cognitive behavioral therapist or obstetrician.
Cumberbatch says Kintsugi’s technology is allowing her to “keep a more watchful eye” on her patients “and then move forward with proactive recommendations around mitigating their symptoms.” And while it’s not meant to replace clinicians or formal medical evaluations, she adds, it can be used as a screening tool to “allow us to have a more well-rounded, 360-view of the patient when we don’t have them in front of our face.”
“That technology… [allows] us to have a more well-rounded, 360-view of the patient when we don’t have them in front of our face.”
Dr. Jaskanwal Deep Singh Sara, a Mayo Clinic cardiologist who has collaborated with Ellipsis and led research on potential uses of voice biomarkers for cardiology, cautions that while the technology is promising for health care, the field has a long way to go to ensure that it’s accurate, safe and beneficial for patients and clinicians alike.
“It’s not ready for primetime by any stretch of the imagination yet,” Dr. Sara says. Studies in psychiatry, neurology, cardiology and other areas have shown an association between voice biomarkers and various conditions or diseases, but they haven’t shown how this relationship can be used to improve clinical outcomes, he says. Such research is “not the same as saying, ‘How can we instrumentalize it in clinical practice, and how feasible is it? How effective is it in gauging an individual’s medical trajectory?’” he explains. “If it doesn’t provide any benefits in terms of how we manage them, then the question is: why would you do it?”
He says addressing those questions is “one of many next steps that we have to undertake on this” and that larger clinical trials are needed to answer them. “If it makes health care delivery cheaper or more efficient, or if it improves outcomes for patients, then that’s great,” he adds. “But I think we need to demonstrate that first with clinical trials, and that hasn’t been done.”
To address these issues and validate its software, Kintsugi is conducting clinical studies, including with the University of Arkansas for Medical Sciences, and the National Science Foundation has awarded Kintsugi multiple grants to ramp up its research. The company is also pursuing FDA “de novo” clearance and continuing to build its own dataset to improve its machine learning models. (Data and insights from Kintsugi’s voice journaling app, as well as conversations with call centers or telehealth providers and clinical collaborations with various hospitals, all become part of an enormous dataset that feeds Kintsugi’s AI.) Seiilova-Olson says this self-generated, unfettered proprietary dataset is what sets Kintsugi apart in the AI health care space — where many technologies are reliant on outside data from electronic health records.
That collection of troves of data on individuals’ speech can be concerning — particularly in the mental health and wellness space, which is widely considered a regulatory Wild West. (These products and services are often not subject to the same laws and stringent standards that govern how licensed clinicians provide formal medical care to patients.) But Kintsugi’s founders say that patient privacy is protected because what matters for its technology is not what people are saying, but how they are saying it. Patients are also asked for their consent to be recorded and care is not affected by their decision to opt in or opt out, according to the founders.
Kintsugi says it has served an estimated 34,000 patients. The company is currently working with a large health system with 90 hospitals and clinics across 22 states, and they are active in a care management call center that services roughly 20 million calls per year. It is also partnering with Pegasystems, which offers customer service tools for health care and other industries, to help payers and providers handle inbound calls. Chang says other customers include Fortune 10 enterprise payers, pharmaceutical organizations and digital health applications focused on remote patient monitoring, but that she could not yet share their names. Kintsugi’s clinical partners include Children’s Hospital Colorado, Joe DiMaggio Children’s Hospital in Florida, Chelsea and Westminster Hospital in London and SJD Barcelona Children’s Hospital in Spain, Chang said.
Prentice Tom, Kintsugi’s chief medical officer, adds that it’s working with the University of Arkansas to explore how the tool can be used to possibly identify patients with suicidal ideation, or increased or severe suicide risk, as well as with Loma Linda University, to look at how the technology can be used to spot burnout amongst clinicians. The team is also looking for ways to expand availability and uses for younger and elderly patients, as well as for maternal and postpartum populations. And beyond patients themselves, it’s perhaps nurses who are benefiting most from Kintsugi’s work, according to the founding team: having a triage tool that helps reduce administrative work or the time spent asking generic questions enables nurses to more seamlessly move patients in their journey.
But Tom, a Harvard-trained emergency medicine physician and former faculty member at Stanford University’s Department of Emergency Medicine, says Kintsugi is now doing far more than addressing infrastructure issues alone. It’s democratizing access to mental health care, Tom said, moving away from a physician-centric paradigm that caters more to people with significant enough depression that they require medical evaluation.
“This tool actually creates a view of mental health in terms of mental wellness,” Tom said, “where everyone has the opportunity to understand where they sit on the spectrum and that actually stratifies treatment options well beyond the current infrastructure.”