Digital messaging services provide significant benefits in behavioral healthcare in terms of accessibility, affordability, and scale, while providing a complete record of interactions between clients and therapists over the course of treatment. The Talkspace platform allows therapists to scale their services to a larger and broader population, but the messaging modality can inhibit the type of high-bandwidth interactions possible during in-person sessions, where therapists may pick up on subtle cues to help make diagnoses or determine appropriate treatment options. One potential solution is to augment the therapist’s ability to give care by providing them with evidence-based suggestions and real time feedback based on machine learning models trained on annotated anonymized transcripts.
We will discuss our approach to bootstrapping and training ML models that can be used to suggest an appropriate screening or assessment to help the therapist diagnose a client’s mental health condition. We treat mental health morbidity as a multiclass text classification problem, where the labels are generated by human experts and features are derived from the client’s anonymized text messages. Our discussion highlights the importance of establishing model performance baselines using simple models and feature representations before scaling up to more complex approaches, such as deep learning. It also emphasizes the need to carefully design human-in-the-loop ML systems by considering the appropriate evaluation metrics in light of the anticipated impact of false positive and false negative errors, and establishing a desirable feedback loop that aids in