EXP × LLM
Overview
Therapists and researchers use something called the Experiencing Scale (EXP) to measure how deeply a patient is engaging with their own emotional experience during therapy sessions. Scoring transcripts with this scale requires trained human raters, which is time-consuming and expensive.
Working alongside Brian Yim, we want to know whether large language models can do this reliably — and how their scores compare to those of expert human raters. The results will be used to evaluate inter-rater reliability between AI and human expert raters, with findings intended for peer-reviewed publication.
My Contributions
HIPAA-Compliant De-Identification Pipeline
Built from scratch in Python using Microsoft Presidio and spaCy — automatically strips all 18 Safe Harbor identifiers from 55 clinical transcripts, ensuring full patient privacy compliance before any data touches an LLM.
LLM Scoring Pipeline
Engineering the pipeline that sends de-identified transcripts through all five models (Claude, GPT-4o, Gemini, LLaMA, and Gemma), collects EXP scores, and structures the results for statistical analysis.
Cloud GPU Infrastructure
Setting up the cloud GPU infrastructure on Google Cloud to run the open-source models (LLaMA and Gemma) locally, ensuring full control over data privacy for the clinical transcripts.
Conference
Presentations
Peer-reviewed findings accepted for presentation at international psychotherapy conferences.
SPR North American Conference
Yim, B., Rodriguez, S. & Muran, J. C. (2026, accepted). Automated coding of a psychotherapy measure: Validating the use of large language models in scoring the Experiencing Scale (EXP). Society for Psychotherapy Research North American Conference, New York, NY.
24th World Congress of Psychotherapy
Yim, B., Rodriguez, S. & Muran, J. C. (2026, June). Validating the use of large language models in scoring the Experiencing Scale (EXP). 24th World Congress of Psychotherapy, New York, NY.
Interested in
This Research?
If you're working on something similar or want to collaborate, I'd love to hear from you.