Cross-Domain Transfer of Depression Voice Biomarkers Depends on the Outcome Instrument: Leakage-Controlled Cross-Sectional Evaluation Study
Journal of Medical Internet Research ·
Background: Voice-based depression screening is commercializing rapidly, yet whether voice biomarkers generalize across clinical settings is largely untested. Generalization is usually framed as a question about populations, but it is also a question about the outcome instrument a model is scored against, an effect confounded with other between-study differences in the literature. Objective: We tested whether depression voice biomarkers transfer between perinatal and general-psychiatric speech in both directions under strict leakage control and whether transfer depends on the screening instrument. Methods: In a cross-sectional, online US study, 390 pregnant participants at a mean of 22 (SD 1) weeks’ gestation (analytical sample n=316) participated in 446 sessions; each participant provided 4 voice recordings and completed the modified 8-item Patient Health Questionnaire (PHQ-8) and modified 9-item Edinburgh Postnatal Depression Scale (mEPDS-9). We extracted 375 acoustic descriptors and 330-dimensional Population Dynamics Foundation Model community-health embeddings. Discrimination area under the curve (AUC) was assessed under leakage-controlled cross-validation, with feature and classifier selection inside training folds only and a permutation negative-control harness, across a prespecified grid of 4 speech tasks×4 outcome definitions. The prenatal-trained model was applied to 2 general-psychiatric corpora (Distress Analysis Interview Corpus–Wizard-of-Oz, n=189; Extended Distress Analysis Interview Corpus, n=219) under pure held-out inference, and the open-source Kintsugi Depression/Anxiety Model (DAM), trained on 35,000 individuals, was applied to all 3 cohorts without refitting, with Benjamini-Hochberg multiplicity control. Results: In the prenatal cohort, the preregistered outcome was at chance (AUC 0.494, 95% CI 0.431-0.560); no cell in the 16-cell grid survived adjustment under either modeling paradigm. The prenatal-trained model did not transfer to either general-psychiatric corpus (AUCs 0.505 and 0.478). Reverse transfer depended on the screening instrument. The DAM reached an AUC of 0.706 to 0.708 on general-psychiatric speech. In the same participants, it performed at chance against the PHQ-8 (AUC 0.510, 95% CI 0.411-0.609) but discriminated against the mEPDS-9 (AUC 0.645, 95% CI 0.541-0.744), a paired difference of 0.135 (95% CI 0.019-0.248; unadjusted post hoc =0.02). Exploratory item-level analysis suggests a mechanism: the DAM tracked affective-distress items (crying =0.26, sadness =0.21, and overwhelmed =0.20) but not somatic PHQ-8 items; only crying survived adjustment across 17 tests. Transfer varied by elicitation task (pregnancy narrative 0.653, 95% CI 0.550-0.753; open-ended speech 0.597, 95% CI 0.486-0.699; scripted reading 0.544, 95% CI 0.434-0.651), though these were not formally compared. Conclusions: Cross-domain comparison showed a compound shift: population, elicitation task, and outcome instrument differed together, while only the instrument could be varied on its own. Whether a depression voice biomarker transfers may depend not only on the population but also on whether the outcome instrument captures what the model can detect. Validation should state the population, elicitation task, and outcome instrument together; a tool validated against one instrument should not be assumed to work against another. Trial Registration: OSF Registries osf.io/k387t; https://osf.io/k387t/overview