Cost and Data Quality Impact of Fraud Mitigation in Online Research: Framework Application and Evaluation Study
Journal of Medical Internet Research ·
Background: Fully remote online health research is vulnerable to fraudulent participation that can threaten data validity and divert study resources. Although fraud mitigation is increasingly necessary in online research, mitigation efforts are rarely described systematically or evaluated transparently across studies. Objective: This study demonstrates the application of the configure, assess, triage, corroborate, and hone (CATCH) framework to an ongoing remote mental health study and evaluates how structured mitigation procedures have affected participant progression and operational costs. Methods: This study was conducted within an ongoing, fully remote mental health study evaluating hallucinations through longitudinal digital data collection. Following detection of fraudulent activity, the CATCH framework was applied to organize mitigation procedures into stages and refine detection and verification workflows. New screening safeguards were configured, classification thresholds were recalibrated, enrolled participants were reassessed and triaged accordingly, and secondary corroboration through synchronous video verification was introduced when warranted. Procedures were honed through ongoing monitoring in response to emerging fraud patterns. We compared 2 temporally defined cohorts before (March-August 2025) and after (September 2025-January 2026) the implementation of the refined procedures. Primary outcomes included the stage at which fraudulent participants were identified and cost per eligible participant, decomposed into costs attributable to fraudulent participants, nonfraudulent participants, and fraud-mitigation technology. Participant-level costs were estimated using a stage-based model incorporating participant compensation and research staff effort. Sensitivity analyses examined alternative compensation and staffing scenarios to assess the generalizability of the proposed cost-accounting mechanism. Results: Following CATCH implementation, fraudulent participants were identified earlier in the study workflow, reducing progression to later study stages. Overall, 41.5% (161/388) of reassessed participants were discharged for suspected fraud, while 58.5% (227/388) were retained following adjudication. Among those flagged as high risk and invited to video verification, 1.8% (3/164) successfully confirmed their identity, providing a lower-bound estimate of the false-positive classification rate. The mean sunk cost per fraudulent participant decreased from US $12.75 (SD US $40.76) in the pre-CATCH cohort to US $3.29 (SD US $10.91) in the post-CATCH cohort. Total cost per eligible participant decreased from US $170.48 to US $134.90, driven by a substantial reduction in the fraud-related portion from US $80.46 to US $19.73. Sensitivity analyses showed reductions in fraud-related cost per eligible participant across all alternative costing scenarios. To our knowledge, this is the first empirical demonstration of a structured fraud-mitigation framework implemented and evaluated within an active health study. Conclusions: Structuring fraud mitigation using an explicit framework enabled systematic documentation of procedural adaptations and their operational consequences. Beyond study-specific findings, this work illustrates how organizing fraud mitigation into defined, reportable stages can promote transparency, reproducibility, and cumulative learning across remote health studies. Treating fraud mitigation as an integral methodological component is essential for safeguarding scientific integrity and legitimate resource expenditure in online research.
Background: Fully remote online health research is vulnerable to fraudulent participation that can threaten data validity and divert study resources. Although fraud mitigation is increasingly necessary in online research, mitigation efforts are rarely described systematically or evaluated transparently across studies. Objective: This study demonstrates the application of the configure, assess, triage, corroborate, and hone (CATCH) framework to an ongoing remote mental health study and evaluates how structured mitigation procedures have affected participant progression and operational costs. Methods: This study was conducted within an ongoing, fully remote mental health study evaluating hallucinations through longitudinal digital data collection. Following detection of fraudulent activity, the CATCH framework was applied to organize mitigation procedures into stages and refine detection and verification workflows. New screening safeguards were configured, classification thresholds were recalibrated, enrolled participants were reassessed and triaged accordingly, and secondary corroboration through synchronous video verification was introduced when warranted. Procedures were honed through ongoing monitoring in response to emerging fraud patterns. We compared 2 temporally defined cohorts before (March-August 2025) and after (September 2025-January 2026) the implementation of the refined procedures. Primary outcomes included the stage at which fraudulent participants were identified and cost per eligible participant, decomposed into costs attributable to fraudulent participants, nonfraudulent participants, and fraud-mitigation technology. Participant-level costs were estimated using a stage-based model incorporating participant compensation and research staff effort. Sensitivity analyses examined alternative compensation and staffing scenarios to assess the generalizability of the proposed cost-accounting mechanism. Results: Following CATCH implementation, fraudulent participants were identified earlier in the study workflow, reducing progression to later study stages. Overall, 41.5% (161/388) of reassessed participants were discharged for suspected fraud, while 58.5% (227/388) were retained following adjudication. Among those flagged as high risk and invited to video verification, 1.8% (3/164) successfully confirmed their identity, providing a lower-bound estimate of the false-positive classification rate. The mean sunk cost per fraudulent participant decreased from US $12.75 (SD US $40.76) in the pre-CATCH cohort to US $3.29 (SD US $10.91) in the post-CATCH cohort. Total cost per eligible participant decreased from US $170.48 to US $134.90, driven by a substantial reduction in the fraud-related portion from US $80.46 to US $19.73. Sensitivity analyses showed reductions in fraud-related cost per eligible participant across all alternative costing scenarios. To our knowledge, this is the first empirical demonstration of a structured fraud-mitigation framework implemented and evaluated within an active health study. Conclusions: Structuring fraud mitigation using an explicit framework enabled systematic documentation of procedural adaptations and their operational consequences. Beyond study-specific findings, this work illustrates how organizing fraud mitigation into defined, reportable stages can promote transparency, reproducibility, and cumulative learning across remote health studies. Treating fraud mitigation as an integral methodological component is essential for safeguarding scientific integrity and legitimate resource expenditure in online research.