Digital Behavioral Risk Stratification Tool Integrated With HIV Online Self-Testing for HIV Positivity Among Men Who Have Sex With Men: Retrospective Observational Study
Journal of Medical Internet Research ·
Background: Due to concerns about privacy breaches and the fear of social stigma, some individuals engaging in high-risk behaviors—particularly men who have sex with men (MSM)—may avoid in-person HIV testing services and instead choose to purchase HIV self-test kits online. Unlike traditional e-commerce platforms that provide self-test kits directly without conducting risk assessments, the “Easy Test Know” platform integrates a structured behavioral risk assessment prior to purchasing an HIV self-test kit, allowing users to be categorized into different HIV risk groups. Objective: This study aimed to evaluate the performance of a digital HIV behavioral risk stratification tool integrated with online self-testing among MSM and to determine whether incorporating multidimensional behavioral indicators—particularly the high-risk venue diversity score—could improve the prediction of HIV positivity compared with the platform’s existing rule-based risk-scoring algorithm. Methods: This retrospective observational study used data from the “Easy Test Know” platform between November 2023 and February 2025. The analysis included 9961 MSM classified as medium or high risk who had a usable HIV test result. The continuous platform score and the deployed categorical output were evaluated, and enhanced and fully adjusted Firth logistic regression models were compared using areas under the receiver operating characteristic curves (AUCs) with stratified bootstrap 95% CIs, DeLong tests, calibration measures, and bootstrap optimism correction based on 1000 successful model refits. Results: Among 9961 participants, 9044 (90.8%) were classified as high risk, and 58 (0.58%) had an observed HIV-positive result. Positivity was 0.6% (n=54) in the high-risk group and 0.4% (4/914) in the medium-risk group, but the difference was not statistically significant (=.70). The deployed categorical output had an AUC of 0.512 (95% CI 0.477‐0.539), with 93.1% (54/58) sensitivity and 9.2% (913/9903) specificity, whereas the continuous platform score had an AUC of 0.646 (95% CI 0.567‐0.725). Each 1-category increase in high-risk venue diversity was associated with higher odds of observed HIV positivity (odds ratio 2.04, 95% CI 1.53‐2.74; for trend <.001). The enhanced and fully adjusted models had apparent AUCs of 0.728 (95% CI 0.656‐0.801) and 0.752 (95% CI 0.686‐0.817) and optimism-corrected AUCs of 0.688 and 0.713, respectively. Their optimism-corrected calibration slopes were 0.940 and 0.936. Conclusions: Compared with the platform’s original rule-based scoring system, the data-driven behavioral models incorporating high-risk venue diversity showed improved predictive performance, although bootstrap optimism correction yielded more conservative estimates. These findings support further development and independent validation of data-driven approaches to optimize digital HIV risk assessment tools.