Authors: Cyrus Elahi; Thiago Rocha, PhD; Anthony Fuller; Catherine Staton; Joao Vissoci, PhD; Michael Haglund, MD, PhD (El Paso, TX)
Health facilities in low and middle income countries (LMICs) could benefit from decision support technologies to reduce time to diagnosis and treatment for patients with traumatic brain injury (TBI). CRASH and IMPACT are robust examples of TBI prognostic models. Despite the strengths of these two models, advanced statistical techniques and improved data quality in LMICs provide an opportunity to develop more accurate, and context specific, prognostic models. We developed a machine learning-based prognostic model using a TBI registry from a hospital in Tanzania. In this study, we compare the performance of our model against CRASH and IMPACT.
We used the CRASH and IMPACT online risk calculators to generate risk scores for each patient in a TBI registry from a regional referral hospital in Moshi, Tanzania. We compared the discrimination (area under the curve [AUC]) and calibration (agreement between predicted and observed outcomes) for CRASH, IMPACT, and our model. We calculated the AUC using Youden’s index and provided the 95% confidence interval (CI). The outcome of interest was unfavorable in-hospital outcome defined as a Glasgow outcome scale score of one, two or three.
We used a 3138 patient TBI registry for the three model comparison. There was an 11% observed unfavorable outcome rate. The AUC for our model, CRASH and IMPACT was 90.3 (CI: 88.6, 92.1), 85.8 (CI: 83.3, 88.3) and 82.0 (CI: 79.3, 84.7), respectively. The interquartile range for predicted risk scores were 10-36% (median = 16%) for our model, 5-14% (median = 5%) for CRASH, and 15-32% (median = 21%) for IMPACT.
Our model had better discrimination and similar calibration compared to CRASH and IMPACT models. This finding supports the hypothesis that locally derived prognostic models will outperform imported prognostic models. Further work is needed to externally validate our model.