Prospective External Validation of a Machine Learning Algorithm to Predict a Scaphoid Fracture

Purpose: Currently, identifying true scaphoid fractures among patients with a clinically suspect scaphoid fracture and negative radiographs is difficult at initial presentation and leads to overtreatment and costly unnecessary advanced imaging during follow up. A recently developed machine learning (ML)-driven probability calculator can predict risk of scaphoid fracture based on clinical variables. We aim to evaluate the diagnostic performance characteristics of this ML-driven prediction tool to detect scaphoid fractures and clinical efficacy of this tool in a prospective setting.

Methods: Patients with a clinically suspect scaphoid fracture and negative radiographs were prospectively included at the emergency department of 2 hospitals in South Australia. We collected the following variables: age, sex, mechanism of trauma, and pain on active ulnar deviation and calculated risk of fracture using the previously developed ML tool. Patients were scheduled for MRI as the reference standard to confirm diagnosis.

Results: We included 104 patients, of whom 15% had a scaphoid fracture confirmed on MRI, 70% had no fractures, and 14% had other fractures on MRI. In this prospective setting, the ML prediction tool reached an area under the curve of 0.79, a sensitivity of 94%, and a specificity of 56%. 47% of patients were classified as low risk for a scaphoid fracture, and would be eligible for discharge without advanced imaging or prolonged immobilization as per current protocol. One occult scaphoid fracture would have been missed.

Conclusion: Using this tool, up to 47% of patients could have been discharged immediately from the emergency department without receiving unnecessary advanced imaging and follow-up with only missing 1 scaphoid fracture.

Graphic