Evaluation of the Interobserver and Intraobserver Reliability of the New 2018 AO/OTA Classification for Pelvic Ring Injuries

Purpose: A fracture classification system must be a reliable and reproducible means of communication between different observers. This study aimed to assess the intra- and interobserver agreement and reliability of the 2018 AO/OTA classification for pelvic ring injuries (PRIs).

Methods: We designed a reliability study, with a calculated sample size of 64 cases. A pelvic trauma surgeon retrospectively selected a representative sample of the new AO/OTA 2018 classification for PRI from our hospital database. At least 1 of each subgroup of the classification was included (23 subgroups). This surgeon did not participate as an evaluator. Four fellowship-trained pelvic orthopaedic trauma surgeons acted as evaluators. Each case included a series of AP, inlet, and outlet pelvis radiographs and axial, sagittal, and coronal CT views. The sample was reviewed on 2 occasions, separated by a period of 3 months, to determine intraobserver reliability. The sample was randomized on each occasion to avoid memory bias. We used the first observation of each evaluator to assess interobserver reliability. Intra- and interobserver reliability was calculated using the percentage of agreement. The reliability of the classification was assessed using kappa coefficients with a 95% confidence interval [CI]. The agreement was interpreted as very good (κ = 1-0.8); good (κ = 0.80-0.61); moderate (κ = 0.60-0.41); weak (k = 0.4-0.21); and poor (κ = 0.2-0).

Results: Intraobserver reliability of the PRI classification was concordant in 49% of cases when analyzed by group and 41% of the subgroups. The corresponding kappa coefficients were 0.35 (CI 0.25-0.46) for groups and 0.36 (0.29-0.42) for subgroups. There was interobserver agreement in 44% of the cases for the group classification and in 47% for the subgroup classification. The corresponding kappa coefficients were 0.37 (0.28-0.46) and 0.33 (0.25-0.40).

Conclusion: Intra- and interobserver reliability using the new 2018 AO classification for PRI is weak among pelvic orthopaedic trauma surgeons when analyzing groups and subgroups. Our results suggest that communication between surgeons may not be entirely reliable and reproducible using this classification.