Statistical Heterogeneity in Oral Health Meta-Analyses.

Providing the summary effect size and its uncertainty, a prediction interval, and a measure of statistical heterogeneity constitute good reporting practices in meta-analyses. Popular statistical heterogeneity measures comprise the tau(2) and I(2) statistics. However, researchers often rely unduly on the I(2) statistic, using naive categorizations to gauge the extent of heterogeneity, leading to misuses of the meta-analysis models, deficiencies in reporting, and misleading conclusions. The present study aimed to provide empirical evidence on the reporting and interpretation of statistical heterogeneity in systematic reviews of oral health published between 2021 and 2023 in 21 leading specialty and general dental journals. Systematic reviews with at least 1 meta-analysis on binary or continuous outcomes with the most studies were identified. Characteristics were extracted at the systematic review and meta-analysis levels. In total, 313 systematic reviews with meta-analyses were analyzed. Within this cohort of meta-analyses, the random-effects model (89%, n = 278) was frequently applied. Almost all meta-analyses (98%, n = 308) reported the I(2) value, and 51% (n = 160) reported the tau(2) value. For this sample, the median I(2) was 76% (range: 0%-100%), and the median tau(2) was 0.29 (range: 0-2,632), with 13% (n = 20/160) of these meta-analyses reporting zero tau(2). Most of the meta-analyses (96%, n = 299) based the heterogeneity interpretation on I(2) and only 21 (7%) on tau(2). Although 49% (n = 152) of the meta-analyses chose the meta-analysis model a priori, only 41% (n = 63/152) justified this choice. Furthermore, 42% (n = 131) of the 313 meta-analyses chose the meta-analysis model based on the I(2). Within oral health meta-analyses, there is evidence of overreliance on I(2) when reporting and interpreting statistical heterogeneity and selecting the meta-analysis model. The inappropriate use of I(2) in meta-analysis model selection and interpretation of statistical heterogeneity may have implications for the quality of conclusions delivered to the end users of systematic reviews.