metaselection
This package is under development.
Selective reporting occurs when statistically significant, affirmative results are more likely to be reported (and therefore more likely to be available for meta-analysis) compared to null, non-affirmative results. Selective reporting is a major concern for research syntheses because it distorts the evidence base available for a meta-analysis, skewing meta-analytic averages toward favorable findings and misrepresenting the true population of effects. Failure to account for selective reporting can lead to inflated effect size estimates from meta-analysis and biased estimates of heterogeneity, making it difficult to draw accurate conclusions from a synthesis.
There are many tools available already to investigate and correct for selective reporting. Widely used methods include: graphical diagnostics like funnel plots, tests and adjustments for funnel plot asymmetry like trim-and-fill, Egger’s regression, PET/PEESE, selection models, and p-value diagnostics. However, very few methods for investigating selective reporting can accommodate dependent effect sizes. This limitation poses a problem for meta-analyses in education, psychology and other social sciences, where dependent effects are a common feature of meta-analytic data.
Dependent effect sizes occur when primary studies report results for multiple measures of an outcome construct, collect repeated measures of an outcome across multiple time-points, or involve comparisons between multiple intervention conditions. Ignoring the dependency of effect size estimates included in a meta-analysis leads to overly narrow confidence intervals, hypothesis tests with inflated type one error rates, and incorrect inferences. Pustejovsky, Citkowicz, and Joshi (2025) developed and examined methods for investigating and accounting for selective reporting in meta-analytic models that also account for dependent effect sizes. Their simulation results show that combining selection models with robust variance estimation to account for dependent effects reduces bias in the estimate of the overall effect size. Combining the selection models with cluster bootstrapping leads to confidence intervals with close-to-nominal coverage rates.
The metaselection package provides an implementation of several meta-analytic selection models. The main function, selection_model(), fits step function and beta density selection models. To handle dependence in the effect size estimates, the function provides options to use cluster-robust (sandwich) variance estimation or cluster bootstrapping to assess uncertainty in the model parameter estimates.