Basic Analyses

Select domain

Select dataset

Select effect size type

This output shows the results of the Random Effects Meta-Analysis for k effect sizes. K is the number of studies included in the meta-analysis. The information given is interpreted as follows:

Tau^2

Estimated between-study variance, reflects the amount of heterogeneity among the true effect sizes across studies.

Tau

Estimated standard deviation of underlying true effects across studies, can be used to describe the distribution of true effects.

I^2

Variability may occur between studies (true heterogeneity, tau^2) and within studies (sampling error). I^2 is the percentage of the total variability, that is due to true heterogeneity.

H^2

Relative excess in Q over its degrees of freedom. The ratio of the Q statistic to its degrees of freedom is interpreted as a measure of the extent of heterogeneity.

Test for heterogeneity

Cochran Q test for statistical heterogeneity tests the null hypothesis, that the underlying true effect size parameters are the same in all studies included. The test statistic Q is chi-squared distributed on k-1 degrees of freedom under the null hypothesis. If the results are statistically significant, the null hypothesis is rejected and statistical heterogeneity is expected.

Model results

The table with the model results provides the meta-analytic estimate, which is the weighted mean of the individual effect sizes, and its corresponding standard error. For means of testing the significance of the estimate, the z-statistic and its corresponding p-value are given. Finally, the 95 % confidence interval limits are given.

The output on the left shows the results of the Multilevel Random Effects Meta-Analysis for k effect sizes. K is the number of effect sizes in the meta-analysis. In the multilevel model, the dependencies in effect sizes due to emerging from the same sample or report are considered. The information given for this model is interpreted as follows:

logLik, Deviance

Deviance statistics are useful to compare a bigger model to a smaller model, nested in the bigger one (certain coefficients in the bigger model are set 0). It is hypothesized, that the coefficients of the additional predictors in the bigger model are 0, that means no improvement of the model due to these predictors.

The likelihood is a measure for the goodness of fit of a model to the data provided by a sample. The deviance statistic used for model testing is derived from the LogLikelihood of the model:

Deviance = -2 * (LogLike(smaller model) – LogLike(bigger model))

Smaller Deviance is better. It means, that adding more predictors to a model reduces deviance and improves the model fit. The deviance statistic is used to determine whether or not the reduction in deviance is significant.

AIC, BIC, AICc

These are information criteria to compare models and evaluate their stability. The general idea of these criteria is to add deviance and a penalty term. The smaller the deviance, the better the model fits the data. The penalty term addresses the complexity of the model. As the goal is a parsimonious model explaining the data with only a few predictors, lower AIC / BIC are better.

Variance components:

In the multilevel mode, the variation attributed to each analysis level ist estimated. For each level, the estimated variance and the corresponding standard error are reported. Furthermore, the number of effect sizes, samples and reports in the meta-analysis is reported in column „nlvls“.

Test for heterogeneity:

Cochran Q test for statistical heterogeneity tests the null hypothesis, that the underlying true effect size parameters are the same in all studies included. The test statistic Q is chi-quared distributed on k-1 degrees of freedom under the null hypothesis. If the results are statistically significant, the null hypothesis is rejected and statistical heterogeneity is expected.

Model results

Forest Plot

switch between normal forest plot and rainforest plot

The forest plot displays the confidence interval for each effect size included in the meta-analysis.

Smaller studies usually have wider confidence intervals, as there is more uncertainty in the estimation.

At the bottom, the summary shows the mean effect size and its corresponding confidence interval.

As for the meta-analytic estimate, information from multiple samples is used, the estimation is based on more evidence and the confidence interval is usually smaller than in the single studies.

Cumulative Forest Plot

The cumulative forest plot shows the increase in evidence over time.

The studies are sorted chronologically and meta-analytic estimations are conducted step by step. Thus, the confidence interval in each row displays the meta-analytic estimation up to the corresponding study.

Typically, the meta-analytic estimate stabilizes over time and the confidence interval decreases, as the estimation is getting more certain with increasing evidence.