The Overestimation of Violent-Video-Game Effects in Experiments

Background: Whether violent video games have effects on aggressive outcomes remains a controversial research topic with potentially broad impact for society.  At the same time, psychology is becoming increasingly aware of the hazards of publication bias and flexible analysis as potential causes of increased Type I error rates and the overestimation of effects. A meta-analysis from Anderson and colleagues (2010) is thought to condense the available research on violent game effects into incontrovertible evidence of increases in aggressive thoughts, feelings, and behavior.  The authors argue that there is minimal bias in the meta-analyzed research, but only apply trim-and-fill, a procedure with questionable efficacy. We re-analyze their meta-analytic dataset with newer, more effective adjustments for research bias. Methods: Using the original authors’ data, we applied new statistical procedures that test and adjustment for research bias. These tests included funnel plots, Egger’s test for funnel-plot asymmetry, PET and PEESE meta-regression, and p-curve. Sufficient studies were available to apply these to the research literatures of effects of violent games on aggressive thoughts, feelings, and behavior in experiments and in cross-sectional research designs. Results:  Effects of short-term gameplay on aggressive feelings and behavior in experiments appeared to be badly overestimated by bias.  Egger’s test found significant funnel-plot asymmetry among all experiments of aggressive feelings and among those studies Anderson et al. selected as “best-practices” experiments of aggressive behavior. Meta-regression and p-curve adjustments for publication bias recommended harsher adjustments than the original authors’ trim-and-fill adjustments, with some outcomes in experiments adjusted as low as zero. Meta-regression and p-curve estimated very small effects of violent games on aggressive behavior in experiments, such that the effect could not be expected to be detected in a single experiment. Conclusions: Although cross-sectional associations appear mostly robust, the magnitude of violent-game effects in experimental research designs has likely been overestimated. A pre-registered collaborative effort is recommended to provide an unbiased estimate of the basic phenomenon. Results illustrate the prevalence of research bias in social psychology and the need for transparency and re-analysis in meta-analytic reviews.