Panning for the gold in health research: Incorporating studies’ methodological quality in meta-analysis

Background: Systematic reviews now routinely assess methodological quality to gauge the validity of the included studies and of the synthesis as a whole. Although trends from higher quality studies should be clearer, it is uncertain how often meta-analyses incorporate methodological quality in models of study results either as predictors, or, more interestingly, in interactions with theoretical moderators.

Objective and methods: We survey 200 meta-analyses in three health promotion domains to examine when and how meta-analyses incorporate methodological quality.

Results: Although methodological quality assessments commonly appear in contemporary meta-analyses (usually as scales), they are rarely incorporated in analyses, and still more rarely analysed in interaction with theoretical determinants of the success of health promotions. The few meta-analyses (2.5%) that did include such an interaction analysis showed that moderator results remained significant in higher quality studies or were present only among higher quality studies. We describe how to model quality interactively with theoretically derived moderators and discuss strengths and weaknesses of this approach and in relation to current meta-analytic practice.

Conclusions: In large literatures exhibiting heterogeneous effects, meta-analyses can incorporate methodological quality and generate conclusions that enable greater confidence not only about the substantive phenomenon but also about the role that methodological quality itself plays.     

Do Bettors Correctly Perceive Odds? Three Studies of How Bettors Interpret Betting Odds as Probabilistic Information

This paper reports on three studies investigating how accurately bettors (=people who regularly bet on sports events) interpret the probabilistic information implied by betting odds. All studies were based on data collected by web surveys prompting a total of 186 experienced bettors to convert sets of representative odds into frequency judgments. Bayesian statistical methods were used to analyze the data. From the results, the following conclusions were made: (i) On the whole, the bettors produced well‐calibrated judgments, indicating that they have realistic perceptions of odds. (ii) Bettors were unable to consciously adjust judgments for different margins. (iii) Although their interval judgments often covered the estimates implied by the odds, the bettors tended to overestimate the variation of expected profitable bets between months. The results are consistent with prior research showing that people tend to make accurate probability judgments when faced with tasks characterized by constant and clear feedback. Copyright © 2015 John Wiley & Sons, Ltd.