Panning for the gold in health research: Incorporating studies’ methodological quality in meta-analysis

Background: Systematic reviews now routinely assess methodological quality to gauge the validity of the included studies and of the synthesis as a whole. Although trends from higher quality studies should be clearer, it is uncertain how often meta-analyses incorporate methodological quality in models of study results either as predictors, or, more interestingly, in interactions with theoretical moderators.

Objective and methods: We survey 200 meta-analyses in three health promotion domains to examine when and how meta-analyses incorporate methodological quality.

Results: Although methodological quality assessments commonly appear in contemporary meta-analyses (usually as scales), they are rarely incorporated in analyses, and still more rarely analysed in interaction with theoretical determinants of the success of health promotions. The few meta-analyses (2.5%) that did include such an interaction analysis showed that moderator results remained significant in higher quality studies or were present only among higher quality studies. We describe how to model quality interactively with theoretically derived moderators and discuss strengths and weaknesses of this approach and in relation to current meta-analytic practice.

Conclusions: In large literatures exhibiting heterogeneous effects, meta-analyses can incorporate methodological quality and generate conclusions that enable greater confidence not only about the substantive phenomenon but also about the role that methodological quality itself plays.     

Do Bettors Correctly Perceive Odds? Three Studies of How Bettors Interpret Betting Odds as Probabilistic Information

This paper reports on three studies investigating how accurately bettors (=people who regularly bet on sports events) interpret the probabilistic information implied by betting odds. All studies were based on data collected by web surveys prompting a total of 186 experienced bettors to convert sets of representative odds into frequency judgments. Bayesian statistical methods were used to analyze the data. From the results, the following conclusions were made: (i) On the whole, the bettors produced well‐calibrated judgments, indicating that they have realistic perceptions of odds. (ii) Bettors were unable to consciously adjust judgments for different margins. (iii) Although their interval judgments often covered the estimates implied by the odds, the bettors tended to overestimate the variation of expected profitable bets between months. The results are consistent with prior research showing that people tend to make accurate probability judgments when faced with tasks characterized by constant and clear feedback. Copyright © 2015 John Wiley & Sons, Ltd.     

Safety–critical events in everyday cycling – Interviews with bicyclists and video annotation of safety–critical events in a naturalistic cycling study

Research in cycling safety seeks to better understand bicycle-related crashes and injuries. The present naturalistic cycling study contributes to this research by collecting data about bicyclists’ behavior and impressions of safety–critical situations, information unavailable in traditional data sources (e.g., accident databases, observational studies). Naturalistic data were collected from 16 bicyclists (8 female; M = 39.1 years, SD = 11.4 years) who rode instrumented bicycles for two weeks. Bicyclists were instructed to report all episodes in which they felt uncomfortable while riding (subjective risk perception), even if they didn’t fall. After data collection, the bicyclists were interviewed in detail regarding their self-reported safety–critical events. Environmental conditions were also recorded via video (e.g., road surface, weather). In total, 63 safety–critical events (56 non-crashes, 7 crashes) were reported by the bicyclists, mainly due to interactions with other road users – but also due to poorly maintained infrastructure. In low-visibility conditions, vehicle-bicycle and bicycle-bicycle events were the most uncomfortable for the bicyclists. Self-reported pedestrian–bicycle events primarily consisted of pedestrians starting to cross the bicycle path without looking. With one exception, all crashes found in the study belonged to poorly maintained road and infrastructure. In particular, construction work or obstacles in the bicycle path were reported as uncomfortable and annoying by the bicyclists. This study shows how naturalistic data and bicyclists’ interviews together can provide a more informative picture of safety–critical situations experienced by the bicyclist than traditional data sources can.     

Hierarchical models for relational event sequences

Interaction within small groups can often be represented as a sequence of events, each event involving a sender and a recipient. Recent methods for modeling network data in continuous time model the rate at which individuals interact conditioned on the previous history of events as well as actor covariates. We present a hierarchical extension for modeling multiple such sequences, facilitating inferences about event-level dynamics and their variation across sequences. The hierarchical approach allows one to share information across sequences in a principled manner—we illustrate the efficacy of such sharing through a set of prediction experiments. After discussing methods for adequacy checking and model selection for this class of models, the method is illustrated with an analysis of high school classroom dynamics from 297 sessions.     

A Test of Order-Constrained Hypotheses for Circular Data With Applications to Human Movement Science

Researchers studying the movements of the human body often encounter data measured in angles (e.g., angular displacements of joints). The evaluation of these circular data requires special statistical methods. The authors introduce a new test for the analysis of order-constrained hypotheses for circular data. Through this test, researchers can evaluate their expectations regarding the outcome of an experiment directly by representing their ideas in the form of a hypothesis containing inequality constraints. The resulting data analysis is generally more powerful than one using standard null hypothesis testing. Two examples of circular data from human movement science are presented to illustrate the use of the test. Results from a simulation study show that the test performs well.     

Evaluating Structural Equation Models for Categorical Outcomes: A New Test Statistic and a Practical Challenge of Interpretation

This research is concerned with two topics in assessing model fit for categorical data analysis. The first topic involves the application of a limited-information overall test, introduced in the item response theory literature, to structural equation modeling (SEM) of categorical outcome variables. Most popular SEM test statistics assess how well the model reproduces estimated polychoric correlations. In contrast, limited-information test statistics assess how well the underlying categorical data are reproduced. Here, the recently introduced C₂ statistic of Cai and Monroe (2014) is applied. The second topic concerns how the root mean square error of approximation (RMSEA) fit index can be affected by the number of categories in the outcome variable. This relationship creates challenges for interpreting RMSEA. While the two topics initially appear unrelated, they may conveniently be studied in tandem since RMSEA is based on an overall test statistic, such as C₂. The results are illustrated with an empirical application to data from a large-scale educational survey.     

Psychometric evaluation and normative data for a Swedish version of the Patient Health Questionnaire 15‐Item Somatic Symptom Severity Scale

The lack of an available Swedish version of the Patient Health Questionnaire 15‐Item Somatic Symptom Severity Scale (PHQ‐15) motivated the present psychometric evaluation of such a version as well as providing normative data for the PHQ‐15. Data from 3,406 individuals who took part in the Västerbotten Environmental Health Study in Sweden were used. The respondents constitute a random sample, aged 18 to 79 years, stratified for age and sex. They responded to a Swedish translation of the PHQ‐15 as well as the Hospital Anxiety and Depression Scale, the Perceived Stress Scale, and the Shirom Melamed Burnout Questionnaire for assessment of convergent validity. The distribution of PHQ‐15 scores was positively skewed and mesokurtic in shape, and the internal consistency of the PHQ‐15 was satisfactory. Correlation coefficients between PHQ‐15 score and the measures of anxiety, depression, stress and mental/physical exhaustion indicate satisfactory validity. Normative data for PHQ‐15 scores as well as for categories of somatic symptom severity are provided. The favorable psychometric properties of the Swedish version of the PHQ‐15 suggest use of this instrument for quantification of somatization in Swedish and similar populations, and has the advantage of available normative data.