Properties of hypothesis testing techniques and (Bayesian) model selection for exploration‐based and theory‐based (order‐restricted) hypotheses

In this paper, the performance of six types of techniques for comparisons of means is examined. These six emerge from the distinction between the method employed (hypothesis testing, model selection using information criteria, or Bayesian model selection) and the set of hypotheses that is investigated (a classical, exploration‐based set of hypotheses containing equality constraints on the means, or a theory‐based limited set of hypotheses with equality and/or order restrictions). A simulation study is conducted to examine the performance of these techniques. We demonstrate that, if one has specific, a priori specified hypotheses, confirmation (i.e., investigating theory‐based hypotheses) has advantages over exploration (i.e., examining all possible equality‐constrained hypotheses). Furthermore, examining reasonable order‐restricted hypotheses has more power to detect the true effect/non‐null hypothesis than evaluating only equality restrictions. Additionally, when investigating more than one theory‐based hypothesis, model selection is preferred over hypothesis testing. Because of the first two results, we further examine the techniques that are able to evaluate order restrictions in a confirmatory fashion by examining their performance when the homogeneity of variance assumption is violated. Results show that the techniques are robust to heterogeneity when the sample sizes are equal. When the sample sizes are unequal, the performance is affected by heterogeneity. The size and direction of the deviations from the baseline, where there is no heterogeneity, depend on the effect size (of the means) and on the trend in the group variances with respect to the ordering of the group sizes. Importantly, the deviations are less pronounced when the group variances and sizes exhibit the same trend (e.g., are both increasing with group number).     

A mixture hierarchical model for response times and response accuracy

In real testing, examinees may manifest different types of test‐taking behaviours. In this paper we focus on two types that appear to be among the more frequently occurring behaviours – solution behaviour and rapid guessing behaviour. Rapid guessing usually happens in high‐stakes tests when there is insufficient time, and in low‐stakes tests when there is lack of effort. These two qualitatively different test‐taking behaviours, if ignored, will lead to violation of the local independence assumption and, as a result, yield biased item/person parameter estimation. We propose a mixture hierarchical model to account for differences among item responses and response time patterns arising from these two behaviours. The model is also able to identify the specific behaviour an examinee engages in when answering an item. A Monte Carlo expectation maximization algorithm is proposed for model calibration. A simulation study shows that the new model yields more accurate item and person parameter estimates than a non‐mixture model when the data indeed come from two types of behaviour. The model also fits real, high‐stakes test data better than a non‐mixture model, and therefore the new model can better identify the underlying test‐taking behaviour an examinee engages in on a certain item.     

A Comparison of Inverse-Wishart Prior Specifications for Covariance Matrices in Multilevel Autoregressive Models

Multilevel autoregressive models are especially suited for modeling between-person differences in within-person processes. Fitting these models with Bayesian techniques requires the specification of prior distributions for all parameters. Often it is desirable to specify prior distributions that have negligible effects on the resulting parameter estimates. However, the conjugate prior distribution for covariance matrices—the Inverse-Wishart distribution—tends to be informative when variances are close to zero. This is problematic for multilevel autoregressive models, because autoregressive parameters are usually small for each individual, so that the variance of these parameters will be small. We performed a simulation study to compare the performance of three Inverse-Wishart prior specifications suggested in the literature, when one or more variances for the random effects in the multilevel autoregressive model are small. Our results show that the prior specification that uses plug-in ML estimates of the variances performs best. We advise to always include a sensitivity analysis for the prior specification for covariance matrices of random parameters, especially in autoregressive models, and to include a data-based prior specification in this analysis. We illustrate such an analysis by means of an empirical application on repeated measures data on worrying and positive affect.     

Exploring Factor Model Parameters across Continuous Variables with Local Structural Equation Models

Using an empirical data set, we investigated variation in factor model parameters across a continuous moderator variable and demonstrated three modeling approaches: multiple-group mean and covariance structure (MGMCS) analyses, local structural equation modeling (LSEM), and moderated factor analysis (MFA). We focused on how to study variation in factor model parameters as a function of continuous variables such as age, socioeconomic status, ability levels, acculturation, and so forth. Specifically, we formalized the LSEM approach in detail as compared with previous work and investigated its statistical properties with an analytical derivation and a simulation study. We also provide code for the easy implementation of LSEM. The illustration of methods was based on cross-sectional cognitive ability data from individuals ranging in age from 4 to 23 years. Variations in factor loadings across age were examined with regard to the age differentiation hypothesis. LSEM and MFA converged with respect to the conclusions. When there was a broad age range within groups and varying relations between the indicator variables and the common factor across age, MGMCS produced distorted parameter estimates. We discuss the pros of LSEM compared with MFA and recommend using the two tools as complementary approaches for investigating moderation in factor model parameters.     

Behavioral variability in an evolutionary theory of behavior dynamics

McDowell’s evolutionary theory of behavior dynamics (McDowell, 2004) instantiates populations of behaviors (abstractly represented by integers) that evolve under the selection pressure of the environment in the form of positive reinforcement. Each generation gives rise to the next via low‐level Darwinian processes of selection, recombination, and mutation. The emergent patterns can be analyzed and compared to those produced by biological organisms. The purpose of this project was to explore the effects of high mutation rates on behavioral variability in environments that arranged different reinforcer rates and magnitudes. Behavioral variability increased with the rate of mutation. High reinforcer rates and magnitudes reduced these effects; low reinforcer rates and magnitudes augmented them. These results are in agreement with live‐organism research on behavioral variability. Various combinations of mutation rates, reinforcer rates, and reinforcer magnitudes produced similar high‐level outcomes (equifinality). These findings suggest that the independent variables that describe an experimental condition interact; that is, they do not influence behavior independently. These conclusions have implications for the interpretation of high levels of variability, mathematical undermatching, and the matching theory. The last part of the discussion centers on a potential biological counterpart for the rate of mutation, namely spontaneous fluctuations in the brain's default mode network.     

Using Marginal Structural Modeling for Grade Retention Effects

Vandecandelaere, Vansteelandt, De Fraine, and Van Damme (this issue) described marginal structural modeling (MSM) and used it to estimate the effects of a time-varying intervention, retention (holding back) in school grades, on students' math achievement. This commentary supplements Vandecandelaere et al. (this issue) and discusses several topics in retention studies and MSM. First, we discuss the importance of equating time-varying confounders in retention studies. Second, we discuss same-grade and same-age comparisons in retention studies. Third, we discuss one important section in the authors' overview of MSM: why standard methods (e.g., ANCOVA, propensity score analysis) cannot properly adjust for time-varying confounders. Finally, using the grade retention analyses in Vandecandelaere et al. (this issue) as an example, we provide our insights on four aspects of MSM: (a) covariate selection, (b) estimation of weights, (c) evaluation of balance properties, and (d) missing data handling.     

Anxiety-related threat bias in recognition memory: the moderating effect of list composition and semantic-similarity effects

Individuals with high anxiety show bias for threatening information, but it is unclear whether this bias affects memory. Recognition memory studies have shown biases for recognising and rejecting threatening items in anxiety, prompting the need to identify moderating factors of this effect. This study focuses on the role of semantic similarity: the use of many semantically related threatening words could increase familiarity for those items and obscure anxiety-related differences in memory. To test this, two recognition memory experiments varied the proportion of threatening words in lists to manipulate the semantic-similarity effects. When similarity effects were reduced, participants with high trait anxiety were biased to respond “new” to threatening words, whereas when similarity effects were strong there was no effect of anxiety on memory bias. Analysis of the data with the drift diffusion model showed that the bias was due to differences in processing of the threatening stimuli rather than a simple response bias. These data suggest that the semantic similarity of the threatening words significantly affects the presence or absence of anxiety-related threat bias in recognition memory. The results indicate that trait anxiety is associated with a bias to decide that threatening stimuli were not previously studied, but only when semantic-similarity effects are controlled. Implications for theories of anxiety and future studies in this domain are discussed.     

四参数Logistic模型潜在特质参数的Warm加权极大似然估计

本文以四参数Logistic(4-parameter Logistic,4PL)模型为研究对象,根据Warm的加权极大似然估计技巧,提出了4PL模型潜在特质参数的加权极大似然估计方法,并借助模拟研究对加权极大似然估计的性质进行验证。研究结果表明,与通常的极大似然估计和后验期望估计相比,加权极大似然估计的偏差(bias)明显减小,并且具有良好的返真性能。此外,在测试的长度较短和项目的区分度较小的情况下,加权极大似然估计依然保持了良好的统计性质,表现出更加显著的优势。     

Personality,hedonic balance and the quality and quantity of sleep in adulthood

Objective: Sufficient and good-quality sleep is important for individual functioning. This study explored associations between personality and sleep duration and sleep quality in adulthood. The mediating role of hedonic balance and the moderating roles of age and sex were also explored.

Method: A nationally representative sample of Australian adults (n = 14,065; M_age = 44.4 years; 53.1% women) completed self-report measures of personality, sleep, hedonic balance and demographic variables (e.g. health status, employment status) in late 2013.

Results: After controlling for demographic variables, we found that high neuroticism was associated with poorer sleep quality, and both long and short sleep durations (a curvilinear relationship). Small effects were also observed relating high extraversion and low openness to better sleep quality. Hedonic balance mediated all linear and non-linear associations between personality and sleep. Additional moderator analyses showed that high openness was more strongly related to poor sleep quality among men and young adults. High neuroticism was more strongly related to poor sleep quality among men.

Conclusion: Findings indicate that personality is important for sleep in adulthood and that hedonic balance features a prominent role in this association.