Applying mixed‐effects modeling to single‐subject designs: An introduction

Behavior analysis and statistical inference have shared a conflicted relationship for over fifty years. However, a significant portion of this conflict is directed toward statistical tests (e.g., t‐tests, ANOVA) that aggregate group and/or temporal variability into means and standard deviations and as a result remove much of the data important to behavior analysts. Mixed‐effects modeling, a more recently developed statistical test, addresses many of the limitations of more basic tests by incorporating random effects. Random effects quantify individual subject variability without eliminating it from the model, hence producing a model that can predict both group and individual behavior. We present the results of a generalized linear mixed‐effects model applied to single‐subject data taken from Ackerlund Brandt, Dozier, Juanico, Laudont, & Mick, 2015, in which children chose from one of three reinforcers for completing a task. Results of the mixed‐effects modeling are consistent with visual analyses and importantly provide a statistical framework to predict individual behavior without requiring aggregation. We conclude by discussing the implications of these results and provide recommendations for further integration of mixed‐effects models in the analyses of single‐subject designs.     

The Effect of modality specific interference on working memory in recalling aversive auditory and visual memories*

Both auditory and visual emotional memories can be made less emotional by loading working memory (WM) during memory recall. Taxing WM during recall can be modality specific (giving an auditory [visuospatial] load during recall of an auditory [visual] memory) or cross modal (an auditory load during visual recall or vice versa). We tested whether modality specific loading taxes WM to a larger extent than cross modal loading. Ninety-six participants undertook a visual and auditory baseline Random Interval Repetition task (i.e. responding as fast as possible to a visual or auditory stimulus by pressing a button). Then, participants recalled a distressing visual and auditory memory, while performing the same visual and auditory Random Interval Repetition task. Increased reaction times (compared to baseline) were indicative of WM loading. Using Bayesian statistics, we compared five models in terms of general and modality specific taxation. There was support for the model describing the effect on WM of dual tasking in general, irrespective of modality specificity, and for the model describing the effect of modality specific loading. Both models combined gained the most support. The results suggest a general effect of dual tasking on taxing WM and a superimposed effect of taxing in matched modality.     

认知诊断测验中基于信息矩阵的多群组DIF检验

基于改进的Wald统计量,将适用于两群组的DIF检测方法拓展至多群组的项目功能差异（DIF）检验;改进的Wald统计量将分别通过计算观察信息矩阵（Obs）和经验交叉相乘信息矩阵（XPD）而得到。模拟研究探讨了此二者与传统计算方法在多个群组下的DIF检验情况,结果表明：（1）Obs和XPD的一类错误率明显低于传统方法,DINA模型估计下Obs和XPD的一类错误率接近理论水平;（2）样本量和DIF量较大时,Obs和XPD具有与传统Wald统计量大体相同的统计检验力。     

An investigation of the effects of self-reported self-control strength on shooting performance

During professional shooting tournaments, which typically last multiple hours, athletes must stay focused at all times in order to perform at their highest levels. Sustaining attention over extended periods of time relies on self-control. Crucially, perceived state self-control strength appears to wane as a function of task duration, which ultimately can impair shooting performance. In the present study, we tested the assumption that the level of self-reported self-control strength decreases over the course of a 1-h shooting task measured twice during a regular training day and separated by a 2-h break. Additionally, we assumed that shooting performance would be linked with fluctuations in self-control. A total of 21 shooters (14 elite and 7 sub-elite) took part in this study and were asked to perform a series of 10 shots at a standardized target, five times in the morning and five times in the afternoon (i.e., 100 shots total). The participants also reported their perceived state self-control strength at the baseline (prior to the start of the morning session as well as the afternoon session) and after a series of 10 shots each in the morning and afternoon (i.e., 12 measurements in total). In line with our hypotheses, we observed that perceived state self-control diminished with the number of shots performed, and that perceived state self-control could explain shooting performance. Additionally, these observations could explain the difference in shooting performance between elite and sub-elite athletes. The results suggest that the perception of self-control strength is highly important for optimal shooting performance. Practical implications are discussed.     

A note on residual M-distances for identifying aberrant response patterns

Although a statistical model might fit well to a large proportion of the individuals of a random sample, some individuals might give ‘unusual’ responses that are not well explained by the hypothesized model. If individual responses are given as continuous response vectors, M-distances can be used to produce real valued indicators of how well an individual's response vector corresponds to a covariance structure implied by a psychometric model. In this note, we focus on the so-called one-factor model. Two M-distances, d_si and d_ri, which are sensitive to different aspects of the assumed factor model, have been proposed. While one of the M-distances, d_ri, has been derived based on Bartlett factor scores, in this note we show that the second M-distance, d_si, can be derived in an analogous fashion based on Thomson factor scores.     

Testing Variance Components in Linear Mixed Modeling Using Permutation

Abstract

Inference of variance components in linear mixed modeling (LMM) provides evidence of heterogeneity between individuals or clusters. When only nonnegative variances are allowed, there is a boundary (i.e., 0) in the variances’ parameter space, and regular inference statistical procedures for such a parameter could be problematic. The goal of this article is to introduce a practically feasible permutation method to make inferences about variance components while considering the boundary issue in LMM. The permutation tests with different settings (i.e., constrained vs. unconstrained estimation, specific vs. generalized test, different ways of calculating p values, and different ways of permutation) were examined with both normal data and non-normal data. In addition, the permutation tests were compared to likelihood ratio (LR) tests with a mixture of chi-squared distributions as the reference distribution. We found that the unconstrained permutation test with the one-sided p-value approach performed better than the other permutation tests and is a useful alternative when the LR tests are not applicable. An R function is provided to facilitate the implementation of the permutation tests, and a real data example is used to illustrate the application. We hope our results will help researchers choose appropriate tests when testing variance components in LMM.     

Modelling monotonic effects of ordinal predictors in Bayesian regression models

Ordinal predictors are commonly used in regression models. They are often incorrectly treated as either nominal or metric, thus under- or overestimating the information contained. Such practices may lead to worse inference and predictions compared to methods which are specifically designed for this purpose. We propose a new method for modelling ordinal predictors that applies in situations in which it is reasonable to assume their effects to be monotonic. The parameterization of such monotonic effects is realized in terms of a scale parameter b representing the direction and size of the effect and a simplex parameter modelling the normalized differences between categories. This ensures that predictions increase or decrease monotonically, while changes between adjacent categories may vary across categories. This formulation generalizes to interaction terms as well as multilevel structures. Monotonic effects may be applied not only to ordinal predictors, but also to other discrete variables for which a monotonic relationship is plausible. In simulation studies we show that the model is well calibrated and, if there is monotonicity present, exhibits predictive performance similar to or even better than other approaches designed to handle ordinal predictors. Using Stan, we developed a Bayesian estimation method for monotonic effects which allows us to incorporate prior information and to check the assumption of monotonicity. We have implemented this method in the R package brms, so that fitting monotonic effects in a fully Bayesian framework is now straightforward.     

How do included and excluded students with SEBD function socially and academically after 1,5 year of special education services?

ABSTRACT

The present study tested three conflicting hypotheses as to how students with social/emotional/behavioural difficulties (SEBD), who showed similar social-emotional, behavioural and academic functioning prior to placement, function socially and academically after they have received additional support either in inclusive regular education or in exclusive special education. Thirty-six included and 15 excluded students with SEBD participated. We collected data from students and teachers with classroom surveys, individual testing sessions with students with SEBD, and from application files. Using Bayesian statistics, our results suggest that excluded students are better socially embedded in exclusive special education and that they perform better academically than comparable included students with SEBD. Special education services in exclusive settings may thus afford certain benefits to some students with SEBD, not typically found in regular education, which can be considered a first indication that there may be valid counterarguments against the ‘inclusion for all’ perspective on educational needs.     

Meaningfulness Beats Frequency in Multiword Chunk Processing

Whereas a growing bulk of work has demonstrated that both adults and children are sensitive to frequently occurring word sequences, little is known about the potential role of meaning in the processing of such multiword chunks. Here, we take a first step toward assessing the contribution of meaningfulness in the processing of multiword sequences, using items that varied in chunk meaningfulness. In a phrasal-decision study, we compared reaction times for triads of three-word sequences, corresponding to idiomatic expressions, compositional phrases, and phrasal fragments, while controlling for phrase and substring frequencies. Chunk meaningfulness, as assessed by a separate subjective rating study, was found to speed up decision times for all three types of strings: The more meaningful a multiword sequence was judged to be, the faster it was processed, independently of whether it was idiomatic, compositional in nature, or a phrasal fragment. These results highlight the importance of taking meaning into account when considering the processing of multiword chunks, consistent with predictions of construction-based approaches to language.