A Hierarchical Rater Model for Longitudinal Data

Research studies in psychology and education often seek to detect changes or growth in an outcome over a duration of time. This research provides a solution to those interested in estimating latent traits from psychological measures that rely on human raters. Rater effects potentially degrade the quality of scores in constructed response and performance assessments. We develop an extension of the hierarchical rater model (HRM), which yields estimates of latent traits that have been corrected for individual rater bias and variability, for ratings that come from longitudinal designs. The parameterization, called the longitudinal HRM (L-HRM), includes an autoregressive time series process to permit serial dependence between latent traits at adjacent timepoints, as well as a parameter for overall growth. We evaluate and demonstrate the feasibility and performance of the L-HRM using simulation studies. Parameter recovery results reveal predictable amounts and patterns of bias and error for most parameters across conditions. An application to ratings from a study of character strength demonstrates the model. We discuss limitations and future research directions to improve the L-HRM.     

A step forward: Bayesian hierarchical modelling as a tool in assessment of individual discrimination performance

Individual assessment of infants’ speech discrimination is of great value for studies of language development that seek to relate early and later skills, as well as for clinical work. The present study explored the applicability of the hybrid visual fixation paradigm (Houston et al., 2007) and the associated statistical analysis approach to assess individual discrimination of a native vowel contrast, /aː/ - /eː/, in Dutch 6 to 10-month-old infants. Houston et al. found that 80% (8/10) of the 9-month-old infants successfully discriminated the contrast between pseudowords boodup - seepug. Using the same approach, we found that 12% (14/117) of the infants in our sample discriminated the highly salient /aː/ -/eː/ contrast. This percentage was reduced to 3% (3/117) when we corrected for multiple testing. Bayesian hierarchical modeling indicated that 50% of the infants showed evidence of discrimination. Advantages of Bayesian hierarchical modeling are that 1) there is no need for a correction for multiple testing and 2) better estimates at the individual level are obtained. Thus, individual speech discrimination can be more accurately assessed using state of the art statistical approaches.     

Perceived fit with an academic environment: Attitudinal and behavioral outcomes

Because person–organization fit relates to important outcomes, and because fit is a time-dependent process, there exists a need to model longitudinal data related to fit. In a multi-university sample (N = 1174), we tested hypotheses concerning person–organization fit and various behavioral and attitudinal outcomes; correlations between changes in these variables over time, and the correlates of these changes. Using autoregressive modeling and latent growth curve modeling, results indicated that academic fit leads to academic satisfaction, which in turn leads to turnover intent, and to a lesser extent GPA and class absenteeism. Changes in academic fit correlated highly with similar changes in satisfaction and GPA and with opposite changes in turnover intent. Academic satisfaction did not correlate with subsequent organizational citizenship behavior (OCB) as predicted; changes across these domains were significantly related, but were small. Consistent with our hypotheses, Realistic interests correlated negatively with changes in academic fit and satisfaction. Investigative interests were correlated positively with changes in fit and initial satisfaction levels. We conclude by discussing implications, limitations, and future research concerning the longitudinal analysis of person–organization fit.     

Interpreting effect sizes when controlling for stability effects in longitudinal autoregressive models: Implications for psychological science

Effect sizes in longitudinal studies often are dramatically smaller than effect sizes in cross-sectional studies. Indeed, autoregressive models (which are often used in longitudinal studies but not in cross-sectional studies) control for past levels on the outcome (i.e., stability effects) in order to predict change in levels of the outcome over time and thus may greatly reduce the magnitude of the effect of a predictor on the outcome. Unfortunately, however, there have been no attempts to differentiate guidelines for interpreting effect sizes for longitudinal studies versus cross-sectional studies. Consequently, longitudinal effect sizes that fall below the universal guidelines for “small” may be incorrectly dismissed as trivial, when they might be meaningful. In the current paper, we first review the present guidelines for interpreting effect sizes. Next, we discuss several examples of how controlling for stability effects can dramatically attenuate effect sizes of other predictors, in order to support our argument that the current guidelines may be misleading for interpreting longitudinal effects. Finally, we conclude by making recommendations for researchers regarding the interpretation of effect sizes in longitudinal autoregressive models.     

(Re)evaluating the Implications of the Autoregressive Latent Trajectory Model Through Likelihood Ratio Tests of Its Initial Conditions

The autoregressive latent trajectory (ALT) model synthesizes the autoregressive model and the latent growth curve model. The ALT model is flexible enough to produce a variety of discrepant model-implied change trajectories. While some researchers consider this a virtue, others have cautioned that this may confound interpretations of the model's parameters. In this article, we show that some—but not all—of these interpretational difficulties may be clarified mathematically and tested explicitly via likelihood ratio tests (LRTs) imposed on the initial conditions of the model. We show analytically the nested relations among three variants of the ALT model and the constraints needed to establish equivalences. A Monte Carlo simulation study indicated that LRTs, particularly when used in combination with information criterion measures, can allow researchers to test targeted hypotheses about the functional forms of the change process under study. We further demonstrate when and how such tests may justifiably be used to facilitate our understanding of the underlying process of change using a subsample (N = 3,995) of longitudinal family income data from the National Longitudinal Survey of Youth.     

Individual differences in adult attachment: Disentangling two assessment traditions

The goal of this paper was to disentangle the contributions of type of measurement and relational domain to the lack of concordance between adult attachment measures. Eighty-six college students completed the Adult Attachment Interview (AAI) and self-reported measures of attachment security adapted to their relationships with four different relational partners. Results showed that whereas the AAI main dimensions were unrelated to self-reports relative to any partner, reports of attachment security with different partners were interrelated. Further, very few of the AAI State of Mind scales related to self-reported attachment security, but the AAI Experience scales did relate to self-reports in a theoretically consistent manner. Finally, the AAI scales of Idealization and Lack of Recall correlated positively with self-reported attachment when controlling for experiences reported on the AAI. The results suggest that a fundamental distinction between adult attachment measures could be the degree of automaticity of the constructs tapped, and that attachment self-reports may be biased by defensive information-processing strategies among dismissing individuals.