Polytomous IRT models and monotone likelihood ratio of the total score

In a broad class of item response theory (IRT) models for dichotomous items the unweighted total score has monotone likelihood ratio (MLR) in the latent trait. In this study, it is shown that for polytomous items MLR holds for the partial credit model and a trivial generalization of this model. MLR does not necessarily hold if the slopes of the item step response functions vary over items, item steps, or both. MLR holds neither for Samejima's graded response model, nor for nonparametric versions of these three polytomous models. These results are surprising in the context of Grayson's and Huynh's results on MLR for nonparametric dichotomous IRT models, and suggest that establishing stochastic ordering properties for nonparametric polytomous IRT models will be much harder.Hemker's research was supported by the Netherlands Research Council, Grant 575-67-034. Junker's research was supported in part by the National Institutes of Health, Grant CA54852, and by the National Science Foundation, Grant DMS-94.04438.     

Stochastic ordering using the latent trait and the sum score in polytomous IRT models

In a restricted class of item response theory (IRT) models for polytomous items the unweighted total score has monotone likelihood ratio (MLR) in the latent trait. MLR implies two stochastic ordering (SO) properties, denoted SOM and SOL, which are both weaker than MLR, but very useful for measurement with IRT models. Therefore, these SO properties are investigated for a broader class of IRT models for which the MLR property does not hold.In this study, first a taxonomy is given for nonparametric and parametric models for polytomous items based on the hierarchical relationship between the models. Next, it is investigated which models have the MLR property and which have the SO properties. It is shown that all models in the taxonomy possess the SOM property. However, counterexamples illustrate that many models do not, in general, possess the even more useful SOL property.Hemker's research was supported by the Netherlands Research Council, Grant 575-67-034. Junker's research was supported in part by the National Institutes of Health, Grant CA54852, and by the National Science Foundation, Grant DMS-94.04438.     

Bayesian inference for finite mixtures of generalized linear models with random effects

We present an hierarchical Bayes approach to modeling parameter heterogeneity in generalized linear models. The model assumes that there are relevant subpopulations and that within each subpopulation the individual-level regression coefficients have a multivariate normal distribution. However, class membership is not known a priori, so the heterogeneity in the regression coefficients becomes a finite mixture of normal distributions. This approach combines the flexibility of semiparametric, latent class models that assume common parameters for each sub-population and the parsimony of random effects models that assume normal distributions for the regression parameters. The number of subpopulations is selected to maximize the posterior probability of the model being true. Simulations are presented which document the performance of the methodology for synthetic data with known heterogeneity and number of sub-populations. An application is presented concerning preferences for various aspects of personal computers.     

Combining mixture distribution and multidimensional IRTree models for the measurement of extreme response styles

Personality constructs, attitudes and other non-cognitive variables are often measured using rating or Likert-type scales, which does not come without problems. Especially in low-stakes assessments, respondents may produce biased responses due to response styles (RS) that reduce the validity and comparability of the measurement. Detecting and correcting RS is not always straightforward because not all respondents show RS and the ones who do may not do so to the same extent or in the same direction. The present study proposes the combination of a multidimensional IRTree model with a mixture distribution item response theory model and illustrates the application of the approach using data from the Programme for the International Assessment of Adult Competencies (PIAAC). This joint approach allows for the differentiation between different latent classes of respondents who show different RS behaviours and respondents who show RS versus respondents who give (largely) unbiased responses. We illustrate the application of the approach by examining extreme RS and show how the resulting latent classes can be further examined using external variables and process data from computer-based assessments to develop a better understanding of response behaviour and RS.     

Learning about the internal structure of categories through classification and feature inference

Previous research on category learning has found that classification tasks produce representations that are skewed toward diagnostic feature dimensions, whereas feature inference tasks lead to richer representations of within-category structure. Yet, prior studies often measure category knowledge through tasks that involve identifying only the typical features of a category. This neglects an important aspect of a category's internal structure: how typical and atypical features are distributed within a category. The present experiments tested the hypothesis that inference learning results in richer knowledge of internal category structure than classification learning. We introduced several new measures to probe learners' representations of within-category structure. Experiment 1 found that participants in the inference condition learned and used a wider range of feature dimensions than classification learners. Classification learners, however, were more sensitive to the presence of atypical features within categories. Experiment 2 provided converging evidence that classification learners were more likely to incorporate atypical features into their representations. Inference learners were less likely to encode atypical category features, even in a “partial inference” condition that focused learners' attention on the feature dimensions relevant to classification. Overall, these results are contrary to the hypothesis that inference learning produces superior knowledge of within-category structure. Although inference learning promoted representations that included a broad range of category-typical features, classification learning promoted greater sensitivity to the distribution of typical and atypical features within categories.     

Nonparametric IRT: Testing the bi-isotonicity of isotonic probabilistic models (ISOP)

Nonparametric tests for testing the validity of polytomous ISOP-models (unidimensional ordinal probabilistic polytomous IRT-models) are presented. Since the ISOP-model is a very general nonparametric unidimensional rating scale model the test statistics apply to a great multitude of latent trait models. A test for the comonotonicity of item sets of two or more items is suggested. Procedures for testing the comonotonicity of two item sets and for item selection are developed. The tests are based on Goodman-Kruskal's gamma index of ordinal association and are generalizations thereof. It is an essential advantage of polytomous ISOP-models within probabilistic IRT-models that the tests of validity of the model can be performed before and without the model being fitted to the data. The new test statistics have the further advantage that no prior order of items or subjects needs to be known.     

Bounds on variances of estimators for multinomial processing tree models

When there are order constraints among the parameters of a binary, multinomial processing tree (MPT) model, methods have been developed for reparameterizing the constrained MPT into an equivalent unconstrained MPT. This note provides a theorem that is useful in computing bounds on the estimator variances for the parameters of the constrained model in terms of estimator variances of the parameters of the unconstrained model. In particular, we show that if X and Y are random variables taking values in [0,1], then Var[XY]?2(Var[X]+Var[Y]).     

Learning the form of causal relationships using hierarchical bayesian models

People learn quickly when reasoning about causal relationships, making inferences from limited data and avoiding spurious inferences. Efficient learning depends on abstract knowledge, which is often domain or context specific, and much of it must be learned. While such knowledge effects are well documented, little is known about exactly how we acquire knowledge that constrains learning. This work focuses on knowledge of the functional form of causal relationships; there are many kinds of relationships that can apply between causes and their effects, and knowledge of the form such a relationship takes is important in order to quickly identify the real causes of an observed effect. We developed a hierarchical Bayesian model of the acquisition of knowledge of the functional form of causal relationships and tested it in five experimental studies, considering disjunctive and conjunctive relationships, failure rates, and cross-domain effects. The Bayesian model accurately predicted human judgments and outperformed several alternative models.     

Time-Varying Treatments in Observational Studies: Marginal Structural Models of the Effects of Early Grade Retention on Math Achievement

One of the main objectives of many empirical studies in the social and behavioral sciences is to assess the causal effect of a treatment or intervention on the occurrence of a certain event. The randomized controlled trial is generally considered the gold standard to evaluate such causal effects. However, for ethical or practical reasons, social scientists are often bound to the use of nonexperimental, observational designs. When the treatment and control group are different with regard to variables that are related to the outcome, this may induce the problem of confounding. A variety of statistical techniques, such as regression, matching, and subclassification, is now available and routinely used to adjust for confounding due to measured variables. However, these techniques are not appropriate for dealing with time-varying confounding, which arises in situations where the treatment or intervention can be received at multiple timepoints. In this article, we explain the use of marginal structural models and inverse probability weighting to control for time-varying confounding in observational studies. We illustrate the approach with an empirical example of grade retention effects on mathematics development throughout primary school.     

Model structure adequacy analysis: selecting models on the basis of their ability to answer scientific questions

Models carry the meaning of science. This puts a tremendous burden on the process of model selection. In general practice, models are selected on the basis of their relative goodness of fit to data penalized by model complexity. However, this may not be the most effective approach for selecting models to answer a specific scientific question because model fit is sensitive to all aspects of a model, not just those relevant to the question. Model Structural Adequacy analysis is proposed as a means to select models based on their ability to answer specific scientific questions given the current understanding of the relevant aspects of the real world.     

On the sampling theory roundations of item response theory models

Item response theory (IT) models are now in common use for the analysis of dichotomous item responses. This paper examines the sampling theory foundations for statistical inference in these models. The discussion includes: some history on the stochastic subject versus the random sampling interpretations of the probability in IRT models; the relationship between three versions of maximum likelihood estimation for IRT models; estimating versus estimating -predictors; IRT models and loglinear models; the identifiability of IRT models; and the role of robustness and Bayesian statistics from the sampling theory perspective.A presidential address can serve many different functions. This one is a report of investigations I started at least ten years ago to understand what IRT was all about. It is a decidedly one-sided view, but I hope it stimulates controversy and further research. I have profited from discussions of this material with many people including: Brian Junker, Charles Lewis, Nicholas Longford, Robert Mislevy, Ivo Molenaar, Donald Rock, Donald Rubin, Lynne Steinberg, Martha Stocking, William Stout, Dorothy Thayer, David Thissen, Wim van der Linden, Howard Wainer, and Marilyn Wingersky. Of course, none of them is responsible for any errors or misstatements in this paper. The research was supported in part by the Cognitive Science Program, Office of Naval Research under Contract No. Nooo14-87-K-0730 and by the Program Statistics Research Project of Educational Testing Service.     

On the asymptotic distributions of two statistics for two-level covariance structure models within the class of elliptical distributions

Since data in social and behavioral sciences are often hierarchically organized, special statistical procedures for covariance structure models have been developed to reflect such hierarchical structures. Most of these developments are based on a multivariate normality distribution assumption, which may not be realistic for practical data. It is of interest to know whether normal theory-based inference can still be valid with violations of the distribution condition. Various interesting results have been obtained for conventional covariance structure analysis based on the class of elliptical distributions. This paper shows that similar results still hold for 2-level covariance structure models. Specifically, when both the level-1 (within cluster) and level-2 (between cluster) random components follow the same elliptical distribution, the rescaled statistic recently developed by Yuan and Bentler asymptotically follows a chi-square distribution. When level-1 and level-2 have different elliptical distributions, an additional rescaled statistic can be constructed that also asymptotically follows a chi-square distribution. Our results provide a rationale for applying these rescaled statistics to general non-normal distributions, and also provide insight into issues related to level-1 and level-2 sample sizes. The authors thank an associate editor and three referees for their constructive comments, which led to an improved version of the paper. This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of Notre Dame faculty research grant.     

Remarks on the identifiability of thurstonian paired comparison models under multiple judgment

It is well-known that the representations of the Thurstonian models for difference judgment data are not unique. It has been shown that equivalence classes can be formed to provide a more meaningful partition of the covariance structures of the Thurstonian ranking models. In this paper, we examine the equivalence relations between Thurstonian covariance structure models for paired comparison data obtained under multiple judgment and discuss their implications on the general identification constraints and methods to check for parameter identifiability in restricted models.The author is indebted to Ulf Böckenholt and Albert Maydeu-Olivares for their significant comments and suggestions which led to considerable improvement in this article.     

Capturing the effects of texting on young drivers behaviour based on copula and Gaussian Mixture Models

This research effort aims to investigate the impact of texting on young drivers' behavior and safety based on data from driving simulator experiments, for different driving contexts, like motorways, urban and rural roads, during daytime and night, and for alternative weather conditions (‘clear sky’ and rain). The study offers a complete and comprehensive investigation of the effects of texting on driving behavior, able to provide evidence on policy-making. For the purposes of this study, a driving simulator experiment was carried out where 34 young participants drove predefined driving scenarios. Initially, multivariate copula analysis was used in order to explore statistical inferences among variables, especially since it retains a parametric specification for bivariate dependencies and allows testing of several parametric structures to characterize them. Secondly, alternative copula configurations were tested, which showed that texting and other road and environmental characteristics affect young drivers behavior and in particular more than one outcome can occur at the same time. Finally, Gaussian Mixture Modeling (GMM) was employed, demonstrating that the variables' pairs that presented the strongest correlations were lane departure and speed, as well as speed and reaction time. GMMs application showed that drivers using mobile phones who were involved in a collision presented a different driving behavior compared to the drivers who were occupied but were not involved in a collision.     

Influence of consonantal context on the pronunciation of vowels: a comparison of human readers and computational models

In two experiments, we found that college students' pronunciations of vowels in nonwords are influenced both by preceding and following consonants. The predominance of rimes in previous studies of reading does not appear to arise because readers are unable to pick up associations that cross the onset-rime boundary, but rather because English has relatively few such associations. Comparisons between people's vowel pronunciations and those produced by various computational models of reading showed that no model provided a good account of human performance on nonwords for which the vowel shows contextual conditioning. Possible directions for improved models are suggested.     

The influence of competitive status (winner/loser) on the behavior of male rats in three models of anxiety

Studies indicate that features such as prior stressful experience, strain, gender, and age can influence the behavior of rats in animal models of anxiety. In the present study, we examined the possible influence of competitive status (winner/loser) in three such models: the elevated plus‐maze, the open field, and the social interaction test. One hundred to 135‐day‐old male Wistar rats were conditioned to traverse a straight runway tube to obtain food. Subsequently, two rats were placed at the same time in the runway tube and, being unable to pass each other, one of them pushed the other to the opposite end‐box. The rats were categorized as winners or losers in this competition. One week after the straight runway tube test, the rats were submitted to the anxiety models, where it was observed that winner rats showed greater locomotor activity than the losers in the three models studied. Furthermore, winner rats showed less immobility and higher central and total locomotor activity in the open field and a greater duration of social interaction in the social interaction test. These results suggest that competitive status has an influence on the locomotor activity of rats in animal models of anxiety. However, whether competitive status influences anxiety as assessed in these models is unclear, and further investigations are warranted. Aggr. Behav. 28:164–171, 2002. © 2002 Wiley‐Liss, Inc.     

Superstars are not necessarily role models: Morality perceptions moderate the impact of competence perceptions on supervisor role modeling

There is a general assumption that we choose role models from the ranks of those who have demonstrated extraordinary competence. However, the person perception literature supports the expectation that morality may also matter, and that we may be most likely to role model competent individuals if we also believe that they have good moral character. To test this possibility, we conducted four studies of adults’ role modeling of workplace supervisors. Study 1 (N = 245) and Study 2 (N = 110) showed that workplace supervisors’ perceived competence was most strongly associated with role model perceptions when the supervisor was also seen as moral. Study 3 (N = 492) and 4 (N = 335) replicated these findings with pre‐registered experiments, and revealed indirect effects of supervisor attributes on role modeling through emulation. Results suggest that we choose organizational role models who have achieved success in ways that are in line with our moral values.     

Development of the Questionnaire on the Acceptance of Automated Driving (QAAD): Data-driven models for Level 3 and Level 5 automated driving

Automated driving comes with many promises like zero traffic casualties that are, however, only realizable given their technological development and public acceptance for wide-spread deployment. To investigate the potential acceptance, we developed a new data-driven questionnaire focusing on drivers and barriers of the anticipated possible (non-)adoption of automated driving (AD). Therefore, we conducted a cross-sectional questionnaire study with 725 respondents (351 female, 374 male) ranging from 18 to 96 years. We applied exploratory and confirmatory factor analyses and structural equation modeling, to pursue the overarching goal to develop the QAAD questionnaire (short and long version for SAE Level 3 (L3) and 5 (L5) AD). Hence, we identified the three latent factors PRO (positive aspects), CON (negative aspects), and NDRTs (non-driving related tasks) of L3 (short: 9 items; long: 16) and L5 (short: 11, long: 17), respectively. Additionally, we queried general questions on AD (8 items) and extracted the two factors Early Adoption/Pro AD and Sustainability. Our findings and the goodness-of-fit indices suggest data-driven models for L3 and L5 automated driving and on general aspects focusing on early adoption and sustainability in the context of AD. They can be applied in future research settings, in particular, in (quasi-)experimental L3 and L5 AD studies and in population surveys on AD. The evidence of the presented study should be validated and compared to other questionnaires on AD in different countries around the globe.     

A temporal common ground for learning: The moderating effect of shared mental models on the relation between team learning behaviours and performance improvement

In this longitudinal study, we integrated a team process and a learning curve perspective on team learning and empirically analysed whether team learning processes lead to performance improvement. In addition, we tested whether this relation is moderated by the similarity of team members’ task, team, and temporal mental models. We tested our model on a sample of 67 teams (314 individuals) competing in a management simulation over five consecutive time periods, using random coefficient modelling (RCM). Our findings suggest that team learning behaviours do not have a direct effect on the team learning curve, but temporal and task mental models are crucial for the translation of team learning behaviours into performance improvement. We found that when teams have similar task and temporal mental models, engaging in team learning processes is beneficial, whereas, when teams have dissimilar task and temporal mental models, it is detrimental to performance improvement. We did not find a significant effect for the moderating role of team mental model similarity. Our study emphasizes the importance of integrating different perspectives on team learning and provides support for the role of team cognition as a catalyst for team learning.