Hierarchical modeling of paired comparison data

The method of paired comparisons belongs to a small group of techniques that provide explicit information about the consistency of individual and aggregated choices. This article investigates the link between the individual- and group-level judgments by extending R. D. Luce's (1959) model, which was originally developed for individual choice behavior, to a mixed-effects paired comparison model. It is shown that standard multilevel software for binary data can be used to estimate the model. The interpretation of the paired comparison parameters and statistical model tests are discussed in detail. An extensive analysis of an experimental study illustrates the usefulness of a hierarchical approach in modeling multiple pairwise judgments.     

On the importance of distinguishing between within- and between-subject effects in intransitive intertemporal choice

Most probabilistic paired comparison models treat inconsistent choices as caused by independent and random errors in the pairwise judgments. In this paper, we argue that this assumption is too restrictive for the analysis of paired comparison data obtained from multiple judges when transitivity violations are systematic. We present a new framework that contains the random error assumption as a special case but also allows for systematic changes in an option's utility assessments over the pairwise comparisons. Accounting for both between- and within-judge sources of variability, we demonstrate in an application on intertemporal choice that the proposed framework can capture systematic transitivity violations as well as individual taste differences.     

Modelling intransitive preferences: A random-effects approach

Current models for individual-level paired comparison data are based on the three assumptions that (1) pairwise judgments are independent, (2) the utility of an item remains invariant across trials, and (3) pair-specific variability can account for intransitive choice behaviour. All three assumptions seem strong and likely to be violated in empirical applications. This paper introduces a new framework for the analysis of paired comparison data which relaxes these three assumptions and considers the utilities associated with the same item across trials to be neither independent nor identical, but related. The proposed approach provides new insights about the reliability and consistency of paired comparison judgments and can account for systematic violations of transitivity. An application, based on a replication of Tversky's [(1969). Intransitivity of preference. Psychological Review, 76, 31-48] gamble study, illustrates the usefulness of the new approach in modelling both transitive and intransitive preferences.     

Paired comparison,triple comparison,and ranking experiments as generalized linear models,and their implementation on GLIM

A wide variety of paired comparison, triple comparison, and ranking experiments may be viewed as generalized linear models. These include paired comparison models based on both the Bradley-Terry and Thurstone-Mosteller approaches, as well as extensions of these models that allow for ties, order of presentation effects, and the presence of covariates. Moreover, the triple comparison model of Pendergrass and Bradley, as well as models for complete rankings of more than three items, can also be represented as generalized linear models. All such models can be easily fit by maximum likelihood, using the widely available GLIM computer package. Additionally, GLIM enables the computation of likelihood ratio statistics for testing many hypotheses of interest. Examples are presented that cover a variety of cases, along with their implementation on GLIM.     

The use of limit theorems in paired comparison model building

The Thurstone and the Bradley-Terry models, both initially advanced on intuitive grounds, have proved useful in the analysis of paired comparisons. The psychological meaning of these models and their relation to one another is unclear, but they fit data. Stevens has observed that there may be two basic mechanisms of discrimination 1) additive and 2) substitutive. We advance two corresponding mathematical models: that experienced sensation is 1) the sum of a large number of independent signals and 2) the maximum of a large number of independent signals. These assumptions yield 1) Thurstone's model and 2) the model of Bradley-Terry. Psychological interpretations of the various parameters, in terms of sensation, present themselves in a natural manner. Thus this paper presents a theory which unifies and interprets two paired comparison models that have proved useful in fitting experimental data.This research was supported in part by a National Science Foundation Grant GP 3807 and by the Army, Navy, and Air Force under ONR Contract # NONR-988(08), Task Order NR 042-004 with the Florida State University.     

A stochastic multidimensional scaling procedure for the empirical determination of convex indifference curves for preference/choice analysis

The vast majority of existing multidimensional scaling (MDS) procedures devised for the analysis of paired comparison preference/choice judgments are typically based on either scalar product (i.e., vector) or unfolding (i.e., ideal-point) models. Such methods tend to ignore many of the essential components of microeconomic theory including convex indifference curves, constrained utility maximization, demand functions, et cetera. This paper presents a new stochastic MDS procedure called MICROSCALE that attempts to operationalize many of these traditional microeconomic concepts. First, we briefly review several existing MDS models that operate on paired comparisons data, noting the particular nature of the utility functions implied by each class of models. These utility assumptions are then directly contrasted to those of microeconomic theory. The new maximum likelihood based procedure, MICROSCALE, is presented, as well as the technical details of the estimation procedure. The results of a Monte Carlo analysis investigating the performance of the algorithm as a number of model, data, and error factors are experimentally manipulated are provided. Finally, an illustration in consumer psychology concerning a convenience sample of thirty consumers providing paired comparisons judgments for some fourteen brands of over-the-counter analgesics is discussed.     

Remarks on the identifiability of thurstonian ranking models: Case V, case III, or neither?

It is well-known that the representations of the Thurstonian Case III and Case V models for paired comparison data are not unique. Similarly, when analyzing ranking data, other equivalent covariance structures can substitute for those given by Thurstone in these cases. That is, we may more broadly define the family of covariance structures satisfying Case III and Case V conditions. This paper introduces the notion of equivalence classes which defines a more meaningful partition of the covariance structures of the Thurstonian ranking models. In addition, the equivalence classes of Case V and Case III are completely characterized.     

A model for repeat paired comparison preference tests

A model is proposed that describes subject behavior on repeat paired comparison preference tests. The model extends prior work in this area in that it explicitly allows for abstentions and permits the derivation of individual true scores for discrimination ability as well as conditional estimates of proportionate preference. With these results, the properties of a paired comparison test can be thoroughly explored. An empirical example is presented, and test design issues are considered. In particular, repeat paired comparison preference tests are shown to be inherently less efficient discrimination tests than are pick 1 of 2 tests.     

Testing the assumptions and interpreting the results of the Rasch model using log-linear procedures in SPSS

This paper shows how to use the log-linear subroutine of SPSS to fit the Rasch model. It also shows how to fit less restrictive models obtained by relaxing specific assumptions of the Rasch model. Conditional maximum likelihood estimation was achieved by including dummy variables for the total scores as covariates in the models. This approach greatly simplifies the specification of the Rasch models. We illustrate these procedures in an analysis of four items selected from the Reiss Premarital Sexual Permissiveness Scale. We found that a modified version of the Rasch model with item dependencies fits the data significantly better than the simple Rasch model. We also found that the item difficulties are the same for men and women, but that the item dependencies are significantly greater for men. Apart from any substantive issues these results raise, the value of this exercise lies in its demonstration of how researchers can use the procedures of popular, accessible software packages to study an increasingly important set of measurement models.     

A latent markov model for the analysis of longitudinal data collected in continuous time: states, durations, and transitions

Markov models provide a general framework for analyzing and interpreting time dependencies in psychological applications. Recent work extended Markov models to the case of latent states because frequently psychological states are not directly observable and subject to measurement error. This article presents a further generalization of latent Markov models to allow for the analysis of rating data that are collected at arbitrary points in time. This extension offers new ways of investigating change processes by focusing explicitly on the durations that are spent in latent states. In an experience sampling application the author shows that such duration analyses can provide valuable insights about chronometric features of emotions.     

Eighteen‐month‐olds’ memory for short movies of simple stories

This study investigated twenty four 18‐month‐olds’ memory for dynamic visual stimuli. During the first visit participants saw one of two brief movies (30 seconds) with a simple storyline displayed in four iterations. After 2 weeks, memory was tested in the visual paired comparison paradigm in which the familiar and the novel movie were contrasted simultaneously and displayed in two iterations for a total of 60 seconds. Eye‐tracking revealed that participants fixated the familiar movie significantly more than the novel movie, thus indicating memory for the familiar movie. Furthermore, time‐dependent analysis of the data revealed that individual differences in the looking‐patterns for the first and second iteration of the movies were related to individual differences in productive vocabulary. We suggest that infants’ vocabulary may be indicative of their ability to understand and remember the storyline of the movies, thereby affecting their subsequent memory.     

The Effect of Prominence and Cue Association on Retrieval Processes: A Computational Account

We present a comprehensive empirical evaluation of the ACT‐R–based model of sentence processing developed by Lewis and Vasishth (2005) (LV05). The predictions of the model are compared with the results of a recent meta‐analysis of published reading studies on retrieval interference in reflexive‐/reciprocal‐antecedent and subject–verb dependencies (Jäger, Engelmann, & Vasishth, 2017). The comparison shows that the model has only partial success in explaining the data; and we propose that its prediction space is restricted by oversimplifying assumptions. We then implement a revised model that takes into account differences between individual experimental designs in terms of the prominence of the target and the distractor in memory‐ and context‐dependent cue‐feature associations. The predictions of the original and the revised model are quantitatively compared with the results of the meta‐analysis. Our simulations show that, compared to the original LV05 model, the revised model accounts for the data better. The results suggest that effects of prominence and variable cue‐feature associations need to be considered in the interpretation of existing empirical results and in the design and planning of future experiments. With regard to retrieval interference in sentence processing and to the broader field of psycholinguistic studies, we conclude that well‐specified models in tandem with high‐powered experiments are needed in order to uncover the underlying cognitive processes.     

Item Response Modeling of Paired Comparison and Ranking Data

The comparative format used in ranking and paired comparisons tasks can significantly reduce the impact of uniform response biases typically associated with rating scales. Thurstone's (1927, 1931) model provides a powerful framework for modeling comparative data such as paired comparisons and rankings. Although Thurstonian models are generally presented as scaling models, that is, stimuli-centered models, they can also be used as person-centered models. In this article, we discuss how Thurstone's model for comparative data can be formulated as item response theory models so that respondents' scores on underlying dimensions can be estimated. Item parameters and latent trait scores can be readily estimated using a widely used statistical modeling program. Simulation studies show that item characteristic curves can be accurately estimated with as few as 200 observations and that latent trait scores can be recovered to a high precision. Empirical examples are given to illustrate how the model may be applied in practice and to recommend guidelines for designing ranking and paired comparisons tasks in the future.     

Recognition-based inference: When is less more in the real world?

Common wisdom tells us that more information can only help and never hurt. Goldstein and Gigerenzer (2002) highlighted an instance violating this intuition. Specifically, in an analysis of their recognition heuristic, they found a counterintuitive less-is-more effect in inference: An individual recognizing fewer objects than another individual can, nevertheless, make more accurate inferences. Goldstein and Gigerenzer emphasized that a sufficient condition for this effect is that the recognition validity be higher than the knowledge validity, assuming that the validities are uncorrelated with the number of recognized objects, n. But how is the occurrence of the less-is-more effect affected when this independence assumption is violated? I show that validity dependencies (i.e., correlations of the validities with n) abound in empirical data sets, and I demonstrate by computer simulations that these dependencies often have a strong limiting effect on the less-is-more effect. Moreover, I discuss what cognitive (e.g., memory) and ecological (e.g., distribution of the criterion variable, environmental frequencies) factors can give rise to a dependency of the recognition validity on the number of recognized objects. Supplemental materials may be downloaded from http://pbr.psychonomic-journals.org/content/supplemental.     

Remarks on the identifiability of thurstonian paired comparison models under multiple judgment

It is well-known that the representations of the Thurstonian models for difference judgment data are not unique. It has been shown that equivalence classes can be formed to provide a more meaningful partition of the covariance structures of the Thurstonian ranking models. In this paper, we examine the equivalence relations between Thurstonian covariance structure models for paired comparison data obtained under multiple judgment and discuss their implications on the general identification constraints and methods to check for parameter identifiability in restricted models.The author is indebted to Ulf Böckenholt and Albert Maydeu-Olivares for their significant comments and suggestions which led to considerable improvement in this article.     

Empathy,sex and fluid intelligence as predictors of theory of mind

Individual differences in theory of mind (ToM) are affected by a variety of factors. We investigated the relationship between empathy, sex and fluid intelligence (FI) as predictors of ToM in a random probabilistic sample of secondary students. First, we explored whether sex, as well as high, average or low levels of empathy and FI affect ToM performance. Furthermore, we assessed the contribution of empathy, sex and FI in predicting ToM by using a path analysis. This method allows testing of causal models of directed dependencies among a set of variables. The causal dependencies of empathy, sex and fluid intelligence were confirmed and identified. In addition, the model confirmed the direct effect of empathy, sex and fluid intelligence on ToM; and the indirect effect of sex mediated by empathy. Thus, individual differences in ToM levels are partially attributable to sex, empathy and fluid intelligence variability, raising important considerations for clinical research as well as ToM’s theoretical models of domain specificity.     

Structural equation modeling of paired-comparison and ranking data

L. L. Thurstone's (1927) model provides a powerful framework for modeling individual differences in choice behavior. An overview of Thurstonian models for comparative data is provided, including the classical Case V and Case III models as well as more general choice models with unrestricted and factor-analytic covariance structures. A flow chart summarizes the model selection process. The authors show how to embed these models within a more familiar structural equation modeling (SEM) framework. The different special cases of Thurstone's model can be estimated with a popular SEM statistical package, including factor analysis models for paired comparisons and rankings. Only minor modifications are needed to accommodate both types of data. As a result, complex models for comparative judgments can be both estimated and tested efficiently.     

On the explaining‐away phenomenon in multivariate latent variable models

Many probabilistic models for psychological and educational measurements contain latent variables. Well‐known examples are factor analysis, item response theory, and latent class model families. We discuss what is referred to as the ‘explaining‐away’ phenomenon in the context of such latent variable models. This phenomenon can occur when multiple latent variables are related to the same observed variable, and can elicit seemingly counterintuitive conditional dependencies between latent variables given observed variables. We illustrate the implications of explaining away for a number of well‐known latent variable models by using both theoretical and real data examples.     

Is an attention-based associative account of adjacent and nonadjacent dependency learning valid?

Pacton and Perruchet (2008) reported that participants who were asked to process adjacent elements located within a sequence of digits learned adjacent dependencies but did not learn nonadjacent dependencies and conversely, participants who were asked to process nonadjacent digits learned nonadjacent dependencies but did not learn adjacent dependencies. In the present study, we showed that when participants were simply asked to read aloud the same sequences of digits, a task demand that did not require the intentional processing of specific elements as in standard statistical learning tasks, only adjacent dependencies were learned. The very same pattern was observed when digits were replaced by syllables. These results show that the perfect symmetry found in Pacton and Perruchet was not due to the fact that the processing of digits is less sensitive to their distance than the processing of syllables, tones, or visual shapes used in most statistical learning tasks. Moreover, the present results, completed with a reanalysis of the data collected in Pacton and Perruchet (2008), demonstrate that participants are highly sensitive to violations involving the spacing between paired elements. Overall, these results are consistent with the Pacton and Perruchet's single-process account of adjacent and nonadjacent dependencies, in which the joint attentional processing of the two events is a necessary and sufficient condition for learning the relation between them, irrespective of their distance. However, this account should be completed to encompass the notion that the presence or absence of an intermediate event is an intrinsic component of the representation of an association.     

Modeling caregiver adaptation over time: the longitudinal impact of behavior problems

Although cross-sectional research has established the link between care demands and various indicators of caregiver adaptation, few studies have examined the impact of care recipients' problematic behavior over time. The present analysis determines the importance of behavior problems when predicting rates of change in subjective stressors (role overload and role captivity) and depression. Using 4-wave longitudinal data (N = 137) on dementia caregivers, the authors fit individual growth curve models for care demands (i.e., behavior problems, activities of daily living dependencies, and cognitive impairment), subjective stressors, and depression. Subsequent structural equation models found that increases in behavior problems were most likely to predict increases in role overload. The findings emphasize the deleterious long-term impact of behavior problems on individuals' emotional adaptation to caregiving.