Assessing the distinguishability of models and the informativeness of data

A difficulty in the development and testing of psychological models is that they are typically evaluated solely on their ability to fit experimental data, with little consideration given to their ability to fit other possible data patterns. By examining how well model A fits data generated by model B, and vice versa (a technique that we call landscaping), much safer inferences can be made about the meaning of a model's fit to data. We demonstrate the landscaping technique using four models of retention and 77 historical data sets, and show how the method can be used to: (1) evaluate the distinguishability of models, (2) evaluate the informativeness of data in distinguishing between models, and (3) suggest new ways to distinguish between models. The generality of the method is demonstrated in two other research areas (information integration and categorization), and its relationship to the important notion of model complexity is discussed.     

Confirmatory factor analysis of the Penn State Worry Questionnaire: Multiple factors or method effects?

The latent structure of the Penn State Worry Questionnaire (PSWQ) was evaluated with confirmatory factor analyses (CFAs) in 1200 outpatients with DSM-IV anxiety and mood disorders. Of particular interest was the comparative fit and interpretability of a two-factor solution (cf. Behaviour Research and Therapy 40 (2002) 313) vs. a one-factor model that specified method effects arising from five reverse-worded items. Consistent with prediction, the superiority of the one-factor model was demonstrated in split-sample CFA replications (ns=600). Multiple-group CFAs indicated that the measurement properties of the PSWQ were invariant in male and female patients. In addition to their direct relevance to the psychometrics of the PSWQ, the results are discussed in regard to methodological considerations for using factor analytic methods in the evaluation of psychological tests.     

Influence of consonantal context on the pronunciation of vowels: a comparison of human readers and computational models

In two experiments, we found that college students' pronunciations of vowels in nonwords are influenced both by preceding and following consonants. The predominance of rimes in previous studies of reading does not appear to arise because readers are unable to pick up associations that cross the onset-rime boundary, but rather because English has relatively few such associations. Comparisons between people's vowel pronunciations and those produced by various computational models of reading showed that no model provided a good account of human performance on nonwords for which the vowel shows contextual conditioning. Possible directions for improved models are suggested.     

A factor simplicity index

We propose an index for assessing the degree of factor simplicity in the context of principal components and exploratory factor analysis. The new index, which is called Loading Simplicity, is based on the idea that the communality of each variable should be related to few components, or factors, so that the loadings in each variable are either zero or as far from zero as possible. This index does not depend on the scale of the factors, and its maximum and minimum are only related to the degree of simplicity in the loading matrix. The aim of the index is to enable the degree of simplicity in loading matrices to be compared.The author would like to thank the review team for their insights and recommendations. This work was supported by a grant SEC2001-3821-C05-C02 from the Spanish Ministry of Science and Technology.     

Remarks on the identifiability of thurstonian paired comparison models under multiple judgment

It is well-known that the representations of the Thurstonian models for difference judgment data are not unique. It has been shown that equivalence classes can be formed to provide a more meaningful partition of the covariance structures of the Thurstonian ranking models. In this paper, we examine the equivalence relations between Thurstonian covariance structure models for paired comparison data obtained under multiple judgment and discuss their implications on the general identification constraints and methods to check for parameter identifiability in restricted models.The author is indebted to Ulf Böckenholt and Albert Maydeu-Olivares for their significant comments and suggestions which led to considerable improvement in this article.     

Orthomax rotation and perfect simple structure

A loading matrix has perfect simple structure if each row has at most one nonzero element. It is shown that if there is an orthogonal rotation of an initial loading matrix that has perfect simple structure, then orthomax rotation with 0 1 of the initial loading matrix will produce the perfect simple structure. In particular, varimax and quartimax will produce rotations with perfect simple structure whenever they exist.     

Model comparison of nonlinear structural equation models with fixed covariates

Recently, it has been recognized that the commonly used linear structural equation model is inadequate to deal with some complicated substantive theory. A new nonlinear structural equation model with fixed covariates is proposed in this article. A procedure, which utilizes the powerful path sampling for computing the Bayes factor, is developed for model comparison. In the implementation, the required random observations are simulated via a hybrid algorithm that combines the Gibbs sampler and the Metropolis-Hastings algorithm. It is shown that the proposed procedure is efficient and flexible; and it produces Bayesian estimates of the parameters, latent variables, and their highest posterior density intervals as by-products. Empirical performances of the proposed procedure such as sensitivity to prior inputs are illustrated by a simulation study and a real example.This research is fully supported by a grant from the Research Grant Council of the Hong Kong Special Administrative Region, China (Project No. CUHK 4346/01H). The authors are thankful to the Editor, the Associate Editor, and anonymous reviewers for valuable comments which improve the paper significantly, and grateful to ICPSR and the relevant funding agency for allowing use of the data in the example. The assistance of Michael K.H. Leung and Esther L.S. Tam is gratefully acknowledged.     

PUNISHMENT IN HUMAN CHOICE: DIRECT OR COMPETITIVE SUPPRESSION?

This investigation compared the predictions of two models describing the integration of reinforcement and punishment effects in operant choice. Deluty's (1976) competitive-suppression model (conceptually related to two-factor punishment theories) and de Villiers' (1980) direct-suppression model (conceptually related to one-factor punishment theories) have been tested previously in nonhumans but not at the individual level in humans. Mouse clicking by college students was maintained in a two-alternative concurrent schedule of variable-interval money reinforcement. Punishment consisted of variable-interval money losses. Experiment 1 verified that money loss was an effective punisher in this context. Experiment 2 consisted of qualitative model comparisons similar to those used in previous studies involving nonhumans. Following a no-punishment baseline, punishment was superimposed upon both response alternatives. Under schedule values for which the direct-suppression model, but not the competitive-suppression model, predicted distinct shifts from baseline performance, or vice versa, 12 of 14 individual-subject functions, generated by 7 subjects, supported the direct-suppression model. When the punishment models were converted to the form of the generalized matching law, least-squares linear regression fits for a direct-suppression model were superior to those of a competitive-suppression model for 6 of 7 subjects. In Experiment 3, a more thorough quantitative test of the modified models, fits for a direct-suppression model were superior in 11 of 13 cases. These results correspond well to those of investigations conducted with nonhumans and provide the first individual-subject evidence that a direct-suppression model, evaluated both qualitatively and quantitatively, describes human punishment better than a competitive-suppression model. We discuss implications for developing better punishment models and future investigations of punishment in human choice.     

Global and local covert visual attention: Evidence from a bayesian hidden markov model

Psychological, psychophysical and physiological research indicates that people switch between two covert attention states, local and global attention, while visually exploring complex scenes. The focus in the local attention state is on specific aspects and details of the scene, and on examining its content with greater visual detail. The focus in the global attention state is on exploring the informative and perceptually salient areas of the scene, and possibly on integrating the information contained therein. The existence of these two visual attention states, their relative prevalence and sequence in time has remained empirically untested to date. To fill this gap, we develop a psychometric model of visual covert attention that extends recent work on hidden Markov models, and we test it using eye-movement data. The model aims to describe the observed time series of saccades typically collected in eye-movement research by assuming a latent Markov process, indicative of the brain switching between global and local covert attention. We allow subjects to be in either state while exploring a stimulus visually, and to switch between them an arbitrary number of times. We relax the no-memory-property of the Markov chain. The model that we develop is estimated with MCMC methodology and calibrated on eye-movement data collected in a study of consumers' attention to print advertisements in magazines.We thank Dominique Claessens of Verify International for making the data available to us.