A FORTRAN 77 program for a nonparametric item response model: The Mokken scale analysis

A nonparametric item response theory model—the Mokken scale analysis (a stochastic elaboration of the deterministic Guttman scale)—and a computer program that performs this analysis are described. Three procedures of scaling are distinguished: a search procedure, an evaluation of the whole set of items, and an extension of an existing scale. All procedures provide a coefficient of scalability for all items that meet the criteria of the Mokken model and an item coefficient of scalability for every item. Four different types of reliability coefficient are computed both for the entire set of items and for the scalable items. A test of robustness of the found scale can be performed to analyze whether the scale is invariant across different subgroups or samples. This robustness test serves as a goodness of fit test for the established scale. The program is written in FORTRAN 77. Two versions are available, an SPSS-X procedure program (which can be used with the SPSS-X mainframe package) and a stand-alone program suitable for both mainframe and microcomputers.     

Development and validation of a brief version of the dyadic adjustment scale with a nonparametric item analysis model

The main purpose of the current research was to develop an abbreviated form of the Dyadic Adjustment Scale (DAS) with nonparametric item response theory. The authors conducted 5 studies, with a total participation of 8,256 married or cohabiting individuals. Results showed that the item characteristic curves behaved in a monotonically increasing function for all items of the brief 4-item version of the DAS (DAS-4). The DAS-4 proved to be informative at all levels of couple satisfaction. Compared with the 32-item version of the DAS (DAS-32), it was as effective in predicting couple dissolution and was significantly less contaminated by socially desirable responding. In addition, structural equation modeling demonstrated that the underlying latent construct measured by the DAS-4 was very stable over a 2-year period. This brief version has the advantage of being less time consuming and constitutes a promising alternative to the original DAS-32 for clinicians and researchers.     

Asymptotic identifiability of nonparametric item response models

The identifiability of item response models with nonparametrically specified item characteristic curves is considered. Strict identifiability is achieved, with a fixed latent trait distribution, when only a single set of item characteristic curves can possibly generate the manifest distribution of the item responses. When item characteristic curves belong to a very general class, this property cannot be achieved. However, for assessments with many items, it is shown that all models for the manifest distribution have item characteristic curves that are very near one another and pointwise differences between them converge to zero at all values of the latent trait as the number of items increases. An upper bound for the rate at which this convergence takes place is given. The main result provides theoretical support to the practice of nonparametric item response modeling, by showing that models for long assessments have the property of asymptotic identifiability. The research was partially supported by the National Institute of Health grant R01 CA81068-01.     

Reliability of test scores in nonparametric item response theory

Three methods for estimating reliability are studied within the context of nonparametric item response theory. Two were proposed originally by Mokken (1971) and a third is developed in this paper. Using a Monte Carlo strategy, these three estimation methods are compared with four classical lower bounds to reliability. Finally, recommendations are given concerning the use of these estimation methods.The authors are grateful for constructive comments from the reviewers and from Charles Lewis.     

Validity of the Learning Process Questionnaire with students of lower academic attainment

The Learning Process Questionnaire was administered to 692 secondary school students (308 boys, 380 girls, and 4 unspecified) in the Philippines. The ages of the participants ranged from 14 to 22 years, with an average of 15 yr. An exploratory factor-analysis procedure was conducted for the entire sample, for the lower achievers (n = 333), and for higher achievers (n = 359). The factor structure of the Learning Process Questionnaire was uninterpretable with data obtained from students of lower academic achievement.     

A structural examination of the Learning Experiences Questionnaire

The underlying factor structure of the Learning Experiences Questionnaire (LEQ; Schaub, 2004) was examined using data from 742 male and female college-age respondents. The LEQ items reflect a variety of learning experiences (generated based on Bandura's (1986, 1997) four sources of self-efficacy perceptions) that might occur in each of Holland's (1997) six RIASEC domains. The LEQ was expected to have a multi-dimensional factor structure, with dimensions representing both RIASEC context and learning experience type. A multi-trait multi-method framework was used to compare the relative fits of a series of models with different theoretically implied dimensionalities, estimated using confirmatory factor analysis. Results supported the proposed multi-dimensional structure. Results of follow-up tests of measurement invariance for men and women revealed that configural and partial factorial invariance held, implying that the LEQ measure functions similarly for men and women.     

Psychometric item evaluations of the Recovery-Stress Questionnaire for athletes

ObjectivesUsing an item-based analysis, the factor structure of the Recovery-Stress Questionnaire for Athletes (RESTQ-Sport) [Kellmann, M., & Kallus, K.W. (2001). Recovery-Stress Questionnaire for Athletes: User manual. Champaign, IL: Human Kinetics] was assessed as a test of its validity. The RESTQ-Sport is a 76-item questionnaire that was developed to assess the physical and mental impact of training stress and to facilitate the formulation of strategies for the enhancement of recovery. According to [Kellmann, M., & Kallus, K.W. (2001). Recovery-Stress Questionnaire for Athletes: User manual. Champaign, IL: Human Kinetics], two factors: Recovery and Stress, comprised the 12 General subscales and seven Sport subscales of the RESTQ-Sport.MethodsA total of 585 male and female athletes who train at a Canadian national sport center were recruited to complete the RESTQ-Sport. Maximum likelihood factor analyses were performed.ResultsThe results confirmed the two-factor structure proposed by Kellmann and Kallus for the Sport-Recovery/Stress Scale but disconfirmed this structure for the General-Recovery/Stress Scale. Item analysis further disconfirmed the two-factor structure for the General Scale and failed to confirm the 19 Subscales proposed by the authors on both of the General and Sport Scales.ConclusionsThese results are interpreted to suggest that while the questionnaire should not be considered to be a diagnostic tool for under-recovery states, the RESTQ-Sport does, nonetheless, still measure general parameters of training stress which can be tracked in recovery planning.     

Analyzing psychopathology items: a case for nonparametric item response theory modeling

The authors discuss the applicability of nonparametric item response theory (IRT) models to the construction and psychometric analysis of personality and psychopathology scales, and they contrast these models with parametric IRT models. They describe the fit of nonparametric IRT to the Depression content scale of the Minnesota Multiphasic Personality Inventory--2 (J. N. Butcher, W. G. Dahlstrom, J. R. Graham, A. Tellegen, & B. Kaemmer, 1989). They also show how nonparametric IRT models can easily be applied and how misleading results from parametric IRT models can be avoided. They recommend the use of nonparametric IRT modeling prior to using parametric logistic models when investigating personality data.     

A brief inventory of Interpersonal Problems-Circumplex using nonparametric item response theory: introducing the IIP-C-IRT

Nonparametric item response theory methods were applied to the responses of 1,000 college students on the 64 items of the Inventory of Interpersonal Problems-Circumplex (IIP-C; Alden, Wiggins, & Pincus, 1990) to develop an abbreviated 32-item version of the instrument. In a separate validation sample of 981 students, the newly selected scale items did not show evidence of differential item functioning across males and females. There was high convergence found between the new scales and IIP-C parent scales, along with commensurate or improved fits to the circular structural model relative to the full scale and its existing brief derivatives-the IIP-32 and the IIP-SC. Results provide evidence that the new brief scales can improve the level of precision and information yielded in brief assessments of interpersonal problems without gender bias.     

Joint consistency of nonparametric item characteristic curve and ability estimation

The simultaneous and nonparametric estimation of latent abilities and item characteristic curves is considered. The asymptotic properties of ordinal ability estimation and kernel smoothed nonparametric item characteristic curve estimation are investigated under very general assumptions on the underlying item response theory model as both the test length and the sample size increase. A large deviation probability inequality is stated for ordinal ability estimation. The mean squared error of kernel smoothed item characteristic curve estimates is studied and a strong consistency result is obtained showing that the worst case error in the item characteristic curve estimates over all items and ability levels converges to zero with probability equal to one.     

Efficient nonparametric approaches for estimating the operating characteristics of discrete item responses

Rationale and the actual procedures of two nonparametric approaches, called Bivariate P.D.F. Approach and Conditional P.D.F. Approach, for estimating the operating characteristic of a discrete item response, or the conditional probability, given latent trait, that the examinee's response be that specific response, are introduced and discussed. These methods are featured by the facts that: (a) estimation is made without assuming any mathematical forms, and (b) it is based upon a relatively small sample of several hundred to a few thousand examinees.Some examples of the results obtained by the Simple Sum Procedure and the Differential Weight Procedure of the Conditional P.D.F. Approach are given, using simulated data. The usefulness of these nonparametric methods is also discussed.This research was mostly supported by the Office of Naval Research (N00014-77-C-0360, N00014-81-C-0569, N00014-87-K-0320, N00014-90-J-1456).     

Examining the multidimensionality of approaches to learning through the development of a revised version of the Learning Process Questionnaire

Aim: This study aimed to produce a revised two‐factor version of the Learning Process Questionnaire (R‐LPQ‐2F) with deep and surface approach scales, measured by a reasonably small number of items, suitable for use by teachers in secondary schools to evaluate the learning approaches of their students. Method: A set of 41 items was derived, with modification, from the original version of the LPQ and from items used to develop the revised version of the SPQ. These items were tested using reliability procedures and confirmatory factor analysis and items were deleted until scales were of a suitable length and confirmatory factor analysis indicated a good fit to the intended two‐factor structure. Sample: The sample consisted of 841 students from 20 secondary schools in Hong Kong. Results: The final two‐factor version of the questionnaire had good Cronbach alpha values and reasonable goodness‐of‐fit values for the confirmatory analysis. There was a much better fit, though, to a hierarchical structure with motive and strategy subscales for each approach, each of which, in turn, had two subcomponents. Conclusion: Approaches to learning have a hierarchical dimensionality with motive and strategy elements. Each motive and strategy element is itself multidimensional. The results are used to question the conventional approach to the testing and acceptance of instruments, which place sole reliance upon reliability tests. The use of confirmatory factor analysis is recommended as a routine procedure in the development and testing of instruments.     

The item structure of the Eysenck Personality Questionnaire across method and culture

Since its publication, the Eysenck Personality Questionnaire (EPQ) has endured much criticism of its psychometric properties. As the most stringent of these attacks has derived from hierarchical factor analytic studies it is reasonable to ask whether the inadequacies reported accrue to the questionnaire itself or to the problematic features of factor analysis as a method. A further feature of empirical studies that are critical of the EPQ is that they are, by-and-large, based upon non-British samples. As the EPQ was constructed in Britain it is quite conceivable that cultural artifacts are responsible for the inconsistent findings of Canadian and American researchers. This paper offers a brief review of some of the critical research and reports a study in which the item structure of the EPQ is explored by non-metric multidimensional scaling techniques upon a sample of Irish adolescents. The MDS configuration was then compared with the higher order factor analytic results of American researchers to examine invariance across method. A clear item structure emerges supporting the Eysencks' theoretical model although items of the P scale are less homogenous than would be ideal for a psychometric tool.     

A psychometric analysis of the Personal Attributes Questionnaire

The psychometric properties of the Personal Attributes Questionnaire were examined in independent samples of male and female high school students, college students, and adults. In each of the six samples a two-factor structure (masculinity/instrumentality and femininity/expressivity) paralleling the empirically derived scales was found. Additional factor analyses of negative masculine and feminine traits were reported. Discriminant analyses revealed highly significant differentiation between the sexes. The reliabilities (Cronbach alpha) of the unit-weighted scales in each sample were also satisfactory.Support for this research was provided by NASA Grant NSG 2065 (Robert L. Helmreich, Principal Investigator) and NSF Grant BNS 78-08911 (Janet T. Spence and Robert L. Helmreich, Principal Investigators). The article was prepared while the second author was a Fellow at the Center for Advanced Study in the Behavioral Sciences, with support from the Spencer Foundation, NIMH Grant 5T32MH14581-03, and the University of Texas Research Institute.     

A multigroup item response theory analysis of the psychopathy checklist--revised

Item response theory was used to investigate the functioning of the Psychopathy Checklist-Revised (PCL-R; R. D. Hare, 1991, 2003) in several offender populations. With male criminal offenders (N=3,847) as a reference group, differential item functioning analyses were performed for 3 comparison groups: female criminal offenders (N=1,219), male forensic psychiatric patients (N=1,246), and male criminal offenders scored from file reviews (N=2,626). Results are discussed in the context of the 2-factor, 4-facet model for the PCL-R (R. D. Hare, 2003; J. Parker, G. Sitarenios, & R. D. Hare, 2003). Application of a multigroup graded response model to all 4 groups suggests scalar equivalence may hold at least approximately for each population, although the PCL-R provided slightly greater information about the latent trait of psychopathy for male criminal offenders scored from the standard procedure.     

A similarity-based smoothing approach to nondimensional item analysis

The probability that an examinee chooses a particular option within an item is estimated by averaging over the responses to that item of examinees with similar response patterns for the whole test. The approach does not presume any latent variable structure or any dimensionality. But simulated and actual data analyses are presented to show that when the responses are determined by a latent ability variable, this similarity-based smoothing procedure can reveal the dimensionality of ability very satisfactorily.The author wishes to acknowledge the support of the Natural Sciences and Engineering Research Council of Canada through grant A320, and to thank Educational Testing Service for making the data on the Advanced Placement Chemistry Exam available.     

A note on item analysis and the criterion of internal consistency

The implications contained in Richardson's article on item analysis in March 1936 issue ofPsychometrika are examined in the light of multiple factor theory. It is shown that item analysis is a necessary, but not a sufficient condition for the construction of a test which shall measure a single trait. The intercorrelations of certain items selected by a method of item analysis are examined, found to contain many zero and some negative correlations. Multiple factor analysis showed that eight traits were measured by the items which had been asserted to measure only one.     

Colorado Learning Difficulties Questionnaire: validation of a parent-report screening measure

This study evaluated the internal structure and convergent and discriminant evidence for the Colorado Learning Difficulties Questionnaire (CLDQ), a 20-item parent-report rating scale that was developed to provide a brief screening measure for learning difficulties. CLDQ ratings were obtained from parents of children in 2 large community samples and 2 samples from clinics that specialize in the assessment of learning disabilities and related disorders (total N = 8,004). Exploratory and confirmatory factor analyses revealed 5 correlated but separable dimensions that were labeled reading, math, social cognition, social anxiety, and spatial difficulties. Results revealed strong convergent and discriminant evidence for the CLDQ Reading scale, suggesting that this scale may provide a useful method to screen for reading difficulties in both research studies and clinical settings. Results are also promising for the other 4 CLDQ scales, but additional research is needed to refine each of these measures.