Test theory without an answer key

A general model is presented for homogeneous, dichotomous items when the answer key is not known a priori. The model is structurally related to the two-class latent structure model with the roles of respondents and items interchanged. For very small sets of respondents, iterative maximum likelihood estimates of the parameters can be obtained by existing methods. For other situations, new estimation methods are developed and assessed with Monte Carlo data. The answer key can be accurately reconstructed with relatively small sets of respondents. The model is useful when a researcher wants to study objectively the knowledge possessed by members of a culturally coherent group that the researcher is not a member of.This research was supported by NSF Grant No. SES-8320173 to the authors. We gratefully acknowledge comments and suggestions from John Boyd, Tarow Indow, and Kathy Maher as well as the editor and several anonymous referees.     

Factor invariance of the Millon Clinical Multiaxial Inventory

Several factor analyses of the Millon Clinical Multiaxial Inventory (MCMI) have resulted in very similar solutions. Interpretation of this consistency is hampered by the fact that the 20 scales of the inventory share items. Overlapping items cause the scales to be linearly dependent and may create structure in the interscale correlation matrix which is separate from the subject response patterns. A factor analysis was performed on the matrix of item-overlap coefficients which describes the underlying artifactual structure of the instrument. Data from two new subject samples were factor analyzed and compared to previously published studies. Similarity coefficients among factors across studies were calculated.     

EFFECTS OF VIDEO-ASSISTED TRAINING ON EMPLOYMENT-RELATED SOCIAL SKILLS OF ADULTS WITH SEVERE MENTAL RETARDATION

Two studies investigated effects of video-assisted training on employment-related social skills of adults with severe mental retardation. In video-assisted training, participants discriminated a model's behavior on videotape and received feedback from the trainer for responses to questions about video scenes. In the first study, 3 adults in an employment program participated in video-assisted training to request their supervisor's assistance when encountering work problems. Results indicated that participants discriminated the target behavior on video but effects did not generalize to the work setting for 2 participants until they rehearsed the behavior. In the second study, 2 participants were taught to fix and report four work problems using video-assisted procedures. Results indicated that after participants rehearsed how to fix and report one or two work problems, they began to fix and report the remaining problems with video-assisted training alone.     

Examining differential item functioning due to item difficulty and alternative attractiveness

A method for analyzing test item responses is proposed to examine differential item functioning (DIF) in multiple-choice items through a combination of the usual notion of DIF, for correct/incorrect responses and information about DIF contained in each of the alternatives. The proposed method uses incomplete latent class models to examine whether DIF is caused by the attractiveness of the alternatives, difficulty of the item, or both. DIF with respect to either known or unknown subgroups can be tested by a likelihood ratio test that is asymptotically distributed as a chi-square random variable.     

Beyond principal component analysis: A trilinear decomposition model and least squares estimation

The paper derives sufficient conditions for the consistency and asymptotic normality of the least squares estimator of a trilinear decomposition model for multiway data analysis.     

Criterion-related construct validity

Established results on latent variable models are applied to the study of the validity of a psychological test. When the test predicts a criterion by measuring a unidimensional latent construct, not only must the total score predict the criterion, but the joint distribution of criterion scores and item responses must exhibit a certain pattern. The presence of this population pattern may be tested with sample data using the stratified Wilcoxon rank sum test. Often, criterion information is available only for selected examinees, for instance, those who are admitted or hired. Three cases are discussed: (i) selection at random, (ii) selection based on the current test, and (iii) selection based on other measures of the latent construct. Discriminant validity is also discussed.This work was supported in part by Grant SES-87-01890 from the Measurement Methods and Data Improvement Program of the U.S. National Science Foundation.     

Influential observations in principal factor analysis

We propose a method for detecting influential observations in iterative principal factor analysis. For this purpose we derive the influence functionsI(x; LL ^T ) andI(x; ) for the common variance matrixT =LL ^T and the unique variance matrix , respectively, in the common factor decomposition =LL ^T + . A numerical example is given for illustration.The authors are grateful to Tomoyuki Tarumi and Atsuhiro Hayashi for their kind permission to use their software Seto/B for drawing Figures 1 and 2 and to anonymous reviewers for comments on the paper.     

Using item-specific instructional information in achievement modeling

The problem of detecting instructional sensitivity (item basis) in test items is considered. An illustration is given which shows that for tests with many biased items, traditional item bias detection schemes give a very poor assessment of bias. A new method is proposed instead. This method extends item response theory (IRT) by including item-specific auxiliary measurement information related to opportunity-to-learn. Item-specific variation in measurement relations across students with varying opportunity-to-learn is allowed for.This paper was presented at the 1987 AERA meeting in Washington, DC. This research was supported by grant OERI-G-86-003 from the Office of Educational Research and Improvement, Department of Education. The author thanks Michael Hollis and Chih-fen Kao for valuable research assistance, and appreciates valuable comments made by an anonymous reviewer.     

The social phobia and anxiety inventory: Construct validity

The Social Phobia and Anxiety Inventory (SPAI) is a new instrument composed of social phobia and agoraphobia subscales. The latter scale is used to detect social anxiety that may result from agoraphobia. The SPAI's construct validity was assessed through several procedures. First, confirmatory factor analyses were conducted to validate the existence of the two subscales. Second, exploratory factor analyses examined the underlying structure of the social phobia subscale. Third, a Q factor procedure determined if different anxiety diagnostic groups could be differentiated by their SPAI response pattern. The results confirmed the utility of the two SPAI subscales and identified a number of dimensions contained within the social phobia subscale which differed depending upon the specific subject sample. In addition, the complaints of social phobies appeared more homogeneous than those of an agoraphobic comparison group. The results are discussed in terms of construct validity and the sensitivity of the SPAI to various dimensions of social phobia fears.This study was supported in part by NIMH Grants 41852, 30915, 18269, and 16884.     

TEACHING AUTISTIC CHILDREN CONVERSATIONAL SPEECH USING VIDEO MODELING

We assessed the effects of video modeling on acquisition and generalization of conversational skills among autistic children. Three autistic boys observed videotaped conversations consisting of two people discussing specific toys. When criterion for learning was met, generalization of conversational skills was assessed with untrained topics of conversation; new stimuli (toys); unfamiliar persons, siblings, and autistic peers; and other settings. The results indicated that the children learned through video modeling, generalized their conversational skills, and maintained conversational speech over a 15-month period. Video modeling shows much promise as a rapid and effective procedure for teaching complex verbal skills such as conversational speech.