Test theory without an answer key

A general model is presented for homogeneous, dichotomous items when the answer key is not known a priori. The model is structurally related to the two-class latent structure model with the roles of respondents and items interchanged. For very small sets of respondents, iterative maximum likelihood estimates of the parameters can be obtained by existing methods. For other situations, new estimation methods are developed and assessed with Monte Carlo data. The answer key can be accurately reconstructed with relatively small sets of respondents. The model is useful when a researcher wants to study objectively the knowledge possessed by members of a culturally coherent group that the researcher is not a member of.This research was supported by NSF Grant No. SES-8320173 to the authors. We gratefully acknowledge comments and suggestions from John Boyd, Tarow Indow, and Kathy Maher as well as the editor and several anonymous referees.     

EFFECTS OF VIDEO-ASSISTED TRAINING ON EMPLOYMENT-RELATED SOCIAL SKILLS OF ADULTS WITH SEVERE MENTAL RETARDATION

Two studies investigated effects of video-assisted training on employment-related social skills of adults with severe mental retardation. In video-assisted training, participants discriminated a model's behavior on videotape and received feedback from the trainer for responses to questions about video scenes. In the first study, 3 adults in an employment program participated in video-assisted training to request their supervisor's assistance when encountering work problems. Results indicated that participants discriminated the target behavior on video but effects did not generalize to the work setting for 2 participants until they rehearsed the behavior. In the second study, 2 participants were taught to fix and report four work problems using video-assisted procedures. Results indicated that after participants rehearsed how to fix and report one or two work problems, they began to fix and report the remaining problems with video-assisted training alone.     

Examining differential item functioning due to item difficulty and alternative attractiveness

A method for analyzing test item responses is proposed to examine differential item functioning (DIF) in multiple-choice items through a combination of the usual notion of DIF, for correct/incorrect responses and information about DIF contained in each of the alternatives. The proposed method uses incomplete latent class models to examine whether DIF is caused by the attractiveness of the alternatives, difficulty of the item, or both. DIF with respect to either known or unknown subgroups can be tested by a likelihood ratio test that is asymptotically distributed as a chi-square random variable.     

Criterion-related construct validity

Established results on latent variable models are applied to the study of the validity of a psychological test. When the test predicts a criterion by measuring a unidimensional latent construct, not only must the total score predict the criterion, but the joint distribution of criterion scores and item responses must exhibit a certain pattern. The presence of this population pattern may be tested with sample data using the stratified Wilcoxon rank sum test. Often, criterion information is available only for selected examinees, for instance, those who are admitted or hired. Three cases are discussed: (i) selection at random, (ii) selection based on the current test, and (iii) selection based on other measures of the latent construct. Discriminant validity is also discussed.This work was supported in part by Grant SES-87-01890 from the Measurement Methods and Data Improvement Program of the U.S. National Science Foundation.     

Using item-specific instructional information in achievement modeling

The problem of detecting instructional sensitivity (item basis) in test items is considered. An illustration is given which shows that for tests with many biased items, traditional item bias detection schemes give a very poor assessment of bias. A new method is proposed instead. This method extends item response theory (IRT) by including item-specific auxiliary measurement information related to opportunity-to-learn. Item-specific variation in measurement relations across students with varying opportunity-to-learn is allowed for.This paper was presented at the 1987 AERA meeting in Washington, DC. This research was supported by grant OERI-G-86-003 from the Office of Educational Research and Improvement, Department of Education. The author thanks Michael Hollis and Chih-fen Kao for valuable research assistance, and appreciates valuable comments made by an anonymous reviewer.     

TEACHING AUTISTIC CHILDREN CONVERSATIONAL SPEECH USING VIDEO MODELING

We assessed the effects of video modeling on acquisition and generalization of conversational skills among autistic children. Three autistic boys observed videotaped conversations consisting of two people discussing specific toys. When criterion for learning was met, generalization of conversational skills was assessed with untrained topics of conversation; new stimuli (toys); unfamiliar persons, siblings, and autistic peers; and other settings. The results indicated that the children learned through video modeling, generalized their conversational skills, and maintained conversational speech over a 15-month period. Video modeling shows much promise as a rapid and effective procedure for teaching complex verbal skills such as conversational speech.     

A structural model for postpartum responses to the somatic and cognitive items on the Beck Depression Inventory

A distinction can be drawn between those items on the Beck Depression Inventory (BDI) that reflect mainly cognitive and emotional symptoms and those that reflect somatic symptoms. Responses to the BDI were obtained from women at 3, 6, 9, and 24 weeks postpartum. The covariances of cognitive and somatic symptom scores across the four time points were closely fit by a structural equations model referred to as the simplex. The modeling indicates that earlier cognitive scores directly predict later somatic scores but that early somatic scores do not directly predict later cognitive scores. Other data confirm the results.Much of this research was supported while the first author was an NIMH Fellow at the University of Iowa (MH15168).     

A nonparametric approach for assessing latent trait unidimensionality

Assuming a nonparametric family of item response theory models, a theory-based procedure for testing the hypothesis of unidimensionality of the latent space is proposed. The asymptotic distribution of the test statistic is derived assuming unidimensionality, thereby establishing an asymptotically valid statistical test of the unidimensionality of the latent trait. Based upon a new notion of dimensionality, the test is shown to have asymptotic power 1. A 6300 trial Monte Carlo study using published item parameter estimates of widely used standardized tests indicates conservative adherence to the nominal level of significance and statistical power averaging 81 out of 100 rejections for examinee sample sizes and psychological test lengths often incurred in practice.The referees' comments were remarkably detailed and greatly enhanced the writeup and sensitized the author to certain pertinent issues. Discussions with Fritz Drasgow, Lloyd Humphreys, Dennis Jennings, Brian Junker, Robert Linn, Ratna Nandakumar, and Robin Shealy were also very useful.This research was supported by the Office of Naval Research under grant N00014-84-K-0186; NR 150-533, and by the National Science Foundation under grant DMS 85-03321.     

Latent trait models and dichotomization of graded responses

This paper discusses thecompatibility of the polychotomous Rasch model with dichotomization of the response continuum. It is argued that in the case of graded responses, the response categories presented to the subject are essentially an arbitrary polychotomization of the response continuum, ranging for example from total rejection or disagreement to total acceptance or agreement of an item or statement. Because of this arbitrariness, the measurement outcome should be independent of the specific polychotomization applied, for example, presenting a specific multicategory response format should not affect the measurement outcome. When such is the case, the original polychotomous model is called compatible with dichotomization.A distinction is made between polychotomization or dichotomization before the fact, that is, in constructing the response format, and polycho- or dichotomization after the fact, for example in dichotomizing existing graded response data.It is shown that, at least in case of dichotomization after-the-fact, the polychotomous Rasch model is not compatible with dichotomization, unless a rather special condition of the model parameters is met. Insofar as it may be argued that dichotomization before the fact is not essentially different from dichotomization after the fact, the value of the unidimensional polychotomous Rasch model is consequently questionable. The impact of our conclusion on related models is also discussed.     

Parent and child adjustment dual trajectories at the beginning of the COVID-19 syndemic

Children and their families have been significantly impacted by the unfolding of the COVID-19 syndemic. We sought to identify (1) groups of families with distinct profiles of joint trajectories of parental anxiety and child emotional distress and (2) protective and risk factors associated with these dual-trajectory profiles. A sample of 488 parents (65% White; 77% mothers) with 3- to 8-year-old children (M_Age = 5.04, SD_Age = 1.59) was followed from late March to early July in 2020. Survey data on parent (i.e., anxiety symptoms) and child (i.e., emotional distress) adjustment were collected at three time points. Using multivariate growth mixture modeling, we identified one group with low parental anxiety and child emotional distress (42.7%) and three other distinct groups with varying risk levels among parents and/or children. We also identified protective (e.g., positive parenting) and risk (e.g., child negative affect, negative parenting, perceived stress with racism) factors in predicting parent and child adjustment. It can be concluded that, overall, our sample (mostly middle- and high-socioeconomic status families) demonstrated family resilience amid COVID-19, consistent with prior disaster coping literature. At the same time, our findings also indicated the need to identify at-risk families and modifiable factors for post-disaster public health interventions.