Test theory without an answer key

A general model is presented for homogeneous, dichotomous items when the answer key is not known a priori. The model is structurally related to the two-class latent structure model with the roles of respondents and items interchanged. For very small sets of respondents, iterative maximum likelihood estimates of the parameters can be obtained by existing methods. For other situations, new estimation methods are developed and assessed with Monte Carlo data. The answer key can be accurately reconstructed with relatively small sets of respondents. The model is useful when a researcher wants to study objectively the knowledge possessed by members of a culturally coherent group that the researcher is not a member of.This research was supported by NSF Grant No. SES-8320173 to the authors. We gratefully acknowledge comments and suggestions from John Boyd, Tarow Indow, and Kathy Maher as well as the editor and several anonymous referees.     

COVARIATION WITHIN FUNCTIONAL RESPONSE CLASSES: IMPLICATIONS FOR TREATMENT OF SEVERE PROBLEM BEHAVIOR

Two studies examined the effects of a reductive treatment versus instruction-based treatments on the generalized reduction of problem behaviors. Each study involved a detailed analysis of multiple problem behaviors performed by school-aged youth with severe intellectual disabilities. The analysis examined the contrasting effects of one of two different positive intervention procedures (teaching a positive alternative behavior or providing additional teacher assistance during instruction) versus blocking and/or verbally reprimanding a problem behavior. The focus of each analysis was on the covariation of multiple problem behaviors within functional response classes. Results of the investigation indicated that when only one member of the response class was blocked, a collateral increase was observed in one or more different problem behaviors from the same response class. Alternatively, when 1 participant was taught a functionally equivalent mand response, all problem behaviors in the response class were reduced. Problem behaviors also were reduced for the remaining participant by presenting antecedent teacher assistance. Implications of the research extend to analysis of covariation within response classes and to procedures that result in generalized reduction of problem behaviors within a response class.     

Examining differential item functioning due to item difficulty and alternative attractiveness

A method for analyzing test item responses is proposed to examine differential item functioning (DIF) in multiple-choice items through a combination of the usual notion of DIF, for correct/incorrect responses and information about DIF contained in each of the alternatives. The proposed method uses incomplete latent class models to examine whether DIF is caused by the attractiveness of the alternatives, difficulty of the item, or both. DIF with respect to either known or unknown subgroups can be tested by a likelihood ratio test that is asymptotically distributed as a chi-square random variable.     

Reflections upon charitable giving: A comparison of individuals from business, ‘manual’ and professional backgrounds

This paper compares three groups of individuals in their beliefs about charitable giving. Focusing upon their preferences for different kinds of needy recipients, the paper compares the views of 49 people from (a) business, (b) professional and (c) ‘manual’ backgrounds. Their justifications of need provided the basis for comparisons on the basic themes of emotional response to requests for funds, to local versus overseas aid and to the relative roles of charity and state provision. It is argued that distinctions between the groups show the need to examine specific beliefs as well as generally held ideologies about charity. In addition, a further comparison of the groups is used to argue that they exemplify differences in the actual form of the gift relationship. These differences in form show the importance of examining charitable giving within specific social contexts, as well as with reference to the various ways of raising funds.     

Criterion-related construct validity

Established results on latent variable models are applied to the study of the validity of a psychological test. When the test predicts a criterion by measuring a unidimensional latent construct, not only must the total score predict the criterion, but the joint distribution of criterion scores and item responses must exhibit a certain pattern. The presence of this population pattern may be tested with sample data using the stratified Wilcoxon rank sum test. Often, criterion information is available only for selected examinees, for instance, those who are admitted or hired. Three cases are discussed: (i) selection at random, (ii) selection based on the current test, and (iii) selection based on other measures of the latent construct. Discriminant validity is also discussed.This work was supported in part by Grant SES-87-01890 from the Measurement Methods and Data Improvement Program of the U.S. National Science Foundation.     

A structural model for postpartum responses to the somatic and cognitive items on the Beck Depression Inventory

A distinction can be drawn between those items on the Beck Depression Inventory (BDI) that reflect mainly cognitive and emotional symptoms and those that reflect somatic symptoms. Responses to the BDI were obtained from women at 3, 6, 9, and 24 weeks postpartum. The covariances of cognitive and somatic symptom scores across the four time points were closely fit by a structural equations model referred to as the simplex. The modeling indicates that earlier cognitive scores directly predict later somatic scores but that early somatic scores do not directly predict later cognitive scores. Other data confirm the results.Much of this research was supported while the first author was an NIMH Fellow at the University of Iowa (MH15168).     

A nonparametric approach for assessing latent trait unidimensionality

Assuming a nonparametric family of item response theory models, a theory-based procedure for testing the hypothesis of unidimensionality of the latent space is proposed. The asymptotic distribution of the test statistic is derived assuming unidimensionality, thereby establishing an asymptotically valid statistical test of the unidimensionality of the latent trait. Based upon a new notion of dimensionality, the test is shown to have asymptotic power 1. A 6300 trial Monte Carlo study using published item parameter estimates of widely used standardized tests indicates conservative adherence to the nominal level of significance and statistical power averaging 81 out of 100 rejections for examinee sample sizes and psychological test lengths often incurred in practice.The referees' comments were remarkably detailed and greatly enhanced the writeup and sensitized the author to certain pertinent issues. Discussions with Fritz Drasgow, Lloyd Humphreys, Dennis Jennings, Brian Junker, Robert Linn, Ratna Nandakumar, and Robin Shealy were also very useful.This research was supported by the Office of Naval Research under grant N00014-84-K-0186; NR 150-533, and by the National Science Foundation under grant DMS 85-03321.     

Units of analysis and kinetic structure of behavioral repertoires

It is suggested that molar streams of behavior are constructed of various arrangements of three elementary constituents (elicited, evoked, and emitted response classes). An eight-cell taxonomy is elaborated as a framework for analyzing and synthesizing complex behavioral repertoires based on these functional units. It is proposed that the local force binding functional units into a smoothly articulated kinetic sequence arises from temporally arranged relative response probability relationships. Behavioral integration is thought to reflect the joint influence of the organism's hierarchy of relative response probabilities, fluctuating biological states, and the arrangement of environmental and behavioral events in time.     

Latent trait models and dichotomization of graded responses

This paper discusses thecompatibility of the polychotomous Rasch model with dichotomization of the response continuum. It is argued that in the case of graded responses, the response categories presented to the subject are essentially an arbitrary polychotomization of the response continuum, ranging for example from total rejection or disagreement to total acceptance or agreement of an item or statement. Because of this arbitrariness, the measurement outcome should be independent of the specific polychotomization applied, for example, presenting a specific multicategory response format should not affect the measurement outcome. When such is the case, the original polychotomous model is called compatible with dichotomization.A distinction is made between polychotomization or dichotomization before the fact, that is, in constructing the response format, and polycho- or dichotomization after the fact, for example in dichotomizing existing graded response data.It is shown that, at least in case of dichotomization after-the-fact, the polychotomous Rasch model is not compatible with dichotomization, unless a rather special condition of the model parameters is met. Insofar as it may be argued that dichotomization before the fact is not essentially different from dichotomization after the fact, the value of the unidimensional polychotomous Rasch model is consequently questionable. The impact of our conclusion on related models is also discussed.     

Critical action among Asian and Hispanic/Latinx youth: Identifying a multidimensional measure and exploring within-group differences

Critical action—behaviours aimed at dismantling systems of oppression—must be examined within youths' racialized experiences and should incorporate cultural and sociohistorical factors. We considered an expansive list of items capturing youth behaviours to create a novel four-factor (service, community change, expression, and care) measure of critical action for Asian and Hispanic/Latinx youth. Multiple distinct profiles of critical action were identified within both racial-ethnic groups, and associations between the profiles and sociodemographic and contextual support variables were explored. Gender differences in the type of critical action were found in both racial-ethnic groups, pointing to the potential influence of gender roles on critical action among these populations. Differences in critical action patterns were also found between those born in the U.S. versus those born outside the U.S.; access to critical action may differ within racial-ethnic groups depending on birthplace and associated nuances in familial and cultural contexts. This paper demonstrated a need for attending to variation between and within groups in the study of critical action in order to effectively support racialized youth's coping within and resistance against systems of oppression.