Ordinal consistency and ordinal true scores

This paper argues that test data are ordinal, that latent trait scores are only determined ordinally, and that test data are used largely for ordinal purposes. Therefore it is desirable to develop a test theory based only on ordinal assumptions. A set of ordinal assumptions is presented, including an ordinal version of local independence. From these assumptions it is first shown that the gamma-correlation between two tests is the product of their gamma-correlations with the true latent order. The theory is generalized to allow for heterogeneous tests by defining a weighted average local independence. The tau-correlations between total score and the latent order can be found in both homogeneous and heterogeneous cases, and a system of differential item weighting to maximize the tau-correlation between weighted items and the latent order is provided. Thus a purely ordinal test theory seems possible.Part of this work was done while the author was a Visiting Fellow at Macquarrie University. The paper has benefitted from discussions with Professors Thomas J. Reynolds and Roderick P. McDonald and from the comments of several anonymous reviewers.     

Reliability of test scores in nonparametric item response theory

Three methods for estimating reliability are studied within the context of nonparametric item response theory. Two were proposed originally by Mokken (1971) and a third is developed in this paper. Using a Monte Carlo strategy, these three estimation methods are compared with four classical lower bounds to reliability. Finally, recommendations are given concerning the use of these estimation methods.The authors are grateful for constructive comments from the reviewers and from Charles Lewis.     

Reactivity and unreliability of husbands as participant observers

This study examined the reliability and reactivity of participant observation. Twelve couples engaged in two discussions in a laboratory, one on a conflict topic and one on a consensus topic. For the eight couples in the experimental sequences, a baseline discussion was followed by participant observation in which the husband recorded his wife's use of I while continuing to converse. The four couples in the control sequences conversed without participant observation. Compared to observations made by a trained observer, the husbands were unreliable observers, underestimating their wives' use of I. Participant observation was reactive (P<.01), with more wives decreasing their use of I in the experimental sequences than in the control sequences (p<.094). The topic of discussion did not differentially influence reliability or reactivity.This research was conducted as the first author's independent doctoral research project. We extend gratitude to the Research Council of the University of North Carolina at Greensboro for Grant 0-2-110-218-XXXXX-7548; the Statistical Consulting Center of the University of North Carolina at Greensboro; two anonymous reviewers; and our research assistants, Suzanne Brannon, Ben Hardie, Kathy Lindamood, Amy Mitchem, Regina Pierce, and Elga Wulfert. Portions of this paper were presented at the meeting of the Association for Advancement of Behavior Therapy, Washington, D.C., December 1983.     

Interobserver agreement and procedural fidelity: An odd asymmetry

We examined articles with experiments published in the Journal of Applied Behavior Analysis and in Behavior Analysis in Practice from 2017 through 2021 to determine how frequently procedural fidelity was assessed. When procedural fidelity was assessed, we determined how often a measure of interobserver agreement for those fidelity data was provided. We also determined how often a measure of interobserver agreement for participants' behavior was provided. Across both journals and all years, 54.7% of relevant articles provided a measure of procedural fidelity. Of them, 17.7% provided a measure of interobserver agreement for procedural fidelity. In marked contrast, 96.4% provided interobserver agreement data for participants' behavior. It is unfortunate that applied behavior analysts frequently fail to provide procedural fidelity data and, when they do, often fail to provide interobserver agreement data for the fidelity data. Reviewers for, and editors of, behavior-analytic journals are encouraged to strongly consider the relative value of procedural fidelity and agreement on procedural fidelity measures when rendering recommendations on the suitability of a given submission.     

Police,courts, and corrections: Experiences of procedural injustice among Black adults

Racial disparities and a corresponding lack of trust have been documented within the criminal legal system. In response, criminal legal system actors have sought to strengthen the legitimacy of their agencies. However, legitimizing these agencies can be problematic. Some argue that the current criminal legal system continues the legacy of slavery and Jim Crow as Blacks are disproportionately policed and incarcerated. As a framework, procedural injustice can offer a unique backdrop and interrogate ways in which the criminal legal system engages in delegitimizing actions that provoke noncompliance and enable social control. Using a procedural injustice lens, this study examines how justice-involved Black adults experience mistreatment by justice system actors. Semistructured interviews were conducted with 84 Black adults in Newark and Cleveland. Study findings offer a comprehensive account of how participants experience procedural injustice as arrestees, defendants, and incarcerated persons. More specifically, participant narratives describe deliberately antagonistic, abusive, and dehumanizing treatment by justice-system agents—often depicted as racially motivated. Participant accounts also describe this mistreatment as occurring in a context of coercion and powerlessness and as being institutionally sanctioned. Implications for the preservation of racial hierarchies, research, practice, and community psychology are discussed.     

Internal consistency and reliability of alternate short forms of the college women's assertion sample: Clinical and research implications

Assertive behavior is most often assessed with self-report or role-play measures. The latter modality is preferred because it provides for the sampling of the structure of behavior and for the consideration of the situational context. MacDonald (1978) has developed such an assessment device but it is limited by the length of time for administration and scoring. Two studies were conducted to reconstruct reliable alternate short forms. The first study describes the selection of items and demonstrates the internal consistency of the alternate forms. The second study demonstrates the alternate form and retest reliability and provides normative statistics. We conclude that reliable alternate short forms have been constructed to be used in research in clinical applications.This research was supported by the Marie Wilson Howells Fund.Alternate short forms of the CWAS may be obtained from the first author.     

Some relationships between the information function of IRT and the signal/noise ratio and reliability coefficient of classical test theory

It is shown that IRTs information function for an item is functionally related to local versions of classical test theories' signal/noise ratio and reliability coefficient.     

Swedish translations of the Suicide Probability Scale, Perceived Social Support from Friends and Family Scales, and the Scale for Interpersonal Behavior: A reliability analysis

The Suicide Probability Scale (SPS), the Perceived Social Support (PSS) from Friends (PSS-Fr) and Family (PSS-Fa) scales, and the Scale for Interpersonal Behavior (SIB) were translated into Swedish and their reliability was estimated in a university student sample. The reliability coefficients indicated that both subscales and the total scales of the SPS, PSS, and SIB possess highly adequate reliabilities. The intercorrelations among the subscales and between the subscales and the total scales were found to be highly significant. The results of the study supported the use of SPS, PSS-Fr and PSS-Fa, and the SIB as reliable methods for assessing suicide risk, perceived social support from friends and family, and assertive behavior.