Sample size determinations for the two rater kappa statistic

This paper gives a method for determining a sample size that will achieve a prespecified bound on confidence interval width for the interrater agreement measure,. The same results can be used when a prespecified power is desired for testing hypotheses about the value of kappa. An example from the literature is used to illustrate the methods proposed here.     

个人化设计在临床研究中的应用——从个体角度研究个体差异

本文从有关研究证据的基础上,介绍了个人化设计的理论基础和实验设计,并在此基础上讨论了个人化设计在临床研究领域中的应用价值,总结出一种适合临床病人使用的研究程序。个人化设计允许对个体的个人价值进行分析,并能同时研究不同刺激信息的作用,在个体水平上分析个体差异。通过简化了的实验程序,个人化设计为解决临床研究中的问题提供了一种新的思维方式。文中还对相关研究法的局限进行了讨论。     

Latent trait models and dichotomization of graded responses

This paper discusses thecompatibility of the polychotomous Rasch model with dichotomization of the response continuum. It is argued that in the case of graded responses, the response categories presented to the subject are essentially an arbitrary polychotomization of the response continuum, ranging for example from total rejection or disagreement to total acceptance or agreement of an item or statement. Because of this arbitrariness, the measurement outcome should be independent of the specific polychotomization applied, for example, presenting a specific multicategory response format should not affect the measurement outcome. When such is the case, the original polychotomous model is called compatible with dichotomization.A distinction is made between polychotomization or dichotomization before the fact, that is, in constructing the response format, and polycho- or dichotomization after the fact, for example in dichotomizing existing graded response data.It is shown that, at least in case of dichotomization after-the-fact, the polychotomous Rasch model is not compatible with dichotomization, unless a rather special condition of the model parameters is met. Insofar as it may be argued that dichotomization before the fact is not essentially different from dichotomization after the fact, the value of the unidimensional polychotomous Rasch model is consequently questionable. The impact of our conclusion on related models is also discussed.     

Interobserver agreement and procedural fidelity: An odd asymmetry

We examined articles with experiments published in the Journal of Applied Behavior Analysis and in Behavior Analysis in Practice from 2017 through 2021 to determine how frequently procedural fidelity was assessed. When procedural fidelity was assessed, we determined how often a measure of interobserver agreement for those fidelity data was provided. We also determined how often a measure of interobserver agreement for participants' behavior was provided. Across both journals and all years, 54.7% of relevant articles provided a measure of procedural fidelity. Of them, 17.7% provided a measure of interobserver agreement for procedural fidelity. In marked contrast, 96.4% provided interobserver agreement data for participants' behavior. It is unfortunate that applied behavior analysts frequently fail to provide procedural fidelity data and, when they do, often fail to provide interobserver agreement data for the fidelity data. Reviewers for, and editors of, behavior-analytic journals are encouraged to strongly consider the relative value of procedural fidelity and agreement on procedural fidelity measures when rendering recommendations on the suitability of a given submission.     

A generalized family of coefficients of relational agreement for numerical scales

A family of coefficients of relational agreement for numerical scales is proposed. The theory is a generalization to multiple judges of the Zegers and ten Berge theory of association coefficients for two variables and is based on the premise that the choice of a coefficient depends on the scale type of the variables, defined by the class of admissible transformations. Coefficients of relational agreement that denote agreement with respect to empirically meaningful relationships are derived for absolute, ratio, interval, and additive scales. The proposed theory is compared to intraclass correlation, and it is shown that the coefficient of additivity is identical to one measure of intraclass correlation.The author thanks the Editor and anonymous reviewers for helpful suggestions.     

The Development of the Toddler Behavior Screening Inventory

The purpose of the present study was to develop the Toddler Behavior Screening Inventory (TBSI) for rapidly assessing maternal reports of toddler behavior problems. The TBSI is a 40-item measure intended for use by pediatricians, psychologists, and other child-care professionals. A group of mothers (n = 312) rated the initial 93-item TBSI in terms of behavior frequency and parental perception of the behavior as problematic. Following item elimination, 40 items were retained due to being both frequent and problematic at all toddler age levels. Data were obtained with a new sample of mothers (n = 581) and showed the TBSI to possess good reliability and validity.     

Human d-amphetamine drug discrimination: methamphetamine and hydromorphone.

Standard measures of subjective and discriminative effects of drugs were compared in 5 human volunteers. Subjects responded on a second-order color-tracking procedure, where 30 mg of d-amphetamine served as a discriminative stimulus for one response and its absence as the discriminative stimulus for another response. Self-reported subjective effects were assessed concurrently using the single-dose questionnaire, subscales of the Addiction Research Center Inventory, and several analogue rating scales. On different days following discrimination acquisition, varying doses of d-amphetamine, methamphetamine, and hydromorphone were administered. In these test sessions, either response was reinforced. Methamphetamine and d-amphetamine occasioned dose-related increases in d-amphetamine appropriate responding; hydromorphone did not. Methamphetamine and d-amphetamine occasioned dose-related increases in reports of the drug received being most like "speed"; hydromorphone occasioned dose-related increases in reports of the drug received being most like "dope." All three drugs occasioned dose-related increases in reports of drug liking, and increases in the morphine-benzedrine group, amphetamine, and benzedrine group scales of the Addiction Research Center Inventory. This experiment demonstrated that although explicit discriminative control of behavior by a drug may covary with drug identification, it does not necessarily covary with other self-reported subjective effects. Thus, the complementary nature of the data provided by drug discrimination and standard subjective-effects measures provides quantitative and qualitative data useful in studying both relatively novel compounds and the behavioral biology of psychoactive drugs in general.     

The Revised Conners' Parent Rating Scale (CPRS-R): Factor Structure, Reliability, and Criterion Validity

The Conners' Parent Rating Scale (CPRS) is a popular research and clinical tool for obtaining parental reports of childhood behavior problems. The present study introduces a revised CPRS (CPRS-R) which has norms derived from a large, representative sample of North American children, uses confirmatory factor analysis to develop a definitive factor structure, and has an updated item content to reflect recent knowledge and developments concerning childhood behavior problems. Exploratory and confirmatory factor-analytic results revealed a seven-factor model including the following factors: Cognitive Problems, Oppositional, Hyperactivity-Impulsivity, Anxious-Shy, Perfectionism, Social Problems, and Psychosomatic. The psychometric properties of the revised scale appear adequate as demonstrated by good internal reliability coefficients, high test-retest reliability, and effective discriminatory power. Advantages of the CPRS-R include a corresponding factor structure with the Conners' Teacher Rating Scale—Revised and comprehensive symptom coverage for attention deficit hyperactivity disorder (ADHD) and related disorders. Factor congruence with the original CPRS as well as similarities with other parent rating scales are discussed.     

Teachers' and mothers' assessment of children's behaviors from kindergarten to grade two: Stability and change within and across informants

Consistency between teachers' and parents' ratings of 229 children's prosocial and problem behaviors was investigated from kindergarten through grade two. Intrarater stability was also analyzed. Results indicated moderate but decreasing interrater agreement from kindergarten to grade two for boys on externalizing behaviors. Interrater agreement for the same behavior dimension for girls was moderate and stable, whereas it was low on internalizing behavior problems and prosocial behaviors, for both genders throughout the 3-year period. Moreover, difference between teachers' and parents' ratings of externalizing behaviors increased from kindergarten to grade two. Decreasing level of agreement through time appeared to be related to a reduction of teacher-rated externalizing behaviors not paralleled by a reduction in parent ratings. Analyses were also performed at the item level of the externalization scale to identify which specific behaviors were responsible for the increasing divergence in interrater agreement from kindergarten through grade two. Alternative interpretations were considered in order to account for present and past results. Implications for screening and assessment purposes were discussed.This research was supported by grants from the Social Sciences and Humanities Research Council of Canada and from Québec's Health Research Fund (FRSQ). We wish to thank the authorities and directors of the schools from the Val d'Or School Board as well as the kindergarten, first and second grade teachers, children, and parents for their collaboration. We also thank Lyse Desmarais-Gervais, Daniel Corriveau, Jean Desrosiers and Hélène Boileau for their participation in the data collection or analysis.     

Assessment of incarcerated juvenile delinquents: Agreement across raters and approaches to psychopathology

The purpose of the current study was to examine the agreement across three informants (youth, teacher, caretaker) when rating behaviors of incarcerated juvenile delinquents. Furthermore, the agreement between a dimensional approach (Child Behavior Checklist) and a categorical approach (DSM-III-R) of assessing behavior was examined. Fifty-two delinquent youth were assessed with a structured interview. In addition, the youth, teacher, and caretaker completed the Child Behavior Checklist (CBCL). The results indicated significant correlations across the teacher and caretaker for externalizing types of problems. The youth report form of the CBCL did not correspond to either adult informant. However, the ratings by the teacher and caretaker were related to externalizing types of problems, primarily hyperactivity/inattention, derived from the structured interview with the delinquent. The results suggest that, in the psychological assessment of incarcerated juvenile delinquents, agreement across informants is dependent on multiple factors, including type of behavior assessed and the approach utilized in assessment.This work was supported, in part, by the Juvenile Justice Coordinating Council of Georgia, the Office of Juvenile Justice and Delinquency Prevention, and the University of Georgia's Institute for Behavioral Research.