基于GT和多面Rasch模型的结构化面试分析

该研究应用GT和多面Rasch模型对结构化面试数据进行分析,并提出一些建议针对某辅导员招聘面试数据,运用GT从宏观上分析应聘者、考官和项目所带来的总体误差大小,在此基础上,运用多面Rasch模型从微观上进一步探查考官严厉度、应聘者能力差异、项目难易度及侧面偏差．结果表明：1）GT分析表明应聘者产生的变异较大（90．65％）,说明面试可靠性较高,且当考官数为2时可靠性已较好.2）多面Rasch模型分析出了各侧面效应中的非拟合因素及交互效应中的偏差因素,表明面试误差主要来自考官间严厉度的差异及其自身一致性的不稳定。将GT与多面Rasch模型相结合分析面试数据不仅能测查出评价过程各方面的问题因素,并能更好地作整体把握。     

EVALUATING PROGRESS IN BEHAVIORAL PROGRAMS FOR CHILDREN WITH AUTISM SPECTRUM DISORDERS VIA CONTINUOUS AND DISCONTINUOUS MEASUREMENT

We evaluated the influence of two different frequencies of data collection on skill acquisition and maintenance within behavioral treatment programs for children with autism spectrum disorders. Six children were taught multiple skills in up to four different behavioral programs. Half of the skills were measured continuously (i.e., trial by trial), and the other half were measured discontinuously (i.e., first trial only). When differences were detected, quicker acquisition was typically associated with discontinuous measurement, and stronger maintenance was typically associated with continuous measurement.     

ASSESSING OBSERVER ACCURACY IN CONTINUOUS RECORDING OF RATE AND DURATION: THREE ALGORITHMS COMPARED

The three algorithms most frequently selected by behavior‐analytic researchers to compute interobserver agreement with continuous recording were used to assess the accuracy of data recorded from video samples on handheld computers by 12 observers. Rate and duration of responding were recorded for three samples each. Data files were compared with criterion records to determine observer accuracy. Block‐by‐block and exact agreement algorithms were susceptible to inflated agreement and accuracy estimates at lower rates and durations. The exact agreement method appeared to be overly stringent for recording responding at higher rates (23.5 responses per minute) and for higher relative duration (72% of session). Time‐window analysis appeared to inflate accuracy assessment at relatively high but not at low response rate and duration (4.8 responses per minute and 8% of session, respectively).     

Incremental and predictive utility of formative assessment methods of reading comprehension

Formative assessment measures are commonly used in schools to assess reading and to design instruction accordingly. The purpose of this research was to investigate the incremental and concurrent validity of formative assessment measures of reading comprehension. It was hypothesized that formative measures of reading comprehension would contribute more to our understanding of students' overall reading abilities than simply oral reading fluency (ORF). It was also hypothesized that measures could be modeled in a meaningful way to explain student performance on criterion measures of academic competence. Four formative measures of reading comprehension – maze (MZ), retell fluency (RTF), written retell (WRT), and sentence verification technique (SVT) – were used to measure unique aspects of reading comprehension through production-type responses. Results suggested that reading comprehension measures, when combined with ORF, added to the total variance associated with reading ability and were reliable indicators of student performance on a high stakes criterion-referenced assessment of grade-level literacy.     

Curriculum-based measurement of oral reading: Passage equivalence and probe-set development

Curriculum-based measurement of reading (CBM-R) is used to estimate oral reading fluency. Unlike many traditional published tests, CBM-R materials are often comprised of 20 to 30 alternate forms/passages. Historically, CBM-R assessment materials were sampled from curricular materials. Recent research has documented the potentially deleterious effects of poorly controlled alternate forms on CBM-R outcomes. The purpose of this study was to examine alternate procedures for the selection of passages that comprise CBM-R passage-sets. The study examined four procedures for the evaluation and selection of passages, including random sampling, Spache readability formula, mean level of performance evaluation, and Euclidean Distance evaluation. The latter two procedures relied on field testing and evaluation of student performance. Each of eighty-eight students in second- and third-grade were administered 50 CBM-R passages. Generalizability and dependability studies were used to examine students' performance on these passages and evaluate CBM-R passage selection procedures. Results provide support for the use of field testing methods (i.e., calculating performance means and Euclidean Distances) for passage selection. Implications are discussed for future research and practice.     

Reliability of the Child and Adolescent Needs and Strengths-Mental Health (CANS-MH) Scale

We examined the interrater reliability of the Child and Adolescent Needs and Strengths-Mental Health (CANS-MH) scale among researchers and between researchers and clinicians. All children presenting to a treatment facility for either protective or mental health needs were eligible to be included in the study. As part of standard assessment procedure, all children entering the facility were described by caseworkers using the CANS-MH. A random sample of 60 cases was selected and researchers made retrospective assessments using the CANS-MH based on chart review. Results of the interrater reliability support previous findings that the CANS-MH is a reliable measure of clinical and psychosocial needs and strengths. Results also suggest that ratings on the CANS-MH based on medical record abstraction by researchers relate to ratings performed by clinical staff, demonstrating the clinical and research utility of the CANS-MH. The CANS-MH can be used reliably to assess the type and severity of problem presentation, risk behaviors, functioning, care intensity and organization, caregiver capacity and strengths among children with protective and mental health needs. In addition, items of the CANS-MH appear to be reliable supporting the use of these items individually. Findings are relevant to the ongoing delivery of clinical services, monitoring of quality assurance, and outcomes.     

The reliability and construct validity of scores on the Japanese version of an academic self‐concept scale for a sample of college students1

Abstract: This study investigated the internal consistency, reliability, and construct validity of scores on the Japanese version of the Dimensions of Self‐Concept (DOSC) academic self‐concept scale for a sample of 177 students at a small Japanese private college. Confirmatory factor analyses showed promising support for the construct validity of scores on the six factor subscales (Level of Aspiration, Anxiety, Academic Interest and Satisfaction, Leadership and Initiative, Identification versus Alienation, and Stress), representing the six hypothesized constructs of academic self‐concept. Internal consistency reliability of scores on the six subscales ranged from 0.82 to 0.87.     

Intentional Faith: An Alternative View of Faith Development

Faith was defined as the search for an integrating center of value and meaning that is cognitional in nature, developmental in process, and transcendental in its dimensions. An original psychometric measure was constructed and tested in two samples in a midwestern metropolitan area. The life span sample of Catholics (N = 509) ranged in ages 18–84, median age 49. Respondents were 40% male, 60% female. The sample of college students (N = 303) had a mode age of 19, but was ethnically and religiously diverse. The results indicated that the covariance structure of the scale was equivalent for the two samples. The scale clearly measures 4 independent aspects of faith: Self and Moral development, God and Death, Ritual, and Authority. Reliability was established for four developmental Ways of Faith, based on Bernard Lonergan's intentionality analysis: Common Sense Faith, Thoughtful Faith, Responsible Faith, and Transcendent Faith. The data suggested that faith development may be best conceived as a continuous reworking of one's faith, rather than a sequence of acquisition and abandonment of beliefs. A mixed pattern of modest gender, age, and attachment effects was found for the Ways of Faith. A measure of self-church identification was found to be a consistent correlate of faith development.     

What sources contribute to variance in observer ratings? Using generalizability theory to assess construct validity of psychological measures

Cronbach and Meehl (Psychol. Bull. 1955; 52:281–302) stated that the key question to be addressed when assessing construct validity is ‘What sources contribute to variance in test performance?’ We illustrate the utility of generalizability theory (GT) as a conceptual framework that encourages psychological researchers to address this question and as a flexible set of analytic tools that can provide answers to inform both substantive theory and measurement practice. To illustrate these capabilities, we analyze observer ratings of 27 caregiver–child dyads, focusing on the importance of situational (contextual) factors as sources of variance in observer ratings of caregiver–child behaviors. Cross‐situational consistency was relatively low for the categories of behavior analyzed, indicating that dyads vary greatly in their interactional patterns from one situation to the next, so that it is difficult to predict behavioral frequencies in one context from behaviors observed in a different context. Our findings suggest that single‐situation behavioral measures may have limited generalizability, either to behavior in other contexts or as measures of global interaction tendencies. We discuss the implications of these findings for research and measurement design in developmental psychology. Copyright © 2008 John Wiley & Sons, Ltd.     

ON EXPLICATING THE CONCEPT THE POWER OF AN ARITHMETICAL THEORY

In this paper I discuss possible ways of measuring the power of arithmetical theories, and the possiblity of making an explication in Carnap’s sense of this concept. Chaitin formulates several suggestions how to construct measures, and these suggestions are reviewed together with some new and old critical arguments. I also briefly review a measure I have designed together with some shortcomings of this measure. The conclusion of the paper is that it is not possible to formulate an explication of the concept.