OBSERVER TRAINING REVISITED: A COMPARISON OF IN VIVO AND VIDEO INSTRUCTION

We compared the effects of 2 observer‐training procedures. In vivo training involved practice during actual treatment sessions. Video training involved practice while watching progressively more complex simulations. Fifty‐nine undergraduate students entered 1 of the 2 training conditions sequentially according to an ABABAB design. Results showed that the 2 training methods produced almost identical scores on a posttraining observational test; however, the video method required fewer training sessions to complete.     

Getting precise and pragmatic about the assessment of bullying: the development of the California Bullying Victimization Scale

Accurate assessment of bullying is essential to intervention planning and evaluation. Limitations to many currently available self-report measures of bullying victimization include a lack of psychometric information, use of the emotionally laden term "bullying" in definition-first approaches to self-report surveys, and not assessing all components of the definition of bullying (chronicity, intentionality, and imbalance of power) in behavioral-based self-report methods. To address these limitations, we developed the California Bullying Victimization Scale (CBVS), which is a self-report scale that measures the three-part definition of bullying without the use of the term bully. We examined test-retest reliability and the concurrent and predictive validity of the CBVS across students in Grades 5-12 in four central California schools. Concurrent validity was assessed by comparing the CBVS with a common, definition-based bullying victimization measure. Predictive validity was examined through the co-administration of measures of psychological well-being. Analysis by grade and gender are included. Results support the test-retest reliability of the CBVS over a 2-week period. The CBVS was significantly, positively correlated with another bullying assessment and was related in expected directions to measures of well-being. Implications for differentiating peer victimization and bullying victimization via self-report measures are discussed.     

A critical review of relationship quality measures: Is a fresh start needed? An agenda to move forward

Social relationships in and around work are a fundamental building block of organizational life. According to a number of relationship theories, the quality of those relationships is a critical aspect of their influence on a variety of employee and organizational outcomes. Relationship quality is, therefore, one of the most commonly studied constructs in management research. Our paper critically reviews the landscape of relationship quality measures used in management studies, identifying critical issues with how the construct has been conceptualized and measured. To help advance research on the subject and improve measurement, we identify and evaluate 21 relationship quality measures that have been developed for, or used in, management research. Overall, we find that the instruments used to measure relationship quality in management research have significant limitations associated with their conceptualizations and operationalizations. Many of these limitations stem from the fact that most of the measures used to assess the construct were not originally designed to measure relationship quality, which impedes research clarity and implications. We offer a future research agenda and several recommendations for the advancement of management research on relationship quality.     

Examiner error in curriculum-based measurement of oral reading

Although curriculum based measures of oral reading (CBM-R) have strong technical adequacy, there is still a reason to believe that student performance may be influenced by factors of the testing situation, such as errors examiners make in administering and scoring the test. This study examined the construct-irrelevant variance introduced by examiners using a cross-classified multilevel model. We sought to determine the extent of variance in student CBM-R scores attributable to examiners and, if present, the extent to which it was moderated by students' grade level and English learner (EL) status. Fit indices indicated that a cross-classified random effects model (CCREM) best fits the data with measures nested within students, students nested within schools, and examiners crossing schools. Intraclass correlations of the CCREM revealed that roughly 16% of the variance in student CBM-R scores was associated between examiners. The remaining variance was associated with the measurement level, 3.59%; between students, 75.23%; and between schools, 5.21%. Results were moderated by grade level but not by EL status. The discussion addresses the implications of this error for low-stakes and high-stakes decisions about students, teacher evaluation systems, and hypothesis testing in reading intervention research.     

Exploring Differential Subgroup Functioning on SAT Writing Items: What Happens When English Is Not a Test Taker's Best Language?

The purpose of this study is to explore patterns in model-data fit related to subgroups of test takers from a large-scale writing assessment. Using data from the SAT, a calibration group was randomly selected to represent test takers who reported that English was their best language from the total population of test takers (N = 322,011). A reference scale for the items was constructed based on EBL responses. Response behaviors of test takers who reported that English was not their best language (ENBL) were examined in relationship to this reference scale. This study illustrates the use of differential subgroup analyses to identify patterns related to person misfit within subgroups, as well as subsets of items, that may affect the validity of writing scores for ENBL test takers. The methodology described here offers an approach that can be used to explore, understand, and improve the validity of scores obtained from ENBL test takers in large-scale writing assessments.     

Validation and Standardization of the Dutch Trauma Symptom Checklist for Young Children in a Normative and Clinical Sample

The study aimed to describe the psychometric properties of the Dutch version of the Trauma Symptom Checklist for Young Children (TSCYC) in normative and clinical populations in the Netherlands. Caregivers’ ratings on the TSCYC were obtained for 1,802 children from the normal population, and for 515 children from a clinical population of traumatized children. In the clinical sample, additional measures were taken. The internal consistency and test-retest reliability of TSCYC scales were adequate. Confirmatory factor analysis showed acceptable fit on the putative scale structure. Regarding criterion validity, the clinical sample scored significantly higher on all clinical scales when compared to the normal population sample. Within the clinical sample, significant associations were found between TSCYC scales and convergent scales of other instruments. The Posttraumatic Stress-Total subscale demonstrated excellent discriminative ability between traumatized children and children from the normal population. The Dutch version of the TSCYC proved a valid and reliable instrument to measure trauma symptoms in young children through caregiver report, similar to the original American version. Further comparisons with diagnostic interviews are warranted.     

An Evaluation of the Word Triad Method for Monitoring Spelling Progress

Although spelling skill progress has typically been studied within the context of students' responses to written story starters (Deno, Marsten, & Mirkin, 1982; Fuchs & Fuchs, 2011; Hosp, Hosp, & Howell, 2007; Shinn & Shinn, 2002), there has been little research conducted within a curriculum-based measurement framework that has studied spelling progress monitoring on a weekly basis throughout an entire school year. The authors evaluated spelling progress using word triads, defined as a group of 3 words (with the same number of letters) that have a similar phonetic or morphological structure. Successful spelling of word triads as the unit of analysis has the potential to be more useful diagnostically to teachers than successful spelling of single words as the unit of analysis. An alternating series of three 15-word spelling tests were administered weekly to 10 third- and fourth-grade classrooms in the same school district throughout the school year for 8 rounds (1 round = 3 weeks of different but phonetically and/or morphologically similar word lists). Results indicate steady progress throughout the school year in the correct spelling of word triads (despite teacher reporting of no explicit spelling instruction in classrooms using in the words employed in the study). Correlations between the number of correctly spelled word triads with standard scores from a group-administered communication arts test were significant and comparable to alternate spelling test scoring methods. The authors conclude with limitations of the study and implications for practitioners.     

The Multigroup Ethnic Identity Measure – Revised (MEIM-R): Psychometric evaluation with adolescents from diverse ethnocultural groups in Italy

The Multigroup Ethnic Identity Measure – Revised (MEIM-R) is an extensively used questionnaire assessing ethnic identity. However, studies on its measurement characteristics in the European context are lacking. The current study addressed this gap by investigating the MEIM-R psychometric proprieties across multiple ethnocultural groups in Italy. Participants were 1445 adolescents (13–18 years) of Italian, East European, and North African origin. Results showed that the MEIM-R has good internal consistency. Multigroup confirmatory factor analyses revealed configural and metric invariance, i.e., an equal, correlated two-factor structure (ethnic identity exploration and commitment) and equal factor loadings across groups. Scalar invariance, i.e., equal item intercepts, was found only for the commitment scores that showed no group differences in latent factor mean levels. Partial structural invariance was evidenced, with the factor covariances varying across groups. These findings suggest that the MEIM-R is a valuable tool to assess the correlates of ethnic identity, although further research is needed.