Examiner error in curriculum-based measurement of oral reading

Although curriculum based measures of oral reading (CBM-R) have strong technical adequacy, there is still a reason to believe that student performance may be influenced by factors of the testing situation, such as errors examiners make in administering and scoring the test. This study examined the construct-irrelevant variance introduced by examiners using a cross-classified multilevel model. We sought to determine the extent of variance in student CBM-R scores attributable to examiners and, if present, the extent to which it was moderated by students' grade level and English learner (EL) status. Fit indices indicated that a cross-classified random effects model (CCREM) best fits the data with measures nested within students, students nested within schools, and examiners crossing schools. Intraclass correlations of the CCREM revealed that roughly 16% of the variance in student CBM-R scores was associated between examiners. The remaining variance was associated with the measurement level, 3.59%; between students, 75.23%; and between schools, 5.21%. Results were moderated by grade level but not by EL status. The discussion addresses the implications of this error for low-stakes and high-stakes decisions about students, teacher evaluation systems, and hypothesis testing in reading intervention research.     

The Dark Triad,happiness and subjective well-being

Narcissism can be expressed in grandiose or vulnerable forms. We examined whether positive psychological states (defined by the Oxford Happiness Inventory (OHI) and the Diener Satisfaction With Life (SWL) scales) assisted differentiation relative to general personality traits and the “the Dark Triad” (psychopathy, narcissism, and Machiavellianism, measured by the D12 and Short Dark Triad (SD3) indices) for 840 persons primarily from the UK, USA and Canada. The best fitting structural equation model comprised two latent variables, one of positive mood (comprising total scores on the OHI and SWL scales), and another forming a “dark dyad” of Machiavellianism and psychopathy (predicted by low agreeableness and lower positive mood), with narcissism regarded as a separate construct correlated with the dark dyad. Latent positive mood was primarily predicted by higher emotional stability and extraversion. Narcissism was predicted by lower emotional stability, lower agreeableness, and higher extraversion. Latent profile analysis identified four groups in the data: “unhappy but not narcissistic”, “vulnerable narcissism”, “happy non-narcissism” and “grandiose narcissism”. Our results suggest more problematic narcissism can be identified by reference to measures indexing positive mood states and general personality traits.     

Exploring Differential Subgroup Functioning on SAT Writing Items: What Happens When English Is Not a Test Taker's Best Language?

The purpose of this study is to explore patterns in model-data fit related to subgroups of test takers from a large-scale writing assessment. Using data from the SAT, a calibration group was randomly selected to represent test takers who reported that English was their best language from the total population of test takers (N = 322,011). A reference scale for the items was constructed based on EBL responses. Response behaviors of test takers who reported that English was not their best language (ENBL) were examined in relationship to this reference scale. This study illustrates the use of differential subgroup analyses to identify patterns related to person misfit within subgroups, as well as subsets of items, that may affect the validity of writing scores for ENBL test takers. The methodology described here offers an approach that can be used to explore, understand, and improve the validity of scores obtained from ENBL test takers in large-scale writing assessments.     

Validation and Standardization of the Dutch Trauma Symptom Checklist for Young Children in a Normative and Clinical Sample

The study aimed to describe the psychometric properties of the Dutch version of the Trauma Symptom Checklist for Young Children (TSCYC) in normative and clinical populations in the Netherlands. Caregivers’ ratings on the TSCYC were obtained for 1,802 children from the normal population, and for 515 children from a clinical population of traumatized children. In the clinical sample, additional measures were taken. The internal consistency and test-retest reliability of TSCYC scales were adequate. Confirmatory factor analysis showed acceptable fit on the putative scale structure. Regarding criterion validity, the clinical sample scored significantly higher on all clinical scales when compared to the normal population sample. Within the clinical sample, significant associations were found between TSCYC scales and convergent scales of other instruments. The Posttraumatic Stress-Total subscale demonstrated excellent discriminative ability between traumatized children and children from the normal population. The Dutch version of the TSCYC proved a valid and reliable instrument to measure trauma symptoms in young children through caregiver report, similar to the original American version. Further comparisons with diagnostic interviews are warranted.     

Investigation of drivers’ thresholds of a subjectively accepted driving performance with a focus on automated driving

Lately, the development and implementation of automated driving moved to the center of interest in the automotive industry. In this context, one of the central issues – the configuration of adequate trajectories – is mainly tackled using a technical approach. However, it appears that a technically ideal driving performance does not necessarily coincide with the drivers’ subjective preferences. This study strives to determine thresholds of a subjectively accepted driving performance regarding lateral vehicle control. A second objective is to analyze the influence of selected personal and situational factors on these thresholds. An empirical online survey with 161 participants rating video sequences of driving performances was conducted. The video sequences differed not only with regard to the lateral offset of the ego-vehicle but also concerning the weather (sun/rain) and traffic conditions (existence/driving behavior of oncoming traffic). Additionally, the participants’ driving experience and sensation seeking were considered in the data evaluation. To analyze the data, binary logistic regression analyses were calculated. They revealed that the subjective evaluation of driving performances varies primarily depending on the lateral offset of both the ego-vehicle and the oncoming traffic. The results indicate that regarding the lateral offset certain thresholds of subjectively accepted driving performances do exist. Regarding the development of automated driving systems, two issues need to be considered in order to ultimately guarantee user acceptance. First, the subjective thresholds need to be integrated into the systems’ trajectory planning. Second, the oncoming traffic’s driving behavior has to be considered.     

Effects of Share the Keys (STK), a comprehensive,behavioral-based training program for parents and new teen drivers in New Jersey

The New Jersey Graduated Driver License (GDL) restrictions were updated in 2010 with changes that included a more active role of parents monitoring practice driving with their teens. In addition to extending the driving phases, there were new curfew and passenger limit restrictions. Teen drivers were also required to use red decals on license plates to identify themselves as inexperienced drivers. The Division of Highway Traffic Safety had partnered with the Motor Vehicle Commission in the past to produce practice driving guides for parents of teen drivers, but recognized the need to support this type of informational campaign with an educational program for helping parents to understand changes in the GDL and recognize the importance of their role in the GDL monitoring process. In 2010, the New Jersey Parent/Teen Driver Orientation program was developed to educate parents of teen drivers on the new GDL restrictions and Children’s Hospital of Philadelphia (CHOP) findings concerning the importance of parenting roles in teen driver safety. Additional resources were incorporated into the orientation that included the Centers for Disease Control (CDC) community based information and the University of Michigan Transportation Research Institute (UMTRI) parent-teen driver contracts.This community-based training, known as the New Jersey Parent/Teen Driver Orientation was later renamed Share the Keys, and had been developed for the state as a component of the New Jersey Highway Safety grant received by Kean University. The comprehensive program was designed to engage parents, teens, educators, and law enforcement in the teen driving process and help parents to understand how personal parenting styles and engagement may impact teen crash risk and compliance with the GDL restrictions. Local representatives from traffic safety and law enforcement teamed with school officials and driver education teachers to present programs in their communities. The evaluation process was built into the program as a pre-survey administered at the beginning of the session, a post-survey completed directly afterward, and a voluntary follow-up survey completed on line or mailed, beginning six months later.Evidence from the first year showed that parents maintained a significant level of engagement throughout the GDL process with their teen drivers. However, findings did establish the need to further examine why Authoritative parenting styles had declined, especially since this parenting approach was considered in the literature as most effective in reducing teen crash risks. Therefore, the scope of this study was broadened to a three year effort that reached over 2800 parents and investigated behavioral outcomes impacted by Share the Keys. While comparisons made between driving phases and parenting styles had yielded no significant results, changes within parenting style scores were shown to be effective over time. The reported levels of Permissive and Uninvolved parents remained consistent throughout all three driving phases, while Authoritarian parenting peaked during the Probationary phase, and Authoritative parenting was noted as highest once teens earned their license.     

An Evaluation of the Word Triad Method for Monitoring Spelling Progress

Although spelling skill progress has typically been studied within the context of students' responses to written story starters (Deno, Marsten, & Mirkin, 1982; Fuchs & Fuchs, 2011; Hosp, Hosp, & Howell, 2007; Shinn & Shinn, 2002), there has been little research conducted within a curriculum-based measurement framework that has studied spelling progress monitoring on a weekly basis throughout an entire school year. The authors evaluated spelling progress using word triads, defined as a group of 3 words (with the same number of letters) that have a similar phonetic or morphological structure. Successful spelling of word triads as the unit of analysis has the potential to be more useful diagnostically to teachers than successful spelling of single words as the unit of analysis. An alternating series of three 15-word spelling tests were administered weekly to 10 third- and fourth-grade classrooms in the same school district throughout the school year for 8 rounds (1 round = 3 weeks of different but phonetically and/or morphologically similar word lists). Results indicate steady progress throughout the school year in the correct spelling of word triads (despite teacher reporting of no explicit spelling instruction in classrooms using in the words employed in the study). Correlations between the number of correctly spelled word triads with standard scores from a group-administered communication arts test were significant and comparable to alternate spelling test scoring methods. The authors conclude with limitations of the study and implications for practitioners.     

The Multigroup Ethnic Identity Measure – Revised (MEIM-R): Psychometric evaluation with adolescents from diverse ethnocultural groups in Italy

The Multigroup Ethnic Identity Measure – Revised (MEIM-R) is an extensively used questionnaire assessing ethnic identity. However, studies on its measurement characteristics in the European context are lacking. The current study addressed this gap by investigating the MEIM-R psychometric proprieties across multiple ethnocultural groups in Italy. Participants were 1445 adolescents (13–18 years) of Italian, East European, and North African origin. Results showed that the MEIM-R has good internal consistency. Multigroup confirmatory factor analyses revealed configural and metric invariance, i.e., an equal, correlated two-factor structure (ethnic identity exploration and commitment) and equal factor loadings across groups. Scalar invariance, i.e., equal item intercepts, was found only for the commitment scores that showed no group differences in latent factor mean levels. Partial structural invariance was evidenced, with the factor covariances varying across groups. These findings suggest that the MEIM-R is a valuable tool to assess the correlates of ethnic identity, although further research is needed.     

Children's Sensitivity to Cost and Reward in Decision Making across Distinct Domains of Probability,Effort, and Delay

Many behavioral paradigms used to study individuals' decision‐making tendencies do not capture the decision components that contribute to behavioral outcomes, such as differentiating decisions driven toward a reward from decisions driven away from a cost. This study tested a novel decision‐making task in a sample of 403 children (age 9 years) enrolled in an ongoing longitudinal study. The task consisted of three blocks representing distinct cost domains (delay, probability, and effort), wherein children were presented with a deck of cards, each of which consisted of a reward and a cost. Children elected whether to accept or skip the card at each trial. Reward–cost pairs were selected by using an adaptive algorithm to strategically sample the decision space in the fewest number of trials. Using person‐specific regression models, decision preferences were quantified for each cost domain with respect to general tolerance (intercept), as well as parameters estimating the effect of incremental increases in reward or cost on the probability of accepting a card. Results support the relative independence of decision‐making tendencies across cost domains, with moderate correlations observed between tolerance for delay and effort. Specific decision parameters showed unique associations with cognitive and behavioral measures including executive function, academic motivation, anxiety, and hyperactivity. Evidence indicates that sensitivity to reward is an important factor in incentivizing decisions to work harder or wait longer. Dissociating the relative contributions of reward and cost sensitivity in multiple domains may facilitate the identification of heterogeneity in suboptimal decision making. Copyright © 2017 John Wiley & Sons, Ltd.