Multilevel Reliability Measures of Latent Scores Within an Item Response Theory Framework

Abstract

This paper evaluated multilevel reliability measures in two-level nested designs (e.g., students nested within teachers) within an item response theory framework. A simulation study was implemented to investigate the behavior of the multilevel reliability measures and the uncertainty associated with the measures in various multilevel designs regarding the number of clusters, cluster sizes, and intraclass correlations (ICCs), and in different test lengths, for two parameterizations of multilevel item response models with separate item discriminations or the same item discrimination over levels. Marginal maximum likelihood estimation (MMLE)-multiple imputation and Bayesian analysis were employed to evaluate the accuracy of the multilevel reliability measures and the empirical coverage rates of Monte Carlo (MC) confidence or credible intervals. Considering the accuracy of the multilevel reliability measures and the empirical coverage rate of the intervals, the results lead us to generally recommend MMLE-multiple imputation. In the model with separate item discriminations over levels, marginally acceptable accuracy of the multilevel reliability measures and empirical coverage rate of the MC confidence intervals were found in a limited condition, 200 clusters, 30 cluster size, .2 ICC, and 40 items, in MMLE-multiple imputation. In the model with the same item discrimination over levels, the accuracy of the multilevel reliability measures and the empirical coverage rate of the MC confidence intervals were acceptable in all multilevel designs we considered with 40 items under MMLE-multiple imputation. We discuss these findings and provide guidelines for reporting multilevel reliability measures.     

Cumulative estrogen exposure and prospective memory in older women

This study looked at cumulative lifetime estrogen exposure, as estimated with a mathematical index (Index of Cumulative Estrogen Exposure (ICEE)) that included variables (length of time on estrogen therapy, age at menarche and menopause, postmenopausal body mass index, time since menopause, nulliparity and duration of breastfeeding) known to influence estrogen levels across the life span, and performance on prospective and retrospective memory measures in a group of 50 postmenopausal women (mean age=69.3years) who, if they were current or former users of estrogen therapy, had started therapy within 5years of menopause. The ICEE was found to be a significant predictor of performance on the Prospective Memory task (F(1)=4.21, p=.046, η(p)(2)=.084). No significant relationship was noted between the ICEE and performance on measures of retrospective memory. The results suggest that the level of cumulative lifetime exposure to estrogen a woman has influences her prospective memory performance later in life and that the influence of reproductive and biological markers of endogenous estrogen exposure are relevant factors to consider when studying the effect of estrogen therapy on cognitive functioning in postmenopausal women. In addition, the finding that performance on a measure of prospective memory, but not performance on measures of retrospective memory, was associated with the ICEE adds further support to the theory that the frontal cortex may be especially sensitive to estrogen.     

The strength and generality of stimulus over-selectivity in simultaneous discrimination procedures

Stimulus over-selectivity refers to behavior being controlled by one element of the environment at the expense of other equally salient aspects of the environment. This is a common problem for many individuals, including those with autism spectrum disorders, and learning difficulties, and presents a considerable problem for information processing in many important situations involving complex cues and environments. Three experiments explored the strength and generality of the over-selectivity effect in non-clinical adults undertaking a cognitively demanding task, by training and testing participants on a two-component trial-and-error discrimination learning task. The over-selectivity effect was found in a variety of test conditions, including when the comparison test stimulus was neutral (Experiment 1), novel (Experiment 2), neutral with no conditioning history (Experiment 3), or when punished during training (Experiments 1, 2 and 3). Such results provide ubiquity to the phenomenon, making it important to investigate further.     

Assessing miserly information processing: An expansion of the Cognitive Reflection Test

The Cognitive Reflection Test (CRT; Frederick, 2005) is designed to measure the tendency to override a prepotent response alternative that is incorrect and to engage in further reflection that leads to the correct response. It is a prime measure of the miserly information processing posited by most dual process theories. The original three-item test may be becoming known to potential participants, however. We examined a four-item version that could serve as a substitute for the original. Our data show that it displays a .58 correlation with the original version and that it has very similar relationships with cognitive ability, various thinking dispositions, and with several other rational thinking tasks. Combining the two versions into a seven-item test resulted in a measure of miserly processing with substantial reliability (.72). The seven-item version was a strong independent predictor of performance on rational thinking tasks after the variance accounted for by cognitive ability and thinking dispositions had been partialled out.     

普通高校大学生综合国防素质的量表编制

本研究通过初始问卷编制、项目分析、探索性因素分析和验证性因素分析等程序,编制普通高校大学生综合国防素质量表,并进行了相关实证测量。结果显示,量表有8个维度共35个题项。量表的α系数和重测信度均符合心理测量学指标要求,具有良好的结构效度、聚敛区分效度与校标关联效度,可用于大学生综合国防素质的测量。测量结果发现,大学生综合国防素质总体处于中等偏上水平,且存在显著的性别差异与年级差异。     

Bien-être,satisfaction au travail et conflit travail–famille : une perspective centrée sur l’influence du leadership

This research investigates the relationship between well-being, work satisfaction, work–family conflict and leadership, based on hierarchical status. Regression analysis confirmed that leadership can predict a significant part of work satisfaction and well-being variation. The most significant results suggest that transformational leadership is related with higher satisfaction and laissez-faire leadership with lower satisfaction. Results showed no significant contribution of leadership on work–family conflict score variation. These results provide evidence that management styles should be considered to increase well-being and employees satisfaction.     

Evaluating the influence of approaching vehicles on pedestrian’s visual patterns and crossing behaviors at an uncontrolled crosswalk

The visual behaviors and movement characteristics of pedestrians are related to their surrounding potential safety hazards, such as approaching vehicles. This study primarily aimed to investigate the visual patterns and walking behaviors of pedestrians interacting with approaching vehicles. Field experiments were conducted at two uncontrolled crosswalks located at the Cuihua and Yanta roads in Xi’an, China. The visual performance of pedestrians was assessed using the eye tracking system from SensoMotoric Instruments (SMI). Moreover, motion trajectories of the pedestrians and approaching vehicles were obtained using an unmanned aerial vehicle. Subsequently, the visual attributes and movement trajectories of pedestrians and motion trajectories of approaching vehicles were statistically analyzed. The results showed that approaching vehicles distracted the fixation of crossing pedestrians significantly, and occupied 29.5% of the total duration of fixation; that is, pedestrians always directed more fixation points to the approaching vehicles compared to other stimuli. As a vehicle approached, pedestrians’ fixation shifted from other areas of interest to the vehicle. Moreover, an increase in the velocity of the vehicle and a closer distance between pedestrian and the vehicle resulted in an increase in the pedestrians’ duration of fixation on the approaching vehicle, and they implemented more saccades. Furthermore, approaching vehicle’s velocity and distance between pedestrian and approaching vehicle are not significantly associated with pedestrian’s movement attributes. These findings provide insights into the crossing behavior of pedestrians during pedestrian-vehicle interactions, which could assist future researchers and policy makers.     

The “Rod and Fran Test”: Relationship Priming Influences Cognitive-Perceptual Performance

We theorized that interpersonal relationships can provide structures for experience. In particular, we tested whether primes of same-sex versus mixed-sex relationships could foster cognitive-perceptual processing styles known to be associated with independence versus interdependence respectively. Seventy-two participants visualized either a same-sex or other-sex relationship partner and then performed two measures of cognitive-perceptual style. On a computerized Rod and Frame Test, individuals were more field-dependent after visualizing a mixed-sex versus same-sex relationship partner. On a measure involving perceptions of group behavior, participants demonstrated more holistic/contextually based perception after being primed with a female versus male relationship partner. These findings support the hypothesis that activated cognitive structures representing interpersonal relationships can shape individuals’ cognitive-perceptual performance.     

Maternally administered interventions for preterm infants in the NICU: Effects on maternal psychological distress and mother–infant relationship

Although studies have examined the effects of interventions focused on preterm infants, few studies have examined the effects on maternal distress (anxiety, depressive symptoms, post-traumatic stress symptoms, parenting stress) or parenting. This study examined the effects of the auditory–tactile–visual–vestibular (ATVV) intervention and kangaroo care (KC) on maternal distress and the mother–infant relationship compared to an attention control group.240 mothers from four hospitals were randomly assigned to the three groups. Maternal characteristics in the three groups were similar: 64.1% of ATVV mothers, 64.2% of KC mothers, and 76.5% of control mothers were African American; maternal age averaged 26.3 years for ATVV mothers, 28.1 for KC mothers, and 26.6 for control mothers; and years of education averaged 13.6 for ATVV and KC mothers, and 13.1 for control mothers. Mothers only differed on parity: 68.4% of ATVV and 54.7% of KC mothers were first-time mothers as compared to 43.6% of control mothers. Their infants had a similar mean gestational ages (27.0 weeks for ATVV, 27.2 for KC, and 27.4 for control) and mean birthweights (993 g for ATVV, 1022 for KC, and 1023 for control).Mothers completed questionnaires during hospitalization, and at 2, 6 and 12 months corrected age on demographic characteristics, depressive symptoms, state anxiety, post-traumatic stress symptoms, parenting stress, worry about child health, and child vulnerability (only at 12 months). At 2 and 6 months, 45-min videotapes of mother–infant interactions were made, and the HOME Inventory was scored. Behaviors coded from the videotapes and a HOME subscale were combined into five interactive dimensions: maternal positive involvement and developmental stimulation and child social behaviors, developmental maturity, and irritability.Intervention effects were examined using general linear mixed models controlling for parity and recruitment site. The groups did not differ on any maternal distress variable. Kangaroo care mothers showed a more rapid decline in worry than the other mothers. The only interactive dimensions that differed between the groups were child social behaviors and developmental maturity, which were both higher for kangaroo care infants. Change over time in several individual infant behaviors was affected by the interventions. When mothers reported on the interventions they performed, regardless of group assignment, massage (any form including ATVV) was associated with a more rapid decline in depressive symptoms and higher HOME scores. Performing either intervention was associated with lower parenting stress. These findings suggest that as short-term interventions, KC and ATVV have important effects on mothers and their preterm infants, especially in the first half of the first year.     

Age,the Big Five,and time-of-day preference: A mediational model

This research examined the extent to which the Big Five personality factors mediated the relationship between age and time-of-day preference. A sample of 491 Americans (M_age = 32 yrs) completed the 240-item NEO-PI-R, the 19-item Horne and Östberg’s (1976) Morningness–Eveningness Questionnaire (MEQ), and provided demographic information. As demonstrated in previous research, correlations revealed that older people expressed a stronger morning preference. More importantly, using bootstrapping procedures, it was found that the Big Five factor of conscientiousness attenuated the relationship between age and time-of-day preference. These findings indicate that conscientiousness plays a significant role in the relationship between age and time-of-day preference.