Modelling partially cross‐classified multilevel data

This article proposes an approach to modelling partially cross‐classified multilevel data where some of the level‐1 observations are nested in one random factor and some are cross‐classified by two random factors. Comparisons between a proposed approach to two other commonly used approaches which treat the partially cross‐classified data as either fully nested or fully cross‐classified are completed with a simulation study. Results show that the proposed approach demonstrates desirable performance in terms of parameter estimates and statistical inferences. Both the fully nested model and the fully cross‐classified model suffer from biased estimates of some variance components and statistical inferences of some fixed effects. Results also indicate that the proposed model is robust against cluster size imbalance.     

The Impact of Inappropriate Modeling of Cross-Classified Data Structures

Cross-classified random effects modeling (CCREM) is used to model multilevel data from nonhierarchical contexts. These models are widely discussed but infrequently used in social science research. Because little research exists assessing when it is necessary to use CCREM, 2 studies were conducted. A real data set with a cross-classified structure was analyzed by comparing parameter estimates when ignoring versus modeling the cross-classified data structure. A follow-up simulation study investigated potential factors affecting the need to use CCREM. Results indicated that when the structure is ignored, fixed-effect estimates were unaffected, but standard error estimates associated with the variables modeled incorrectly were biased. Estimates of the variance components also displayed bias, which was related to several study factors.     

Conditional mixed models with crossed random effects

The analysis of continuous hierarchical data such as repeated measures or data from meta‐analyses can be carried out by means of the linear mixed‐effects model. However, in some situations this model, in its standard form, does pose computational problems. For example, when dealing with crossed random‐effects models, the estimation of the variance components becomes a non‐trivial task if only one observation is available for each cross‐classified level. Pseudolikelihood ideas have been used in the context of binary data with standard generalized linear multilevel models. However, even in this case the problem of the estimation of the variance remains non‐trivial. In this paper, we first propose a method to fit a crossed random‐effects model with two levels and continuous outcomes, borrowing ideas from conditional linear mixed‐effects model theory. We also propose a crossed random‐effects model for binary data combining ideas of conditional logistic regression with pseudolikelihood estimation. We apply this method to a case study with data coming from the field of psychometrics and study a series of items (responses) crossed with participants. A simulation study assesses the operational characteristics of the method.     

Bias in longitudinal data analysis with missing data using typical linear mixed‐effects modelling and pattern‐mixture approach: An analytical illustration

We analytically derive the fixed‐effects estimates in unconditional linear growth curve models by typical linear mixed‐effects modelling (TLME) and by a pattern‐mixture (PM) approach with random‐slope‐dependent two‐missing‐pattern missing not at random (MNAR) longitudinal data. Results showed that when the missingness mechanism is random‐slope‐dependent MNAR, TLME estimates of both the mean intercept and mean slope are biased because of incorrect weights used in the estimation. More specifically, the estimate of the mean slope is biased towards the mean slope for completers, whereas the estimate of the mean intercept is biased towards the opposite direction as compared to the estimate of the mean slope. We also discuss why the PM approach can provide unbiased fixed‐effects estimates for random‐coefficients‐dependent MNAR data but does not work well for missing at random or outcome‐dependent MNAR data. A small simulation study was conducted to illustrate the results and to compare results from TLME and PM. Results from an empirical data analysis showed that the conceptual finding can be generalized to other real conditions even when some assumptions for the analytical derivation cannot be met. Implications from the analytical and empirical results were discussed and sensitivity analysis was suggested for longitudinal data analysis with missing data.     

Examiner error in curriculum-based measurement of oral reading

Although curriculum based measures of oral reading (CBM-R) have strong technical adequacy, there is still a reason to believe that student performance may be influenced by factors of the testing situation, such as errors examiners make in administering and scoring the test. This study examined the construct-irrelevant variance introduced by examiners using a cross-classified multilevel model. We sought to determine the extent of variance in student CBM-R scores attributable to examiners and, if present, the extent to which it was moderated by students' grade level and English learner (EL) status. Fit indices indicated that a cross-classified random effects model (CCREM) best fits the data with measures nested within students, students nested within schools, and examiners crossing schools. Intraclass correlations of the CCREM revealed that roughly 16% of the variance in student CBM-R scores was associated between examiners. The remaining variance was associated with the measurement level, 3.59%; between students, 75.23%; and between schools, 5.21%. Results were moderated by grade level but not by EL status. The discussion addresses the implications of this error for low-stakes and high-stakes decisions about students, teacher evaluation systems, and hypothesis testing in reading intervention research.     

Emotion Regulation in Workgroups: The Roles of Demographic Diversity and Relational Work Context

Drawing on the social identity perspective, we investigate the cross‐level relationship between demographic diversity in workgroups and emotion regulation. We propose that age, racial, and gender diversity in workgroups relate positively to emotion regulation because of demography‐related in‐group/out‐group dynamics. We also examine the moderating role of the relational work context, specifically task interdependence and social interaction, on the relationship between demographic diversity and emotion regulation. Results from a sample of 2,072 employees in 274 workgroups indicate that working in a group with greater age diversity is positively related to an employee's emotion regulation. Results suggest the operation of the age diversity effect can be attributed primarily to younger employees when they are in workgroups with older coworkers. Results reveal asymmetric effects for racial diversity such that racial out‐group members engage in higher levels of emotion regulation than racial in‐group members when racial diversity is low, whereas racial in‐group members engage in higher levels of emotion regulation than racial out‐group members when racial diversity is high. Race effects also suggest a moderating effect of social interaction; specifically, social interaction weakens the relationship between racial diversity and emotion regulation. Gender effects are not significant.     

Same‐gender and same‐race bias in assessment center ratings: A rating error approach to understanding subgroup differences

This study investigated leniency and similar‐to‐me bias as mechanisms underlying demographic subgroup differences among assessees in assessors’ initial dimension ratings from three assessment center (AC) simulation exercises used as part of high‐stakes promotional testing. It examined whether even small individual‐level effects can accumulate (i.e., “trickle‐up”) to produce larger subgroup‐level differences. Individual‐level analyses were conducted using cross‐classified multilevel modeling and conducted separately for each exercise. Results demonstrated weak evidence of leniency toward White assessees and similar‐to‐me bias among non‐White assessee–assessor pairs. Similar leniency was found toward female assessees, but no statistically significant effects were found for assessee or assessor gender or assessee–assessor gender similarity. Using traditional d effect size estimates, weak individual level assessee effects translated into small but consistent subgroup differences favoring White and female assessees. Generally small but less consistent subgroup differences indicated that non‐White and male assessors gave higher ratings. Moreover, analyses of overall promotion decisions indicate the absence of adverse impact. Findings from this AC provide some support for the “trickle‐up” effect, but the effect on subgroup differences is trivial. The results counter recent reviews of AC studies suggesting larger than previously assumed subgroup differences. Consequently, the findings demonstrate the importance of following established best practices when developing and implementing the AC method for selection purposes to minimize subgroup differences.     

A dynamic investigation of cognitive dedifferentiation with control for retest: evidence from the Swiss Interdisciplinary Longitudinal Study on the Oldest Old

Empirical examinations of the hypothesis of dedifferentiation of cognitive abilities in old and very old age (a) do not account for possible retest effects, which consequently may yield biased estimates of age effects, and (b) focus on time-independent relations (e.g., number of latent constructs, correlations between latent or measured variables). The authors applied a structural equation model with statistical control for retest effects to investigate the dynamic relations between a marker of perceptual speed (cross out) and a marker of verbal fluency (category-fruits). Longitudinal data are from 5 waves of the Swiss Interdisciplinary Longitudinal Study on the Oldest Old (N = 377, baseline age range = 79.5- 84.5 years). The authors found that, independently of retest effects, performance on the cross-out task affected changes in performance on the category task while the opposite did not hold true. This analytical technique could be applied to various markers of broad fluid-mechanic and broad crystallized-pragmatic components of cognition.     

Asymptotic bias of normal-distribution-based maximum likelihood estimates of moderation effects with data missing at random

Moderation analysis is useful for addressing interesting research questions in social sciences and behavioural research. In practice, moderated multiple regression (MMR) models have been most widely used. However, missing data pose a challenge, mainly because the interaction term is a product of two or more variables and thus is a non-linear function of the involved variables. Normal-distribution-based maximum likelihood (NML) has been proposed and applied for estimating MMR models with incomplete data. When data are missing completely at random, moderation effect estimates are consistent. However, simulation results have found that when data in the predictor are missing at random (MAR), NML can yield inaccurate estimates of moderation effects when the moderation effects are non-null. Simulation studies are subject to the limitation of confounding systematic bias with sampling errors. Thus, the purpose of this paper is to analytically derive asymptotic bias of NML estimates of moderation effects with MAR data. Results show that when the moderation effect is zero, there is no asymptotic bias in moderation effect estimates with either normal or non-normal data. When the moderation effect is non-zero, however, asymptotic bias may exist and is determined by factors such as the moderation effect size, missing-data proportion, and type of missingness dependence. Our analytical results suggest that researchers should apply NML to MMR models with caution when missing data exist. Suggestions are given regarding moderation analysis with missing data.     

Cultural and Institutional Determinants of Social Dominance Orientation: A Cross‐Cultural Meta‐Analysis of 27 Societies

We investigate macrocontextual antecedents of national levels of Social Dominance Orientation (SDO). The majority of previous research, in contrast, has tended to focus on individual‐level correlates of SDO. We extend research on Social Dominance Theory by modelling national‐level differences in institutional discrimination, macroeconomic development, and value ideologies as broad situational factors affecting SDO mean levels in previous studies. Our hypotheses were tested in a three‐level meta‐analysis of aggregate data from 50,971 individuals in 95 samples from 27 different societies. Strong effects for hierarchy‐attenuating legitimizing ideologies and gender empowerment were found. Aggregate discrimination against arbitrary‐set groups was less consistently linked to SDO, suggesting that these hierarchies are context‐specific. Using mixed‐effects three‐level modelling, the patterns can be generalized to new contexts and suggest a particular institutional and social climate that fosters high SDO.     

Investigation of age and gender effects on positive orientation in Italian twins

We investigated age and gender effects on “Positive Orientation” (POS)—an individual's tendency to view life with a positive outlook—using a genetically informed design. Study subjects were 1016 twins aged 22–75 from the Italian twin registry. We assessed POS by the recently developed P‐scale. First, we used confirmatory factor analysis to investigate scale's measurement invariance by age and gender. Then, we applied biometric modelling to estimate genetic and environmental components of POS score. Overall, we found a satisfactory degree of measurement invariance by both age and gender. Results from these analyses further indicated an increasing mean level of POS across the lifespan. Additive genetic and unshared environmental factors explained respectively 58% and 42% of variance in POS score, with no significant gender differences; furthermore, the pattern of change of gene‐environment architecture of POS over time was consistent with a greater plasticity of personality at older ages.     

Training the elderly on the ability factors of spatial orientation and inductive reasoning

We examined the effects of cognitive training with elderly participants from the Seattle Longitudinal Study. Subjects were classified as having remained stable or having declined over the previous 14-year interval on each of two primary abilities, spatial orientation and inductive reasoning. Subjects who had declined on one of these abilities received training on that ability; subjects who had declined on both abilities or who had remained stable on both were randomly assigned to the spatial orientation or inductive reasoning training programs. Training outcomes were examined within an ability-measurement framework with empirically determined factorial structure. Significant training effects, at the level of the latent ability constructs, occurred for both spatial orientation and inductive reasoning. These effects were general, in that no significant interactions with decline status or gender were found. Thus, training interventions were effective both in remediating cognitive decline on the target abilities and in improving the performance of stable subjects.     

Individual and dyadic associations among relational self‐expansion potential,affect, and perceived health

A growing body of literature suggests that specific markers of relationship quality are meaningfully linked to health outcomes. We tested whether relational self‐expansion potential might be one of these markers in cross‐sectional samples of individuals and romantic couples. Study 1 found that greater self‐expansion potential was linked to better perceived physical health via both higher positive affect (PA) and lower negative affect (NA). Study 2 replicated these findings for PA (but not NA) and revealed both actor and partner effects of self‐expansion potential. Results remained robust when statistically accounting for gender, age, body mass index, agreeableness, neuroticism, and perceived partner responsiveness. These findings identify a new relationship‐level “active ingredient” associated with health and have implications for future physical health studies.     

Effects of Modeling the Heterogeneity on Inferences Drawn from Multilevel Designs

This article uses Monte Carlo techniques to examine the effect of heterogeneity of variance in multilevel analyses in terms of relative bias, coverage probability, and root mean square error (RMSE). For all simulated data sets, the parameters were estimated using the restricted maximum-likelihood (REML) method both assuming homogeneity and incorporating heterogeneity into multilevel models. We find that (a) the estimates for the fixed parameters are unbiased, but the associated standard errors are frequently biased when heterogeneity is ignored; by contrast, the standard errors of the fixed effects are almost always accurate when heterogeneity is considered; (b) the estimates for the random parameters are slightly overestimated; (c) both the homogeneous and heterogeneous models produce standard errors of the variance component estimates that are underestimated; however, taking heterogeneity into account, the REML-estimations give correct estimates of the standard errors at the lowest level and lead to less underestimated standard errors at the highest level; and (d) from the RMSE point of view, REML accounting for heterogeneity outperforms REML assuming homogeneity; a considerable improvement has been particularly detected for the fixed parameters. Based on this, we conclude that the solution presented can be uniformly adopted. We illustrate the process using a real dataset.     

Socio‐emotional development: From infancy to young adulthood

Results from the Uppsala Longitudinal Study (ULS), which started in 1985, are reported in two sections. The first section gives a summary of longitudinal data from infancy to middle childhood (age 9 years; n = 96) concerning predictions of social functioning aspects from the theoretical perspectives of temperament, attachment, and health psychology (social factors). The second section presents the first results emanating from a follow‐up when participants were 21 years old (n = 85). The developmental roots of social anxiety symptoms were studied from the same perspectives as above, although with a special focus on the predictive power of the temperament trait of shyness/inhibition. Results for middle childhood outcomes showed that temperament characteristics were relevant for most outcomes, whereas the contribution of attachment was most convincingly shown in relation to social competence and personality. Social factors were found to have moderating functions, but direct effects were also shown, the most interesting perhaps being positive effects of non‐parental day care. Results from the 21‐year data confirmed the expected predictive relation from shyness/inhibition to symptoms of social anxiety and further showed this relation to be specific; the relation to symptoms of depression did not survive control for social anxiety, although the opposite was true. The broad analysis of predictor associations with social anxiety, showing the relevance of other temperament factors as well as interactive effects, again attested to the need for multi‐faceted models to analyze developmental trajectories.     

Burnout among Finnish and Chinese university students

In this study the levels of experienced burnout of Finnish and Chinese university students are compared using School Burnout Inventory (SBI). This study is motivated by earlier studies, which suggest that the level of student burnout is different in the culturally distinct Finnish and Chinese university systems, but which are based on different research instruments for the two groups. The sample studied consisted of 3,035 Finnish students and 2,309 Chinese students. Because of the cross‐cultural nature of this study the level of structural equivalence of SBI between the cultural groups was examined and the effect of different response styles on the results was taken into account. Both standard and robust statistical methods were used for the analyses. The results showed that SBI with two extracted components is suitable for cross‐cultural analysis between Finnish and Chinese university students. Virtually no difference was found in experienced overall burnout between the Finnish and Chinese students, which means that both university systems contain factors causing similar levels of student burnout. This study also verified that controlling for the response styles is important in cross‐cultural studies as it was found to have a distinct effect on the results obtained from mean‐level comparisons.     

无均值结构的潜变量交互效应模型的标准化估计

潜变量交互效应建模研究近年来有两项重要进展, 一是提出了潜变量交互效应模型的标准化估计及其计算公式; 二是发现无均值结构模型可以取代传统的有均值结构模型, 建模大为简化。但标准化估计是在传统的有均值结构模型中建立的, 在简化的模型中同样适用吗？本文在无均值结构模型的框架内, 给出了潜变量交互效应模型的标准化形式、计算公式和建模步骤, 并通过模拟研究比较了极大似然和广义最小二乘两种估计方法、配对乘积指标和全部乘积指标两种指标类型, 结果表明, 在计算交互效应的标准化估计时, 应当使用配对乘积指标建模, 并且首选极大似然估计。