Understanding and estimating the power to detect cross-level interaction effects in multilevel modeling

[Correction Notice: An Erratum for this article was reported in Vol 97(5) of Journal of Applied Psychology (see record 2012-18665-001). The article contained production-related errors in a number of the statistical symbols presented in Table 1, the Power in Multilevel Designs section, the Simulation Study section, and the Appendix.] Cross-level interaction effects lie at the heart of multilevel contingency and interactionism theories. Researchers have often lamented the difficulty of finding hypothesized cross-level interactions, and to date there has been no means by which the statistical power of such tests can be evaluated. We develop such a method and report results of a large-scale simulation study, verify its accuracy, and provide evidence regarding the relative importance of factors that affect the power to detect cross-level interactions. Our results indicate that the statistical power to detect cross-level interactions is determined primarily by the magnitude of the cross-level interaction, the standard deviation of lower level slopes, and the lower and upper level sample sizes. We provide a Monte Carlo tool that enables researchers to a priori design more efficient multilevel studies and provides a means by which they can better interpret potential explanations for nonsignificant results. We conclude with recommendations for how scholars might design future multilevel studies that will lead to more accurate inferences regarding the presence of cross-level interactions. (PsycINFO Database Record (c) 2012 APA, all rights reserved).     

Analyzing individual differences in sentence processing performance using multilevel models

The use of multilevel models is increasingly common in the behavioral sciences for analyzing hierarchically structured data, including repeated measures data. These models are flexible and easily implemented via a variety of commercially available statistical software programs. We consider their application in the context of an eye-movement experiment testing readers' responses to a semantic plausibility manipulation in temporarily ambiguous sentences. Multilevel models were used to study the relationship between working memory capacity and the extent to which readers were disrupted by syntactic misanalysis. This represented a cross-level interaction between an individual difference measure and a sentence-level characteristic. We compare a multilevel modeling approach to a standard approach based on ANOVA.     

组织研究中的多层面问题

多层面理论（multilevel theory）已成为组织研究中的一个新范式,它可以研究微观和宏观二者之间的交互作用。为了给组织研究者提供一个思考和决策框架,文章在简要回顾多层面研究的来源后,介绍了3种集体层面的结构、集体层面结构的5种“构成模型”（composition model）、以及1个判断框架,并介绍了多层面研究中常用的4个论证指标（rwg、ICC（1）、ICC（2）、WABAI）的算法和区别,最后对常用的3种分析方法（WABA：within-and-between-analysis、HLM：hierarchical linear modeling、CLOP：cross-level operator）进行了简单介绍和比较。     

On the power of multivariate latent growth curve models to detect correlated change

We evaluated the statistical power of single-indicator latent growth curve models (LGCMs) to detect correlated change between two variables (covariance of slopes) as a function of sample size, number of longitudinal measurement occasions, and reliability (measurement error variance). Power approximations following the method of Satorra and Saris (1985) were used to evaluate the power to detect slope covariances. Even with large samples (N = 500) and several longitudinal occasions (4 or 5), statistical power to detect covariance of slopes was moderate to low unless growth curve reliability at study onset was above .90. Studies using LGCMs may fail to detect slope correlations because of low power rather than a lack of relationship of change between variables. The present findings allow researchers to make more informed design decisions when planning a longitudinal study and aid in interpreting LGCM results regarding correlated interindividual differences in rates of development.     

Procedural justice climate and group power distance: an examination of cross-level interaction effects

In this article, the authors extend research on the cross-level effects of procedural justice climate by theorizing and testing its interaction with group power distance. The results indicated that group power distance moderated the relationships between procedural justice climate and individual-level outcomes (organizational commitment and organization-directed citizenship behavior). More specifically, a larger group power distance was found to attenuate the positive cross-level effects of procedural justice climate. Implications for procedural justice climate research are discussed.     

员工社会资本跨层次契合的动态演化过程及双向作用机制

员工社会资本向企业社会资本的转化是拓展员工价值、获取外部资源的有效途径, 为阐释这一多层次主体互动现象, 提出社会资本跨层次契合的构念, 运用跨层次追踪研究设计, 对其动态演化过程和双向作用机制进行剖析。首先, 探索员工社会资本跨层次契合的维度结构, 并基于此开发跨层次契合量表; 其次, 对社会资本跨层次契合进行过程解构, 探讨员工心理与行为、企业能力与情境在动态演化过程中的作用; 再次, 构建自上而下和自下而上的嵌入与涌现机制理论模型, 将影响员工社会资本跨层次契合的多层次因素整合到同一个理论框架, 并以纵向追踪数据进行实证检验, 厘清作用路径和作用边界。在此基础上, 运用追踪跨案例研究方法, 探讨社会资本跨层次契合的战略选择, 研究结论能为激发员工角色外行为, 有效利用社会资本的管理实践提供启示。     

团队内隐异质性：内涵拓展及其效能机制

从内隐异质性的内涵维度及其效能机制, 包括内隐异质性作用于绩效的中间过程和情境因素、内隐异质性和外显异质性的交互作用、团队断裂带和内隐异质性的跨层次研究方面阐述了团队内隐异质性研究的最新成果和进展, 并在此基础上提出了未来研究的一个整合框架。未来的研究需要在内隐异质性内涵维度及其前因变量、内隐异质性的跨层次研究、团队断裂带及其与团队结果变量之间关系、社会网络背景下内隐异质性与团队绩效之间的关系等方面进一步探索。     

Multilevel Mixture Factor Models

Factor analysis is a statistical method for describing the associations among sets of observed variables in terms of a small number of underlying continuous latent variables. Various authors have proposed multilevel extensions of the factor model for the analysis of data sets with a hierarchical structure. These Multilevel Factor Models (MFMs) have in common that—as in multilevel regression analysis—variation at the higher level is modeled using continuous random effects. In this article, we present an alternative multilevel extension of factor analysis which we call the Multilevel Mixture Factor Model (MMFM). It is based on the assumption that higher level units belong to latent classes that differ in terms of the parameters of the factor model specified for the lower level units. We demonstrate the added value of MMFM compared with MFM, both from a theoretical and applied perspective, and we illustrate the complementarity of the two approaches with an empirical application on students' satisfaction with the University of Florence. The multilevel aspect of this application is that students are nested within study programs, which makes it possible to cluster these programs based on their differences in students' satisfaction.     

Doubly-Latent Models of School Contextual Effects: Integrating Multilevel and Structural Equation Approaches to Control Measurement and Sampling Error

This article is a methodological-substantive synergy. Methodologically, we demonstrate latent-variable contextual models that integrate structural equation models (with multiple indicators) and multilevel models. These models simultaneously control for and unconfound measurement error due to sampling of items at the individual (L1) and group (L2) levels and sampling error due the sampling of persons in the aggregation of L1 characteristics to form L2 constructs. We consider a set of models that are latent or manifest in relation to sampling items (measurement error) and sampling of persons (sampling error) and discuss when different models might be most useful. We demonstrate the flexibility of these 4 core models by extending them to include random slopes, latent (single-level or cross-level) interactions, and latent quadratic effects.

Substantively we use these models to test the big-fish-little-pond effect (BFLPE), showing that individual student levels of academic self-concept (L1-ASC) are positively associated with individual level achievement (L1-ACH) and negatively associated with school-average achievement (L2-ACH)—a finding with important policy implications for the way schools are structured. Extending tests of the BFLPE in new directions, we show that the nonlinear effects of the L1-ACH (a latent quadratic effect) and the interaction between gender and L1-ACH (an L1 × L1 latent interaction) are not significant. Although random-slope models show no significant school-to-school variation in relations between L1-ACH and L1-ASC, the negative effects of L2-ACH (the BFLPE) do vary somewhat with individual L1-ACH.

We conclude with implications for diverse applications of the set of latent contextual models, including recommendations about their implementation, effect size estimates (and confidence intervals) appropriate to multilevel models, and directions for further research in contextual effect analysis.     

The relationship between multilevel models and non‐parametric multilevel mixture models: Discrete approximation of intraclass correlation,random coefficient distributions,and residual heteroscedasticity

Multilevel data structures are common in the social sciences. Often, such nested data are analysed with multilevel models (MLMs) in which heterogeneity between clusters is modelled by continuously distributed random intercepts and/or slopes. Alternatively, the non‐parametric multilevel regression mixture model (NPMM) can accommodate the same nested data structures through discrete latent class variation. The purpose of this article is to delineate analytic relationships between NPMM and MLM parameters that are useful for understanding the indirect interpretation of the NPMM as a non‐parametric approximation of the MLM, with relaxed distributional assumptions. We define how seven standard and non‐standard MLM specifications can be indirectly approximated by particular NPMM specifications. We provide formulas showing how the NPMM can serve as an approximation of the MLM in terms of intraclass correlation, random coefficient means and (co)variances, heteroscedasticity of residuals at level 1, and heteroscedasticity of residuals at level 2. Further, we discuss how these relationships can be useful in practice. The specific relationships are illustrated with simulated graphical demonstrations, and direct and indirect interpretations of NPMM classes are contrasted. We provide an R function to aid in implementing and visualizing an indirect interpretation of NPMM classes. An empirical example is presented and future directions are discussed.     

Joint Relationships of Task Proficiency With Aptitude,Experience, and Task Difficulty: A Cross-Level,Interactional Study

We adopted a cross-level interactional perspective in investigating the prediction of United States Air Force (USAF) jet engine mechanic task-level proficiency from aptitude, experience, and task difficulty. Aptitude, job and task experience, and task difficulty measures all were significant predictors of task proficiency. An hypothesis that task difficulty would moderate the aptitude-task proficiency relationship received no support. Similarly, hypothesized moderating effects of job experience on relationships of task proficiency with aptitude and task difficulty received little support. lhsk-level experience, however, had significant moderating effects on relationships of task proficiency with both aptitude and task difficulty. Aptitude and task difficulty had weaker relationships with task proficiency with increased task experience. We recommend that future research on performance determinants (a) clarify further the experience construct and (b) exploit theoretical and analytic advantages of cross-level and multilevel research designs.     

Unmitigated agency, social support, and psychological adjustment in men with cancer

Unmitigated agency (UA), a gender-linked characteristic, has been associated with poorer cancer adjustment. Support from one's social network typically predicts adjustment but may be poorly matched to UA. The influence of UA on the utility of social support on adjustment over time is examined. Men with cancer (N=55) were assessed initially and 6 months later on three indicators of adjustment. Multilevel modeling analyses varied by adjustment indicator. UA was associated with increased cancer-related psychosocial symptoms but not depressive symptoms or cancer-related thought intrusion. Social support predicted fewer depressive symptoms and less cancer-related thought intrusion. However, a cross-level interaction revealed that the utility of social support on cancer-related thought intrusion was weaker for men with greater levels of UA. Men with cancer likely respond differently to changes in social support depending on their endorsement of UA.     

Multilevel analysis of matching behavior

Multilevel modeling has been considered a promising statistical tool in the field of the experimental analysis of behavior and may serve as a convenient statistical analysis for matching behavior because it structures data in groups (or levels) to account simultaneously for the within‐subject and between‐subject variances. Heretofore, researchers have sometimes pooled data erroneously from different subjects in a single analysis by using average ratios, average response and reinforcer rates, aggregation of subjects, etc. Unfortunately, this leads to loss of information and biased estimations, which can severely undermine generalization of the results. Instead, a multilevel approach is advocated to combine several subjects' matching behavior. A reanalysis of previous data on matching behavior is provided to illustrate the method and point out its advantages. It illustrates that multilevel regression leads to better estimations, is more convenient, and offers more behavioral information. We hope this paper will encourage the use of multilevel modeling in the statistical practices of behavior analysts.     

Neighborhood Protective Effects on Depression in Latinos

Neighborhood social ecologies may have protective effects on depression in Latinos, after adjusting for demographic risk factors, such as nativity and length of stay in the US. This study examines the effects of neighborhood collective efficacy and linguistic isolation on depression in a heterogeneous urban Latino population from 1,468 adult respondents in Los Angeles County. We used multilevel models to analyze how major depression is associated with socioeconomic background, length of stay in the U.S., neighborhood collective efficacy and linguistic isolation among Latinos. A significant cross-level interaction effect was found between collective efficacy and foreign-born Latinos who resided in the US ≥ 15 years. We report cross-level interaction effects between linguistic isolation and nativity for U.S.-born and nativity and duration of residence for foreign-born Latinos who had lived in the U.S. at least 15 years. The moderating effects reported in this study suggest that the benefits of neighborhood collective efficacy and linguistic isolation vary by Latino subgroup and are conceptually discrete forms of social capital and offer insights for community based interventions.     

Using multilevel analyses with sibling data to increase analytic power: an illustration and simulation study

This study investigates the extent to which analytic power can be increased through the inclusion of siblings in a data set and the concomitant use of random coefficient multilevel models. Analyses of real-world data regarding the predictors of young adult alcohol use illustrate how parallel single-level analyses of a 1-child-per-family data set and multilevel analyses of a data set including all siblings in each family would be conducted. A simulation study, closely based on the illustrative analyses, compares the empirical power to detect main, moderation, and mediation effects under three conditions: (a) single-level analyses of 1-child-per-family data, (b) multilevel analyses of all-siblings data, and (c) single-level analyses of independent data with sample size equivalent to the all-siblings condition. Supplementary analyses are conducted to determine the conditions under which greater analytic power could be achieved with the addition of siblings to a data set than with the addition of a lesser number of independent individuals at equivalent cost.     

Catch me if I fall! Enacted uncertainty avoidance and the social safety net as country-level moderators in the job insecurity-job attitudes link

Job insecurity is related to many detrimental outcomes, with reduced job satisfaction and affective organizational commitment being the 2 most prominent reactions. Yet, effect sizes vary greatly, suggesting the presence of moderator variables. On the basis of Lazarus's cognitive appraisal theory, we assumed that country-level enacted uncertainty avoidance and a country's social safety net would affect an individual's appraisal of job insecurity. More specifically, we hypothesized that these 2 country-level variables would buffer the negative relationships between job insecurity and the 2 aforementioned job attitudes. Combining 3 different data sources, we tested the hypotheses in a sample of 15,200 employees from 24 countries by applying multilevel modeling. The results confirmed the hypotheses that both enacted uncertainty avoidance and the social safety net act as cross-level buffer variables. Furthermore, our data revealed that the 2 cross-level interactions share variance in explaining the 2 job attitudes. Our study responds to calls to look at stress processes from a multilevel perspective and highlights the potential importance of governmental regulation when it comes to individual stress processes.     

社会交往情境中权力和反馈对情绪的影响

以往关于权力是否影响情绪的研究存在结果争议,这意味着可能存在其它因素的作用。本研究基于权力控制理论和社会距离理论,探究了社交情境中权力和反馈对情绪的影响。研究1采用经验取样法收集了140名被试五天内的1706段社交经历,研究2采用实验法考察了148名被试的社交经历。结果表明,权力和反馈对情绪存在交互影响:(1)当个体处于低权力情境时,反馈影响情绪,反馈越积极,情绪也越积极;(2)当个体获得积极反馈时,权力不影响情绪;当个体获得消极反馈时,权力影响情绪,权力越高,情绪相对更积极。本研究有助于厘清以往研究关于权力与情绪关系的争议。     

Demonstration and evaluation of a method for assessing mediated moderation

Mediated moderation occurs when the interaction between two variables affects a mediator, which then affects a dependent variable. In this article, we describe the mediated moderation model and evaluate it with a statistical simulation using an adaptation of product-of-coefficients methods to assess mediation. We also demonstrate the use of this method with a substantive example from the adolescent tobacco literature. In the simulation, relative bias (RB) in point estimates and standard errors did not exceed problematic levels of ±10%, although systematic variability in RB was accounted for by parameter size, sample size, and nonzero direct effects. Power to detect mediated moderation effects appears to be severely compromised under one particular combination of conditions: when the component variables that make up the interaction terms are correlated and partial mediated moderation exists. Implications for the estimation of mediated moderation effects in experimental and nonexperimental research are discussed.