A Comparison of Population-Averaged and Cluster-Specific Approaches in the Context of Unequal Probabilities of Selection

Sampling designs of large-scale survey studies are typically complex, involving multiple design features such as clustering and unequal probabilities of selection. Single-level (i.e., population-averaged) methods that use adjusted variance estimators and multilevel (i.e., cluster-specific) methods provide two alternatives for modeling clustered data. Although the literature comparing these methods is vast, comparisons have been limited to the context in which all sampling units are selected with equal probabilities (thus circumventing the need for sampling weights). The goal of this study was to determine under what conditions single-level and multilevel estimators outperform one another in the context of a two-stage sampling design with unequal probabilities of selection. Monte Carlo simulation methods were used to evaluate the impact of several factors, including population model, informativeness of the design, distribution of the outcome variable, intraclass correlation coefficient, cluster size, and estimation method. Results indicated that the unweighted estimators performed similarly across conditions, whereas the weighted single-level estimators tended to outperform the weighted multilevel estimators, particularly under nonideal sample conditions. Multilevel weight approximation methods did not perform well when the design was informative. An empirical example is provided to demonstrate how researchers might investigate the implications of the simulation results in practice.     

Bayesian and maximum likelihood estimation of hierarchical response time models

Hierarchical (or multilevel) statistical models have become increasingly popular in psychology in the last few years. In this article, we consider the application of multilevel modeling to the ex-Gaussian, a popular model of response times. We compare single-level and hierarchical methods for estimation of the parameters of ex-Gaussian distributions. In addition, for each approach, we compare maximum likelihood estimation with Bayesian estimation. A set of simulations and analyses of parameter recovery show that although all methods perform adequately well, hierarchical methods are better able to recover the parameters of the ex-Gaussian, by reducing variability in the recovered parameters. At each level, little overall difference was observed between the maximum likelihood and Bayesian methods.     

基于多层验证性因素分析的各种信度系数方法

近年来,心理学研究的复现性受到广泛关注,许多实证研究难以重复验证,信度较低。大量研究使用多层技术,但只报告整体信度,导致研究可重复危机。基于各种信度系数和验证性因素分析的理论,以二层模型为例,总结多层信度计算方法,通过文献综述检索应用情况,并用MPLUS进行实例演示,最后讨论单层信度估计存在的偏差及分层估计的好处。总之,对多层数据进行分层信度估计是很有必要的,可消除因测量工具缺乏信度而导致的研究不可重复。     

Multiple Group Analysis in Multilevel Structural Equation Model Across Level 1 Groups

This article introduces and evaluates a procedure for conducting multiple group analysis in multilevel structural equation model across Level 1 groups (MG1-MSEM; Ryu, 2014). When group membership is at Level 1, multiple group analysis raises two issues that cannot be solved by a simple extension of the standard multiple group analysis in single-level structural equation model. First, the Level 2 data are not independent between Level 1 groups. Second, the standard procedure fails to take into account the dependency between members of different Level 1 groups within the same cluster. The MG1-MSEM approach provides solutions to these problems. In MG1-MSEM, the Level 1 mean structure is necessary to represent the differences between Level 1 groups within clusters. The Level 2 model is the same regardless of Level 1 group membership. A simulation study examined the performance of MUML (Muthén's maximum likelihood) estimation in MG1-MSEM. The MG1-MSEM approach is illustrated for both a multilevel path model and a multilevel factor model using empirical data sets.     

The Performance of Multivariate Methods for Two-Group Comparisons with Small Samples and Incomplete Data

Abstract

In intervention studies having multiple outcomes, researchers often use a series of univariate tests (e.g., ANOVAs) to assess group mean differences. Previous research found that this approach properly controls Type I error and generally provides greater power compared to MANOVA, especially under realistic effect size and correlation combinations. However, when group differences are assessed for a specific outcome, these procedures are strictly univariate and do not consider the outcome correlations, which may be problematic with missing outcome data. Linear mixed or multivariate multilevel models (MVMMs), implemented with maximum likelihood estimation, present an alternative analysis option where outcome correlations are taken into account when specific group mean differences are estimated. In this study, we use simulation methods to compare the performance of separate independent samples t tests estimated with ordinary least squares and analogous t tests from MVMMs to assess two-group mean differences with multiple outcomes under small sample and missingness conditions. Study results indicated that a MVMM implemented with restricted maximum likelihood estimation combined with the Kenward–Roger correction had the best performance. Therefore, for intervention studies with small N and normally distributed multivariate outcomes, the Kenward–Roger procedure is recommended over traditional methods and conventional MVMM analyses, particularly with incomplete data.     

Personality characteristics of alcohol dependent inpatients: relationship of MCMI subtypes to self-reported drinking behavior.

Recent cluster analytic research with alcoholic inpatients has demonstrated the existence of several Millon Clinical Multiaxial Inventory (MCMI) clusters that appear to be consistent across different subject samples. The validity of these data would be strengthened by a statistical demonstration of the similarity of attained clusters across studies--a demonstration of concordance of subject classification across different clustering techniques on the same data set- and the inclusion of external, independent measures against which to evaluate the predictive validity of the cluster typology. We found a high level of concordance in subject classification across different clustering methods on the same data set and a high level of agreement with cluster typologies attained in previous studies. Subsequent multivariate analyses employing independent scales measuring various aspects of alcohol use confirmed differences among cluster members on perceived benefits of alcohol use and deleterious effects of alcohol use. The prominent differences in alcohol use along with a rationale for their development are discussed.     

Multilevel covariance structure analysis of sibling substance use and intrafamily conflict

Conventional covariance structure analysis, such as factor analysis, is often applied to data that are obtained in a hierarchical fashion, such as siblings observed within families. A more appropriate specification is demonstrated which explicitly models the within-level and between-level covariance matrices of sibling substance use and intrafamily conflict. Participants were 267 target adolescents (mean age=13.11 years) and 318 siblings (mean age=15.03 years). The level of homogeneity within sibling clusters, and heterogeneity among families, was sufficient to conduct a multilevel covariance structure analysis (MCA). Parental and family-level variables consisting of marital status, socioeconomic status, marital discord, parent use, and modeling of substances were used to explain heterogeneity among families. Marital discord predicted intrafamily conflict, and parent marital status and modeling of substances predicted sibling substance use. Advantages and uses of hierarchical designs and conventional covariance structure software for multilevel data are discussed.     

Failure Avoidance Motivation in a Goal-Setting Situation

Scholars have consistently identified contextual performance or organizational citizenship behavior as a core component of job performance. The current literature on this issue has been dominated by a single-level approach, typically conducted at the individual level of analysis. This study adopts a multilevel approach to simultaneously examine main effects of and cross-level interactions among individual- and group-level predictors of interpersonal helping behavior. Results from a large-scale longitudinal data set show that at the individual level, helping behavior was predicted by perceived organizational support (POS), fairness, and affective commitment. At the group level, helping behavior was predicted by trust among group members. Trust among members also significantly moderated the individual-level relationships between POS and helping behavior and between fairness and helping. These crosslevel moderations indicated that the group- and individual-level predictors were complementary (instead of mutually reinforcing) in predicting interpersonal helping behavior. This finding indicates that various antecedents of interpersonal helping are characterized by distinct dynamics at the individual and group levels of analysis.     

Structural Equation Modeling Approaches for Analyzing Partially Nested Data

Study designs involving clustering in some study arms, but not all study arms, are common in clinical treatment-outcome and educational settings. For instance, in a treatment arm, persons may be nested in therapy groups, whereas in a control arm there are no groups. Methodological approaches for handling such partially nested designs have recently been developed in a multilevel modeling framework (MLM-PN) and have proved very useful. We introduce two alternative structural equation modeling (SEM) approaches for analyzing partially nested data: a multivariate single-level SEM (SSEM-PN) and a multiple-arm multilevel SEM (MSEM-PN). We show how SSEM-PN and MSEM-PN can produce results equivalent to existing MLM-PNs and can be extended to flexibly accommodate several modeling features that are difficult or impossible to handle in MLM-PNs. For instance, using an SSEM-PN or MSEM-PN, it is possible to specify complex structural models involving cluster-level outcomes, obtain absolute model fit, decompose person-level predictor effects in the treatment arm using latent cluster means, and include traditional factors as predictors/outcomes. Importantly, implementation of such features for partially nested designs differs from that for fully nested designs. An empirical example involving a partially nested depression intervention combines several of these features in an analysis of interest for treatment-outcome studies.     

On the Multilevel Nature of Meta-Analysis: A Tutorial,Comparison of Software Programs,and Discussion of Analytic Choices

The term “multilevel meta-analysis” is encountered not only in applied research studies, but in multilevel resources comparing traditional meta-analysis to multilevel meta-analysis. In this tutorial, we argue that the term “multilevel meta-analysis” is redundant since all meta-analysis can be formulated as a special kind of multilevel model. To clarify the multilevel nature of meta-analysis the four standard meta-analytic models are presented using multilevel equations and fit to an example data set using four software programs: two specific to meta-analysis (metafor in R and SPSS macros) and two specific to multilevel modeling (PROC MIXED in SAS and HLM). The same parameter estimates are obtained across programs underscoring that all meta-analyses are multilevel in nature. Despite the equivalent results, not all software programs are alike and differences are noted in the output provided and estimators available. This tutorial also recasts distinctions made in the literature between traditional and multilevel meta-analysis as differences between meta-analytic choices, not between meta-analytic models, and provides guidance to inform choices in estimators, significance tests, moderator analyses, and modeling sequence. The extent to which the software programs allow flexibility with respect to these decisions is noted, with metafor emerging as the most favorable program reviewed.     

控制单水平研究中的多水平误差：基于设计的方法

由于取样设计的原因,多水平数据结构不仅存在于多水平研究,也广泛存在于单水平研究,需要在单水平分析中控制多水平误差。此时使用多层线性模型发挥不了优势,反而因模型的复杂性带来麻烦。基于设计的方法相对更简单、高效和稳健,更契合含多水平误差的单水平研究情境。在详细介绍基于设计的方法及其优势后,利用数据实例展示基于设计的方法在单水平研究中控制多水平误差的效果,并为应用研究者提供方法选用建议。     

Methodological issues in growth‐curve analyses with married couples

Because longitudinal data are increasingly being used to test predictions regarding close relationships, researchers are also increasingly being confronted with methodological issues unique to the analysis of longitudinal data. In this paper, four issues in conducting growth‐curve analyses with married couples are examined: assessing statistical assumptions about homoscedastic and independent errors, handling information about missing data, dealing with couples with only one assessment, and estimating quadratic effects. Each issue is illustrated with actual data, and syntax from the multilevel module in LISREL 8.52 is provided for specific analyses. Guidelines are presented for helping researchers think through each issue with their own data set.     

Differential effects of arousal in positive and negative autobiographical memories

Autobiographical memories are characterised by a range of emotions and emotional reactions. Recent research has demonstrated that differences in emotional valence (positive vs. negative emotion) and arousal (the degree of emotional intensity) differentially influence the retrieved memory narrative. Although the mnemonic effects of valence and arousal have both been heavily studied, it is currently unclear whether the effects of emotional arousal are equivalent for positive and negative autobiographical events. In the current study, multilevel models were used to examine differential effects of emotional valence and arousal on the richness of autobiographical memory retrieval both between and within subjects. Thirty-four young adults were asked to retrieve personal autobiographical memories associated with popular musical cues and to rate the valence, arousal and richness of these events. The multilevel analyses identified independent influences of valence and intensity upon retrieval characteristics at the within- and between-subject levels. In addition, the within-subject interactions between valence and arousal highlighted differential effects of arousal for positive and negative memories. These findings have important implications for future studies of emotion and memory, highlighting the importance of considering both valence and arousal when examining the role emotion plays in the richness of memory representation.     

A REVIEW OF COMPARISON STUDIES IN APPLIED BEHAVIOR ANALYSIS

A commonly used research design in applied behavior analysis involves comparing two or more independent variables. Typically, the relative effectiveness of two different interventions is measured on a single dependent variable. In the current review, 54 comparison studies from seven different peer‐reviewed, behavior analytic journals were evaluated between the years 2002 and 2011. Each study was evaluated across seven dimensions: (1) experimental design, (2) setting, (3) participants, (4) type of comparison, (5) number of comparisons, (6) treatment integrity, and (7) outcome. There were some consistencies across studies, with half resulting in equivalent outcomes across comparisons. In addition, most studies employed the use of an alternating treatments or multi‐element single‐subject design and compared a teaching methodology. On the basis of these results, the value of comparison study as well as directions for future comparison research is discussed. Overall, comparison study is a worthy and important enterprise that requires a high degree of experimental control and a careful analyses of the results, regardless of whether the outcome clearly favored one independent variable or not. Copyright © 2013 John Wiley & Sons, Ltd.     

有中介的调节模型的拓展及其效应量

传统的有中介的调节(mediated moderation, meMO)模型关于误差方差齐性的假设经常被违背, 应用研究中也缺乏测量meMO效应大小的指标。对于单层数据, 本文借助于两层建模的思想, 提出了一种可用于处理方差非齐性的两层有中介的调节(2meMO)模型; 给出了用于测量meMO分析中总调节效应、直接调节效应和有中介调节效应大小的效应量。通过Monte Carlo模拟研究, 比较了meMO和2meMO模型在参数和效应量估计上的表现。并通过实际案例解释了2meMO模型的应用以及效应量的计算和解释。     

Effectively Analyzing Change over Time in Laboratory Research on Stress and Health: A Multilevel Modeling Approach

Stress and health researchers often utilize standardized laboratory stress tasks to evaluate the physical and psychological consequences of challenging experiences. These laboratory sessions usually include multiple measurements of physical and psychological responses collected over time. Multilevel modeling allows researchers to make use of all available data points to model the trajectory of change over time, and within distinct task periods such as baseline, stressor, and recovery. To effectively predict future health outcomes it is important to examine both stress‐related reactivity and recovery. In this paper, we review the analytic approaches used in recent laboratory stress research and note that many recent articles have aggregated multiple responses, used difference scores, or conducted repeated measures analysis of variance (ANOVA). Relatively few studies used a multilevel modeling approach. We highlight the advantages of a multilevel modeling approach and provide an example for using this approach as an alternative to repeated measures ANOVA and difference scores.     

Twin and sibling attachment in a nationally representative sample

The objective of this research was to evaluate the hypothesis that twin relationships are attachments, using data from a nationally representative sample. The results indicated that twin siblings were more likely than nontwin siblings to be attached to their siblings. Moreover, analyses indicated that both attachment theoretical and inclusive fitness perspectives are necessary for explaining these findings. Namely, twins were more likely to be attached than nontwin siblings, as expected from an attachment perspective. But identical twins were more likely than fraternal twins to be attached to one another, as might be expected from an inclusive fitness perspective. Cross-sectional analyses indicated that older people are less likely than younger people to use their sibling as an attachment figure compared to younger people and that married adults are less likely to use their sibling as an attachment figure than nonmarried people.     

Does elevated power lead to approach and reduced power to inhibition? Comment on Keltner, Gruenfeld, and Anderson (2003)

D. Keltner, D. H. Gruenfeld, and C. Anderson (see record 2003-00307-004) stated a set of propositions postulating independent effects for elevated power and reduced power. The present commentary argues that past studies have permitted examining the opposite effects but not the specific effects of high and low power. Suggestions are made for improving designs and formulating analytic strategies that would permit evaluating the specific assertions that elevated power increases approach and reduced power increases inhibition.