Univariate repeated measures techniques applied to multivariate data

Repeated measures designs in psychology have traditionally been analyzed by the univariate mixed model approach, in which the repeated measures effect is tested against an error term based on the subject by treatment interaction. This paper considers the extension of this analysis to designs in which the individual repeated measures are multivariate. Sufficient conditions for a valid multivariate mixed model analysis are given, and a test is described to determine whether or not given data satisfy these conditions.     

Multiway frequency analysis for experimental psychologists.

Many research designs in experimental psychology generate data that are fundamentally discrete or categorical in nature, and produce multiway tables of frequencies. Despite an extensive and, more recently, accessible literature on the topic, multiway frequency analysis is rarely used in experimental psychology. A reason may be the form of exposition in the literature, with emphases and concerns far removed from those of the typical experimental psychologist. An approach to multiway frequency analysis for experimental psychologists is described that has the features we want: asymmetrical designs, factors assessed for their respective main and interactive effects in a manner analogous to ANOVA, and the ability to handle within-subject designs.     

Performing high‐powered studies efficiently with sequential analyses

Running studies with high statistical power, while effect size estimates in psychology are often inaccurate, leads to a practical challenge when designing an experiment. This challenge can be addressed by performing sequential analyses while the data collection is still in progress. At an interim analysis, data collection can be stopped whenever the results are convincing enough to conclude that an effect is present, more data can be collected, or the study can be terminated whenever it is extremely unlikely that the predicted effect will be observed if data collection would be continued. Such interim analyses can be performed while controlling the Type 1 error rate. Sequential analyses can greatly improve the efficiency with which data are collected. Additional flexibility is provided by adaptive designs where sample sizes are increased on the basis of the observed effect size. The need for pre‐registration, ways to prevent experimenter bias, and a comparison between Bayesian approaches and null‐hypothesis significance testing (NHST) are discussed. Sequential analyses, which are widely used in large‐scale medical trials, provide an efficient way to perform high‐powered informative experiments. I hope this introduction will provide a practical primer that allows researchers to incorporate sequential analyses in their research. Copyright © 2014 John Wiley & Sons, Ltd.     

Regression analyses of repeated measures data in cognitive research

Repeated measures designs involving nonorthogonal variables are being used with increasing frequency in cognitive psychology. Researchers usually analyze the data from such designs inappropriately, probably because the designs are not discussed in standard textbooks on regression. Two commonly used approaches to analyzing repeated measures designs are considered in this article. It is argued that both approaches use inappropriate error terms for testing the effects of independent variables. A more appropriate analysis is presented, and two alternative computational procedures for the analysis are illustrated.     

Is the Implicit Association Test a Valid and Valuable Measure of Implicit Consumer Social Cognition?

This article discusses the need for more satisfactory implicit measures in consumer psychology and assesses the theoretical foundations, validity, and value of the Implicit Association Test (IAT) as a measure of implicit consumer social cognition. Study 1 demonstrates the IAT's sensitivity to explicit individual differences in brand attitudes, ownership, and usage frequency, and shows their correlations with lAT‐based measures of implicit brand attitudes and brand relationship strength. In Study 2, the contrast between explicit and implicit measures of attitude toward the ad for sportswear advertisements portraying African American (Black) and European American (White) athlete–spokespersons revealed different patterns of responses to explicit and implicit measures in Black and White respondents. These were explained in terms of self‐presentation biases and system justification theory. Overall, the results demonstrate that the IAT enhances our understanding of consumer responses, particularly when consumers are either unable or unwilling to identify the sources of influence on their behaviors or opinions.     

Bayesian Causal Mediation Analysis for Group Randomized Designs with Homogeneous and Heterogeneous Effects: Simulation and Case Study

A fully Bayesian approach to causal mediation analysis for group-randomized designs is presented. A unique contribution of this article is the combination of Bayesian inferential methods with G-computation to address the problem of heterogeneous treatment or mediator effects. A detailed simulation study shows that this approach has excellent frequentist properties, particularly in the case of small sample sizes with accurate informative priors. The simulation study also demonstrates that the proposed approach can take into account heterogeneous treatment or mediator effects without bias. A case study using data from a school-based randomized intervention designed to increase parent social capital leading to improved behavioral and academic outcomes in children is offered to illustrate the Bayesian approach to causal mediation in group-randomized designs.     

基于两水平被试内设计的中介效应分析方法

在心理学和其他社科研究领域,通常的中介效应分析都基于被试间设计,研究者对于如何分析基于被试内设计的中介效应往往并不清楚。本文阐述了两水平被试内设计的中介效应分析方法(依次检验法和路径分析法),综合各方法优缺点给出一个分析流程,并用应用研究实例演示如何分析两水平被试内中介效应,最后展望了基于被试内设计的中介效应分析研究的拓展方向。     

Non‐parametric three‐way mixed ANOVA with aligned rank tests

Research problems that require a non‐parametric analysis of multifactor designs with repeated measures arise in the behavioural sciences. There is, however, a lack of available procedures in commonly used statistical packages. In the present study, a generalization of the aligned rank test for the two‐way interaction is proposed for the analysis of the typical sources of variation in a three‐way analysis of variance (ANOVA) with repeated measures. It can be implemented in the usual statistical packages. Its statistical properties are tested by using simulation methods with two sample sizes (n = 30 and n = 10) and three distributions (normal, exponential and double exponential). Results indicate substantial increases in power for non‐normal distributions in comparison with the usual parametric tests. Similar levels of Type I error for both parametric and aligned rank ANOVA were obtained with non‐normal distributions and large sample sizes. Degrees‐of‐freedom adjustments for Type I error control in small samples are proposed. The procedure is applied to a case study with 30 participants per group where it detects gender differences in linguistic abilities in blind children not shown previously by other methods.     

Bayesian techniques for analyzing group differences in the Iowa Gambling Task: A case study of intuitive and deliberate decision-makers

The Iowa Gambling Task (IGT) is one of the most popular experimental paradigms for comparing complex decision-making across groups. Most commonly, IGT behavior is analyzed using frequentist tests to compare performance across groups, and to compare inferred parameters of cognitive models developed for the IGT. Here, we present a Bayesian alternative based on Bayesian repeated-measures ANOVA for comparing performance, and a suite of three complementary model-based methods for assessing the cognitive processes underlying IGT performance. The three model-based methods involve Bayesian hierarchical parameter estimation, Bayes factor model comparison, and Bayesian latent-mixture modeling. We illustrate these Bayesian methods by applying them to test the extent to which differences in intuitive versus deliberate decision style are associated with differences in IGT performance. The results show that intuitive and deliberate decision-makers behave similarly on the IGT, and the modeling analyses consistently suggest that both groups of decision-makers rely on similar cognitive processes. Our results challenge the notion that individual differences in intuitive and deliberate decision styles have a broad impact on decision-making. They also highlight the advantages of Bayesian methods, especially their ability to quantify evidence in favor of the null hypothesis, and that they allow model-based analyses to incorporate hierarchical and latent-mixture structures.     

Psychometric Modeling of response speed and accuracy with mixed and conditional regression

Human performance in cognitive testing and experimental psychology is expressed in terms of response speed and accuracy. Data analysis is often limited to either speed or accuracy, and/or to crude summary measures like mean response time (RT) or the percentage correct responses. This paper proposes the use of mixed regression for the psychometric modeling of response speed and accuracy in testing and experiments. Mixed logistic regression of response accuracy extends logistic item response theory modeling to multidimensional models with covariates and interactions. Mixed linear regression of response time extends mixed ANOVA to unbalanced designs with covariates and heterogeneity of variance. Related to mixed regression is conditional regression, which requires no normality assumption, but is limited to unidimensional models. Mixed and conditional methods are both applied to an experimental study of mental rotation. Univariate and bivariate analyzes show how within-subject correlation between response and RT can be distinguished from between-subject correlation, and how latent traits can be detected, given careful item design or content analysis. It is concluded that both response and RT must be recorded in cognitive testing, and that mixed regression is a versatile method for analyzing test data.I am grateful to Rogier Donders for putting his data at my disposal.     

Neuromarketing as a scale validation tool: Understanding individual differences based on the style of processing scale in affective judgements

This paper revisits a well-cited and widely applied consumer scale, Style of Processing (SOP) (Childers et al., Journal of Consumer Research, 1985, 12 , 125), that has been used to investigate individual differences in processing visual versus verbal information in marketing. The scale has advanced knowledge in fields related to marketing communications, product development, psychology, advertising, education and learning theories, shedding light on our understanding of consumer psychology related to persuasion, comprehension, memory, and other consumer cognitive processes involving information. In a research dialog that took place in 2008, a need for further SOP validation was suggested using a neuromarketing approach. We took this call forward and conducted an event-related-potential (ERP) experimental research study using electroencephalogram (EEG) to validate the SOP scale, focusing on differential affective processing between verbalizers and visualizers. We not only demonstrate how neuromarketing tools can be utilized to provide evidence for scale validity, providing advantages over self-reported measures; but more importantly, address issues related to understanding differential fluency effects that exist between visualizers and verbalizers. Behavioral data revealed varying reaction times to emotional stimuli of a pictorial nature. We further identify two ERP components in our data, early left anterior negativity (ELAN) and late negative slow wave (LNSW), that differentiate individual processing fluency in affective versus evaluative-based judgements. Findings confirmed the construct validity of the SOP scale and enhance our understanding of individual differences in emotional processing of pictorial information.     

A Bayesian Approach for Estimating Mediation Effects With Missing Data

Methodologists have developed mediation analysis techniques for a broad range of substantive applications, yet methods for estimating mediating mechanisms with missing data have been understudied. This study outlined a general Bayesian missing data handling approach that can accommodate mediation analyses with any number of manifest variables. Computer simulation studies showed that the Bayesian approach produced frequentist coverage rates and power estimates that were comparable to those of maximum likelihood with the bias-corrected bootstrap. We share an SAS macro that implements Bayesian estimation and use 2 data analysis examples to demonstrate its use.     

Standard errors and confidence intervals in within-subjects designs: Generalizing Loftus and Masson (1994) and avoiding the biases of alternative accounts

Repeated measures designs are common in experimental psychology. Because of the correlational structure in these designs, the calculation and interpretation of confidence intervals is nontrivial. One solution was provided by Loftus and Masson (Psychonomic Bulletin & Review 1:476-490, 1994). This solution, although widely adopted, has the limitation of implying same-size confidence intervals for all factor levels, and therefore does not allow for the assessment of variance homogeneity assumptions (i.e., the circularity assumption, which is crucial for the repeated measures ANOVA). This limitation and the method's perceived complexity have sometimes led scientists to use a simplified variant, based on a per-subject normalization of the data (Bakeman & McArthur, Behavior Research Methods, Instruments, & Computers 28:584-589, 1996; Cousineau, Tutorials in Quantitative Methods for Psychology 1:42-45, 2005; Morey, Tutorials in Quantitative Methods for Psychology 4:61-64, 2008; Morrison & Weaver, Behavior Research Methods, Instruments, & Computers 27:52-56, 1995). We show that this normalization method leads to biased results and is uninformative with regard to circularity. Instead, we provide a simple, intuitive generalization of the Loftus and Masson method that allows for assessment of the circularity assumption.     

Applying mixed‐effects modeling to single‐subject designs: An introduction

Behavior analysis and statistical inference have shared a conflicted relationship for over fifty years. However, a significant portion of this conflict is directed toward statistical tests (e.g., t‐tests, ANOVA) that aggregate group and/or temporal variability into means and standard deviations and as a result remove much of the data important to behavior analysts. Mixed‐effects modeling, a more recently developed statistical test, addresses many of the limitations of more basic tests by incorporating random effects. Random effects quantify individual subject variability without eliminating it from the model, hence producing a model that can predict both group and individual behavior. We present the results of a generalized linear mixed‐effects model applied to single‐subject data taken from Ackerlund Brandt, Dozier, Juanico, Laudont, & Mick, 2015, in which children chose from one of three reinforcers for completing a task. Results of the mixed‐effects modeling are consistent with visual analyses and importantly provide a statistical framework to predict individual behavior without requiring aggregation. We conclude by discussing the implications of these results and provide recommendations for further integration of mixed‐effects models in the analyses of single‐subject designs.     

Power analysis for multivariate and repeated measures designs: A flexible approach using the SPSS MANOVA procedure

Although power analysis is an important component in the planning and implementation of research designs, it is often ignored. Computer programs for performing power analysis are available, but most have limitations, particularly for complex multivariate designs. An SPSS procedure is presented that can be used for calculating power for univariate, multivariate, and repeated measures models with and without time-varying and time-constant covariates. Three examples provide a framework for calculating power via this method: an ANCOVA, a MANOVA, and a repeated measures ANOVA with two or more groups. The benefits and limitations of this procedure are discussed.     

A review of research methods in IO/OB work-family research

A methodological review was conducted of work-family (WF) research published in industrial-organizational psychology and organizational behavior journals over a period of 24 years (1980-2003). Content analysis was conducted on 225 individual studies published in 210 articles to categorize methodological features, including the research design, sources of data used, data analysis techniques, reliability and validity of measures used, and sociodemographic characteristics of the samples. Results support many of the criticisms of WF research and suggest that scholars publishing WF research in industrial-organizational psychology and organizational behavior journals could make greater use of longitudinal and experimental research designs, gather more multisource data, and move beyond the individual level of analysis. Adopting more diverse conceptualizations of family, including a greater proportion of racial and ethnic minorities, and studying workers in occupations other than managerial or professional positions also appear warranted. Finally, methodological trends varied across specific WF content areas, which suggests that distinct methodologies might be useful to advance knowledge of specific WF topics.     

大学积极心理学课程的双元互动教学模式改革实验研究

采用实验组对照组前后测设计对232名大学生进行重复测量方差分析和中介效应检验,考查双元互动教学模式下大学积极心理学教学对其学生积极心理学知识、态度、情感、价值观的改善效果。结果发现：（1）双元互动教学模式能显著提升大学生的积极心理学知识;（2）双元互动教学模式下大学生的总体幸福感、积极的价值观念显著提高,生活满意度得到保持;（3）在双元互动教学模式下大学生个人价值观以生活满意度为部分中介促进总体幸福感形成,以促成教学目标。由此可见,双元互动教学模式可以实现大学积极心理学学科教育和情感培养的双目标。     

Power analysis for multivariate and repeated measurements designs via SPSS: Correction and extension of D’Amico, Neilands, and Zambarano (2001)

D'Amico, Neilands, and Zambarano (2001) published SPSS syntax to perform power analyses for three complex procedures: ANCOVA, MANOVA, and repeated measures ANOVA. Unfortunately, the published SPSS syntax for performing the repeated measures analysis needed some minor revision in order to perform the analysis correctly. This article presents the corrected syntax that will successfully perform the repeated measures analysis and provides some guidance on modifying the syntax to customize the analysis.     

Multilevel models for the experimental psychologist: Foundations and illustrative examples

Although common in the educational and developmental areas, multilevel models are not often utilized in the analysis of data from experimental designs. This article illustrates how multilevel models can be useful with two examples from experimental designs with repeated measurements not involving time. One example demonstrates how to properly examine independent variables for experimental stimuli or individuals that are categorical, continuous, or semicontinuous in the presence of missing data. The second example demonstrates how response times and error rates can be modeled simultaneously within a multivariate model in order to examine speed—accuracy trade-offs at the experimental-condition and individual levels, as well as to examine differences in the magnitude of effects across outcomes. SPSS and SAS syntax for the examples are available electronically.