Regression analyses of repeated measures data in cognitive research

Repeated measures designs involving nonorthogonal variables are being used with increasing frequency in cognitive psychology. Researchers usually analyze the data from such designs inappropriately, probably because the designs are not discussed in standard textbooks on regression. Two commonly used approaches to analyzing repeated measures designs are considered in this article. It is argued that both approaches use inappropriate error terms for testing the effects of independent variables. A more appropriate analysis is presented, and two alternative computational procedures for the analysis are illustrated.     

Adjusting researchers’ approach to adjustment: On the use of covariates when testing interactions

A common design in social psychology involves the use of two independent variables, an experimental manipulation and a measured individual difference, and the interest is in the interaction between them. In such designs, there are often obvious covariate(s), correlated with the measured independent variable, which the researcher wishes to control. Typically this is done by including the covariate in the analytic model. We show that in most cases this is an inadequate model. In general, the interaction between the two independent variables will be estimated without bias only when the interaction between the covariate and the manipulated independent variable is included in the analysis. We present simulations showing the factors affecting the magnitude of the bias and provide a survey of recent social psychological literature illustrating the frequency of the problem. Finally, we discuss cases where both independent variables are manipulated and the covariate is a potential mediator.     

Uneasiness with facet analysis: time for a reappraisal

Although already several decades old, the facet analysis (also called facet theory) has not been able to assert itself in the field of psychology. Here the reasons for the widespread uneasiness with the facet analysis will be presented starting with an outline of the approach. This will make it clear that the facet analysis does not represent a research method in the narrower sense and definitely not a "theory" but a method with the status of a logical principle of thought. In experimental psychology this principle has been used successfully for a long time in the form of multifactorial experimental designs. However, multifactorial measurement designs are still few and far between in differential and diagnostic psychology. This can be explained especially by the fact that an important aspect of validity--the validity of construct differentiations--has been ignored. Because of a principle rejection of factor analytic methods, even the proponents of the facet analysis have overlooked the central contribution of their approach with respect to the validity of measurement methods.     

BANOVA: Bayesian Analysis of Experiments in Consumer Psychology

This article introduces a Bayesian extension of ANOVA for the analysis of experimental data in consumer psychology. The approach, called BANOVA (Bayesian ANOVA), addresses some common challenges that consumer psychologists encounter in their experimental work, and is specifically suited for the analysis of repeated measures designs. There appears to be a recent surge in interest in those designs based on the recognition that they are sensitive to individual differences in response to experimental treatments and that they offer advantages for assessing causal mediating mechanisms, even at the individual level. BANOVA enables the analysis of repeated measures data derived from mixed within–between‐subjects experiments with Normal and nonNormal‐dependent variables and accommodates unobserved individual differences. It allows for the calculation of effect sizes, planned comparisons, simple effects, spotlight and floodlight analyses, and includes a wide range of mediation, moderation, and moderated mediation analyses. An R software package implements these analyses, and aims to provide a one‐stop shop for the analysis of experiments in consumer psychology. The package is illustrated through applications to a number of data sets from previously published studies.     

F-test bias for experimental designs in educational research

Reference is made to Neyman's study ofF-test bias for the randomized blocks and Latin square designs employed in agriculture, and some account is given of later statistical developments which sprang from his work—in particular, the classification of model-types and the technique of variance component analysis. It is claimed that there is a need to carry out an examination ofF-test bias for experimental designs in education and psychology which will utilize the method and, where appropriate, the known' results of this new branch of variance analysis. In the present paper, such an investigation is carried out for designs which may be regarded as derivatives of the agricultural randomized blocks design. In a paper to follow, a similar investigation will be carried out for experimental designs of the Latin square type.     

Multiway contingency tables: Monte Carlo resampling probability values for the chi-squared and likelihood-ratio tests

Monte Carlo resampling methods to obtain probability values for chi-squared and likelihood-ratio test statistics for multiway contingency tables are presented. A resampling algorithm provides random arrangements of cell frequencies in a multiway contingency table, given fixed marginal frequency totals. Probability values are obtained from the proportion of resampled test statistic values equal to or greater than the observed test statistic value.     

The Impact of CACREP Accreditation: A Multiway Frequency Analysis of Ethics Violations and Sanctions

The impact of CACREP accreditation on counselor competency has received little empirical investigation. Differences in the frequency and type of ethical misconduct between graduates of CACREP‐accredited and non‐CACREP‐accredited counselor education programs were investigated. Results of a multiway frequency analysis indicated that fully licensed graduates of CACREP‐accredited programs were sanctioned for ethical misconduct significantly less frequently than were graduates of non‐CACREP‐accredited programs. Additionally, the accreditation factor was among the highest‐ranking parameter estimates of expected cell frequencies.     

Resampling programs for multiway contingency tables with fixed marginal frequency totals

A resampling algorithm is presented for analyzing multiway contingency tables with fixed marginal frequency totals. Applications are illustrated with extensions of Fisher's exact, Pearson's chi-squared, and likelihood-ratio tests to three-way contingency tables.     

125 years of the American Journal of Psychology: a historical overview

The American journal of Psychology celebrates 125 years of publication this year. From its inception, the Journal has attempted to record and communicate the results of research conducted in laboratories of psychology. It has also provided its readers with laboratory plans and designs for apparatus for research and demonstrations and described experimental procedures to facilitate the conduct of research. Its attention to reviews of books over a wide range of psychological topics and its inclusion of articles that provide historical perspectives on the development of psychology and its concerns broaden the context in which laboratory research is carried out. This brief overview of the Journal's history offers a perspective on the role of the Journal in, and its contributions to, the development of scientific psychology.     

Disability research in counseling psychology journals: a 20-year content analysis

We conducted an exploratory content analysis of disability research in 5 major counseling psychology journals between 1990 and 2010. The goal was to review the counseling psychology literature to better understand the prevalence of disability research, identify research methods most often conducted, and elucidate the types of concerns most studied. We searched 5 journals (Journal of Counseling & Development, Journal of Counseling Psychology, The Counseling Psychologist, Journal of Multicultural Counseling and Development, and Cultural Diversity and Ethnic Minority Psychology) using keywords derived from disability terms defined by the Individuals With Disabilities Education Improvement Act of 2004 (IDEA). Articles were categorized by methodology and disability category examined. We found that disability research continues to comprise an extremely small amount (from less than 1% to 2.7%) of the counseling psychology literature, with the frequency of articles plateauing in recent years. The research design of articles published has changed, with an increased number of empirical articles and a decrease in literature reviews. We conclude by calling for increased empirical investigation of disability among journals specific to counseling psychology to recognize disability as an important aspect of diversity within the field.     

Moving beyond the enduring dominance of positivism in psychological research: Implications for psychology in Australia

Almost since its inception, the dominant narrative of modern psychology has embraced positivism through its insistence that psychological science is objective, generalisable, and value free (or neutral). Consequently, quantitative research and, in particular, experimental designs, are privileged over other forms of enquiry, and other epistemologies, methodologies, and methods remain marginalised within the discipline. We argue that the enduring hegemony of positivism needs to be opposed to enable psychology to genuinely understand the antecedents of, and provide meaningful sustainable solutions for, complex human issues without being constrained by a narrow focus on method. We discuss the ways in which psychology in Australia can move towards embracing a constructionist epistemology that provides the framework for methodological pluralism. We provide a number of suggestions for change across the interrelated areas of accreditation, curriculum, the Australian Psychological Society, and research.     

A review of research methods in IO/OB work-family research

A methodological review was conducted of work-family (WF) research published in industrial-organizational psychology and organizational behavior journals over a period of 24 years (1980-2003). Content analysis was conducted on 225 individual studies published in 210 articles to categorize methodological features, including the research design, sources of data used, data analysis techniques, reliability and validity of measures used, and sociodemographic characteristics of the samples. Results support many of the criticisms of WF research and suggest that scholars publishing WF research in industrial-organizational psychology and organizational behavior journals could make greater use of longitudinal and experimental research designs, gather more multisource data, and move beyond the individual level of analysis. Adopting more diverse conceptualizations of family, including a greater proportion of racial and ethnic minorities, and studying workers in occupations other than managerial or professional positions also appear warranted. Finally, methodological trends varied across specific WF content areas, which suggests that distinct methodologies might be useful to advance knowledge of specific WF topics.     

Experimental psychology and the courtroom

Experimental social and cognitive psychology cover a range of topics such as attitudes, attribution, decision making, perception, memory, comprehension, and reasoning. The relevance of these topics to the law is apparent in such domains as jury selection, jury behavior, plea bargaining, eyewitness testimony, sentencing, and police investigation strategies. Despite early attempts to apply experimental psychology to law (circa 1900–1920), it was not until the mid 1970s that a robust law-relevant literature in experimental psychology began to unfold. The experimental psychology/law interface continues to experience some problems in (1) generalizing from specific research experiments to real-world conditions, and (2) identifying solutions rather than just problems.     

A note on the analysis of designs in which subjects receive each stimulus only once

In social psychological experiments, the manipulations of interest are often presented to subjects along with, or as part of, some stimulus. An example would be a manipulation of the verbal label associated with a stimulus photograph, in a person perception study. Ordinarily it is not possible for stimuli to be completely crossed with treatments because subjects cannot be exposed to any stimulus more than once. In such designs it is wise to counterbalance the assignment of stimuli to treatment conditions, but this gives rise to difficulties in the analysis of the experimental data. Data from such designs have frequently been misanalyzed in the published literature. This paper presents a method of constructing a counter-balancing scheme to simplify the analysis, and appropriate methods of analysis for both the simplified and the general case. It is emphasized that stimuli are generally best treated as a random factor in such designs, permitting increased generalizability, but the case where the stimulus factor is fixed is also considered.     

When less is more and when more is more: The mediating roles of capacity and speed in brain-behavior efficiency

An enduring enterprise of experimental psychology has been to account for individual differences in human performance. Recent advances in neuroimaging have permitted testing of hypotheses regarding the neural bases of individual differences but this burgeoning literature has been characterized by inconsistent results. We argue that careful design and analysis of neuroimaging studies is required to separate individual differences in processing capacity from individual differences in processing speed to account for these differences in the literature. We utilized task designs which permitted separation of processing capacity influences on brain-behavior relationships from those related to processing speed. In one set of studies, participants performed verbal delayed-recognition tasks during blocked and event-related fMRI scanning. The results indicated that those participants with greater working memory (WM) capacity showed greater prefrontal cortical activity, strategically capitalized on the additional processing time available in the delay period, and evinced faster WM-retrieval rates than low-capacity participants. In another study, participants performed a digit-symbol substitution task (DSST) designed to minimize WM storage capacity requirements and maximize processing speed requirements during fMRI scanning. In some prefrontal cortical (PFC) brain regions, participants with faster processing speed showed less PFC activity than slower performers while in other PFC and parietal regions they showed greater activity. Regional-causality analysis indicated that PFC exerted more influence over other brain regions for slower than for faster individuals. These results support a model of neural efficiency in which individuals differ in the extent of direct processing links between neural nodes. One benefit of direct processing links may be a surplus of resources that maximize available capacity permitting fast and accurate performance.     

Univariate repeated measures techniques applied to multivariate data

Repeated measures designs in psychology have traditionally been analyzed by the univariate mixed model approach, in which the repeated measures effect is tested against an error term based on the subject by treatment interaction. This paper considers the extension of this analysis to designs in which the individual repeated measures are multivariate. Sufficient conditions for a valid multivariate mixed model analysis are given, and a test is described to determine whether or not given data satisfy these conditions.     

Justice and the human alarm system: The impact of exclamation points and flashing lights on the justice judgment process

Extending theory within the justice domain and work on the human alarm system, the current paper argues that the process by which justice judgments are formed may be influenced reliably by the activation of psychological systems that people use to detect and handle alarming situations. Building on this analysis, it is further proposed that if this line of reasoning is true then presenting alarm-related stimuli, such as exclamation points and flashing lights, to people should lead to more extreme judgments about subsequent justice-related events than not presenting these alarming stimuli. Findings collected using different experimental paradigms provide evidence supporting these predictions both inside and outside the psychology lab. Implications for the social psychology of justice and the human alarm system literature are discussed.     

Using confidence intervals for graphically based data interpretation.

As a potential alternative to standard null hypothesis significance testing, we describe methods for graphical presentation of data--particularly condition means and their corresponding confidence intervals--for a wide range of factorial designs used in experimental psychology. We describe and illustrate confidence intervals specifically appropriate for between-subject versus within-subject factors. For designs involving more than two levels of a factor, we describe the use of contrasts for graphical illustration of theoretically meaningful components of main effects and interactions. These graphical techniques lend themselves to a natural and straightforward assessment of statistical power.     

On the use of wireless response systems in experimental psychology: Implications for the behavioral researcher

A method of data collection is presented that unites the efficiency of mass testing with the ease of instant electronic data collection that is typical of computer-based experiments run on individual participants. A wireless response system (WRS), originally designed as a teaching tool, is used to replicate three classic and robust effects from the memory literature (effects of false memory, levels of processing, and word frequency). It is shown that for these types of experimental designs, data can be collected more efficiently (in both time and effort) with the WRS method than through traditional mass- and individual-testing methods alone. The advantages and limitations of WRSs for use in mass electronic data collection are discussed.