Fractional Design Wizard: A computer program for cost-effective experimental research design

The computer program Fractional Design Wizard creates fractional factorial designs that are cost-effective and especially useful for discarding irrelevant factors from a large number of possible candidates. The program is intended for researchers who are relativelynew to the field of fractional design and who want to acquaint themselves with the use of fractions for the reduction of large experimental designs. Fractional designs allow estimation of main effects, and sometimes two-way interactions, without one’s having to examine all treatment conditions. The program needs Microsoft Windows 95 or better and 32 MB of memory. In a step-by-step fashion, the user can specify the required properties of the fractional design. When there are more valid designs, the user can generate these successively If necessary the user can go back to diminish the requirements. The output can be copied, printed, and saved. The program generates all the information that is needed for the use and interpretation of fractional designs. A help file explains the use of the program and also the purpose, the analysis, and the interpretation of fractional designs. The program, which is written in Object Pascal, is available as freeware on www.fss.uu.nl/ms/hl/fracdes.htm.     

Fractional factorial designs for legal psychology

Researchers considering novel or exploratory psycholegal research are often able to easily generate a sizable list of independent variables (IVs) that might influence a measure of interest. Where the research question is novel and the literature is not developed, however, choosing from among a long list of potential variables those worthy of empirical investigation often presents a formidable task. Many researchers may feel compelled by legal psychology's heavy reliance on full-factorial designs to narrow the IVs under investigation to two or three in order to avoid an expensive and unwieldy design involving numerous high-order interactions. This article suggests that fractional factorial designs provide a reasonable alternative to full-factorial designs in such circumstances because they allow the psycholegal researcher to examine the main effects of a large number of factors while disregarding high-order interactions. An introduction to the logic of fractional factorial designs is provided and several examples from the social sciences are presented.     

Modeling external events in the three-level analysis of multiple-baseline across-participants designs: A simulation study

In this study, we focus on a three-level meta-analysis for combining data from studies using multiple-baseline across-participants designs. A complicating factor in such designs is that results might be biased if the dependent variable is affected by not explicitly modeled external events, such as the illness of a teacher, an exciting class activity, or the presence of a foreign observer. In multiple-baseline designs, external effects can become apparent if they simultaneously have an effect on the outcome score(s) of the participants within a study. This study presents a method for adjusting the three-level model to external events and evaluates the appropriateness of the modified model. Therefore, we use a simulation study, and we illustrate the new approach with real data sets. The results indicate that ignoring an external event effect results in biased estimates of the treatment effects, especially when there is only a small number of studies and measurement occasions involved. The mean squared error, as well as the standard error and coverage proportion of the effect estimates, is improved with the modified model. Moreover, the adjusted model results in less biased variance estimates. If there is no external event effect, we find no differences in results between the modified and unmodified models.     

Using analysis of covariance (ANCOVA) with fallible covariates

Analysis of covariance (ANCOVA) is used widely in psychological research implementing nonexperimental designs. However, when covariates are fallible (i.e., measured with error), which is the norm, researchers must choose from among 3 inadequate courses of action: (a) know that the assumption that covariates are perfectly reliable is violated but use ANCOVA anyway (and, most likely, report misleading results); (b) attempt to employ 1 of several measurement error models with the understanding that no research has examined their relative performance and with the added practical difficulty that several of these models are not available in commonly used statistical software; or (c) not use ANCOVA at all. First, we discuss analytic evidence to explain why using ANCOVA with fallible covariates produces bias and a systematic inflation of Type I error rates that may lead to the incorrect conclusion that treatment effects exist. Second, to provide a solution for this problem, we conduct 2 Monte Carlo studies to compare 4 existing approaches for adjusting treatment effects in the presence of covariate measurement error: errors-in-variables (EIV; Warren, White, & Fuller, 1974), Lord's (1960) method, Raaijmakers and Pieters's (1987) method (R&P), and structural equation modeling methods proposed by S?rbom (1978) and Hayduk (1996). Results show that EIV models are superior in terms of parameter accuracy, statistical power, and keeping Type I error close to the nominal value. Finally, we offer a program written in R that performs all needed computations for implementing EIV models so that ANCOVA can be used to obtain accurate results even when covariates are measured with error.     

计划缺失设计——通过有意缺失让研究更高效

缺失值是社会科学研究中非常普遍的现象。全息极大似然估计和多重插补是目前处理缺失值最有效的方法。计划缺失设计利用特殊的实验设计有意产生缺失值, 再用现代的缺失值处理方法来完成统计分析, 获得无偏的统计结果。计划缺失设计可用于横断面调查减少(或增加)问卷长度和纵向调查减少测量次数, 也可用于提高测量有效性。常用的计划缺失设计有三式设计和两种方法测量。     

Establishing a causal chain: why experiments are often more effective than mediational analyses in examining psychological processes

The authors propose that experiments that utilize mediational analyses as suggested by R. M. Baron and D. A. Kenny (1986) are overused and sometimes improperly held up as necessary for a good social psychological paper. The authors argue that when it is easy to manipulate and measure a proposed psychological process that a series of experiments that demonstrates the proposed causal chain is superior. They further argue that when it is easy to manipulate a proposed psychological process but difficult to measure it that designs that examine underlying process by utilizing moderation can be effective. It is only when measurement of a proposed psychological process is easy and manipulation of it is difficult that designs that rely on mediational analyses should be preferred, and even in these situations careful consideration should be given to the limiting factors of such designs.     

Development of assertive responses: Clinical, measurement and research considerations

The development of assertive responses in unassertive individuals is surveyed in terms of clinical application, measurement techniques and experimental study. Case reports indicate that assertive training is an effective technique for a large variety of disorders as it enables patients to overcome deficits in inter-personal functioning. Some of the effective elements in assertive training have been identified by using analogue research designs. In most of these investigations the major focus has been on expressing hostile feelings and thoughts. However, more efforts at instigating positive responses (clinically and experimentally) are warranted. In addition, there is a need to assess the effects on ‘clinically’ unassertive subjects of techniques in which the transfer of training is programmed rather than expected.     

Multilevel factorial experiments for developing behavioral interventions: power, sample size, and resource considerations

Factorial experimental designs have many potential advantages for behavioral scientists. For example, such designs may be useful in building more potent interventions by helping investigators to screen several candidate intervention components simultaneously and to decide which are likely to offer greater benefit before evaluating the intervention as a whole. However, sample size and power considerations may challenge investigators attempting to apply such designs, especially when the population of interest is multilevel (e.g., when students are nested within schools, or when employees are nested within organizations). In this article, we examine the feasibility of factorial experimental designs with multiple factors in a multilevel, clustered setting (i.e., of multilevel, multifactor experiments). We conduct Monte Carlo simulations to demonstrate how design elements-such as the number of clusters, the number of lower-level units, and the intraclass correlation-affect power. Our results suggest that multilevel, multifactor experiments are feasible for factor-screening purposes because of the economical properties of complete and fractional factorial experimental designs. We also discuss resources for sample size planning and power estimation for multilevel factorial experiments. These results are discussed from a resource management perspective, in which the goal is to choose a design that maximizes the scientific benefit using the resources available for an investigation.     

Methodological Challenges in Causal Research on Racial and Ethnic Patterns of Cognitive Trajectories: Measurement, Selection, and Bias

Research focused on understanding how and why cognitive trajectories differ across racial and ethnic groups can be compromised by several possible methodological challenges. These difficulties are especially relevant in research on racial and ethnic disparities and neuropsychological outcomes because of the particular influence of selection and measurement in these contexts. In this article, we review the counterfactual framework for thinking about causal effects versus statistical associations. We emphasize that causal inferences are key to predicting the likely consequences of possible interventions, for example in clinical settings. We summarize a number of common biases that can obscure causal relationships, including confounding, measurement ceilings/floors, baseline adjustment bias, practice or retest effects, differential measurement error, conditioning on common effects in direct and indirect effects decompositions, and differential survival. For each, we describe how to recognize when such biases may be relevant and some possible analytic or design approaches to remediating these biases.     

Design efficiency for imbalanced multilevel data

The importance of accurate estimation and of powerful statistical tests is widely recognized but has rarely been acknowledged in practice in the social and behavioral sciences. This is especially true for estimation and testing when one is dealing with multilevel designs, not least because approximating accuracy and power is more complex due to having multiple variances and research units at several levels. The complexity further increases for imbalanced designs, often necessitating simulation studies that perform accuracy and power calculations. However, we show, using such simulation studies, that the distortion of balance can be ignored in most cases, making efficiency studies simpler and the use of existing software valid. An exception is suggested for imbalanced data from a large majority of small groups. Furthermore, an empirical sampling distribution of variance parameters may show substantial skewness and kurtosis, depending on the number of groups and, for the random slope, depending also on the group’s size, adding another caveat to the recommendation to ignore imbalance.     

How Do Propensity Score Methods Measure Up in the Presence of Measurement Error? A Monte Carlo Study

Considering that the absence of measurement error in research is a rare phenomenon and its effects can be dramatic, we examine the impact of measurement error on propensity score (PS) analysis used to minimize selection bias in behavioral and social observational studies. A Monte Carlo study was conducted to explore the effects of measurement error on the treatment effect and balance estimates in PS analysis across seven different PS conditioning methods. In general, the results indicate that even low levels of measurement error in the covariates lead to substantial bias in estimates of treatment effects and concomitant reduction in confidence interval coverage across all methods of conditioning on the PS.     

Balanced incomplete blocks to control individual differences

Balanced incomplete block designs are used when there are reasons that preclude the use of repeated measurements designs, particularly when there are at least two factors, each with several levels. However, these designs are usually not used when interaction effects are of interest, because they involve the confounding of the interactions with subject or group differences. This paper describes special computational procedures that can be employed to obtain partial, but unconfounded, information about interaction effects.     

The Impact of Inappropriate Modeling of Cross-Classified Data Structures

Cross-classified random effects modeling (CCREM) is used to model multilevel data from nonhierarchical contexts. These models are widely discussed but infrequently used in social science research. Because little research exists assessing when it is necessary to use CCREM, 2 studies were conducted. A real data set with a cross-classified structure was analyzed by comparing parameter estimates when ignoring versus modeling the cross-classified data structure. A follow-up simulation study investigated potential factors affecting the need to use CCREM. Results indicated that when the structure is ignored, fixed-effect estimates were unaffected, but standard error estimates associated with the variables modeled incorrectly were biased. Estimates of the variance components also displayed bias, which was related to several study factors.     

A 2 × 2 taxonomy of multilevel latent contextual models: accuracy-bias trade-offs in full and partial error correction models

In multilevel modeling, group-level variables (L2) for assessing contextual effects are frequently generated by aggregating variables from a lower level (L1). A major problem of contextual analyses in the social sciences is that there is no error-free measurement of constructs. In the present article, 2 types of error occurring in multilevel data when estimating contextual effects are distinguished: unreliability that is due to measurement error and unreliability that is due to sampling error. The fact that studies may or may not correct for these 2 types of error can be translated into a 2 × 2 taxonomy of multilevel latent contextual models comprising 4 approaches: an uncorrected approach, partial correction approaches correcting for either measurement or sampling error (but not both), and a full correction approach that adjusts for both sources of error. It is shown mathematically and with simulated data that the uncorrected and partial correction approaches can result in substantially biased estimates of contextual effects, depending on the number of L1 individuals per group, the number of groups, the intraclass correlation, the number of indicators, and the size of the factor loadings. However, the simulation study also shows that partial correction approaches can outperform full correction approaches when the data provide only limited information in terms of the L2 construct (i.e., small number of groups, low intraclass correlation). A real-data application from educational psychology is used to illustrate the different approaches.     

When extraordinary injustice leads to ordinary response: How perpetrator power and size of an injustice event affect bystander efficacy and collective action

Although bystanders can play an integral role in the process of social change, relatively few studies have examined the factors that influence bystander collective action. The present research explores the effect of perpetrator power on bystander efficacy and collective action, as well as the moderating role of impact of the injustice event. Across two experiments, bystanders perceived that collective action would be less effective and were less willing to engage in collective action when a high‐power perpetrator engaged in injustice, compared with a low‐power perpetrator. These effects were moderated by impact of the injustice event, such that the effects of power were especially present under conditions of large impact (many victims), compared with small impact (fewer victims). Whereas the effect of the interaction of perpetrator power and impact on bystander efficacy was explained by perceptions of normativity of the injustice event, the effect of the interaction on bystander collective action was explained by bystander efficacy. Implications for bystander collective action and social change are discussed. Copyright © 2014 John Wiley & Sons, Ltd.     

Structural equation modeling of multitrait-multimethod data: different models for different types of methods

The question as to which structural equation model should be selected when multitrait-multimethod (MTMM) data are analyzed is of interest to many researchers. In the past, attempts to find a well-fitting model have often been data-driven and highly arbitrary. In the present article, the authors argue that the measurement design (type of methods used) should guide the choice of the statistical model to analyze the data. In this respect, the authors distinguish between (a) interchangeable methods, (b) structurally different methods, and (c) the combination of both kinds of methods. The authors present an appropriate model for each type of method. All models allow separating measurement error from trait influences and trait-specific method effects. With respect to interchangeable methods, a multilevel confirmatory factor model is presented. For structurally different methods, the correlated trait-correlated (method-1) model is recommended. Finally, the authors demonstrate how to appropriately analyze data from MTMM designs that simultaneously use interchangeable and structurally different methods. All models are applied to empirical data to illustrate their proper use. Some implications and guidelines for modeling MTMM data are discussed.     

年幼儿童的尺寸匹配误差现象

近十年来,尺寸匹配误差作为年幼儿童生活中的一种常见的尺寸误用现象逐渐受到了一些研究者的关注。现有研究主要集中在尺寸匹配误差的特点、出现频率和产生原因等方面。抑制控制的失败、神经通路的协调失败、“计划-控制”模型和“感觉-行动”模型、功能性推理偏好以及身体意识发展的不成熟等观点能够对某些类型的尺寸匹配误差现象做出解释。未来的研究应从进一步深入探讨尺寸匹配误差与假装的区别、完善研究方法以及跨文化研究的开展几方面进行。     

内隐关系评估程序(IRAP)：测量原理及应用

内隐关系评估程序(Implicit Relational Assessment Procedure, 简称IRAP)是基于关系结构理论直接测量社会认知、信念或态度的新内隐测量方法, 具备一定的可靠性和有效性, 并与其它相关测量方法的适用性存在一定差异。不同的理论模型为IRAP的不同效应提供了解释。IRAP最初应用于临床诊断性研究, 新近已扩展到自我、社会认知、群体和态度等研究领域。进一步验证不同形式IRAP的信效度、探究IRAP的心理机制及产生的心理效应、在不同领域发挥IRAP的方法优势等将是未来研究的重要方向。     

Distinguishing the Ratio Bias from Unsystematic Error: Situation and Individual‐difference Effects

People often perceive the occurrence of events to be less likely when the likelihood of the event is expressed in ratios consisting of smaller numbers versus larger numbers, an effect known as the ratio bias. This work presents a theoretical framework for the conditions that need to be met for the ratio bias to occur. In doing so, we contrast effects on the ratio bias to those on unsystematic error, which have often been confounded in previous research. We find that the ratio bias is weaker (1) when both sets of numbers are relatively large than when both sets of numbers are relatively small; (2) for scenarios involving lottery tickets than for scenarios involving drawing balls from a bin; and (3) when a physical display depicting the numbers is provided to participants. Each of these factors reduced the ratio bias without reducing unsystematic error. Additionally, we show that unsystematic error is lower among people who (1) reason on the basis of proportions rather than on the basis of the numerator and denominator individually; (2) score higher on the rational scale of the Rational–Experiential Inventory; and (3) are of higher numeracy. We use these results to distinguish causes of error generally from those on the ratio bias specifically and discuss the implications for our understanding of when the ratio bias occurs. Copyright © 2018 John Wiley & Sons, Ltd.