Robust tests for multivariate factorial designs under heteroscedasticity

The question of how to analyze several multivariate normal mean vectors when normality and covariance homogeneity assumptions are violated is considered in this article. For the two-way MANOVA layout, we address this problem adapting results presented by Brunner, Dette, and Munk (BDM; 1997) and Vallejo and Ato (modified Brown-Forsythe [MBF]; 2006) in the context of univariate factorial and split-plot designs and a multivariate version of the linear model (MLM) to accommodate heterogeneous data. Furthermore, we compare these procedures with the Welch-James (WJ) approximate degrees of freedom multivariate statistics based on ordinary least squares via Monte Carlo simulation. Our numerical studies show that of the methods evaluated, only the modified versions of the BDM and MBF procedures were robust to violations of underlying assumptions. The MLM approach was only occasionally liberal, and then by only a small amount, whereas the WJ procedure was often liberal if the interactive effects were involved in the design, particularly when the number of dependent variables increased and total sample size was small. On the other hand, it was also found that the MLM procedure was uniformly more powerful than its most direct competitors. The overall success rate was 22.4% for the BDM, 36.3% for the MBF, and 45.0% for the MLM.     

Robust tests of equivalence for k independent groups

A common question of interest to researchers in psychology is the equivalence of two or more groups. Failure to reject the null hypothesis of traditional hypothesis tests such as the ANOVA F‐test (i.e., H₀: μ₁ = … = μ_k) does not imply the equivalence of the population means. Researchers interested in determining the equivalence of k independent groups should apply a one‐way test of equivalence (e.g., Wellek, 2003). The goals of this study were to investigate the robustness of the one‐way Wellek test of equivalence to violations of homogeneity of variance assumption, and compare the Type I error rates and power of the Wellek test with a heteroscedastic version which was based on the logic of the one‐way Welch (1951) F‐test. The results indicate that the proposed Wellek–Welch test was insensitive to violations of the homogeneity of variance assumption, whereas the original Wellek test was not appropriate when the population variances were not equal.     

Some approximate tests for repeated measurement designs

Four approximate tests are considered for repeated measurement designs in which observations are multivariate normal with arbitrary covariance matrices. In these tests traditional within-subject mean square ratios are compared with critical values derived fromF distributions with adjusted degrees of freedom. Two of them—the approximate and the improved general approximate (IGA) tests—behave adequately in terms of Type I error. Generally, the IGA test functions better than the approximate test, however the latter involves less computations. In regards to power, the IGA test may compete with one multivariate procedure when the assumptions of the latter are tenable.The author wishes to thank Garrett K. Mandeville for his careful reading of the final version of the paper.     

Rank‐based permutation approaches for non‐parametric factorial designs

Inference methods for null hypotheses formulated in terms of distribution functions in general non‐parametric factorial designs are studied. The methods can be applied to continuous, ordinal or even ordered categorical data in a unified way, and are based only on ranks. In this set‐up Wald‐type statistics and ANOVA‐type statistics are the current state of the art. The first method is asymptotically exact but a rather liberal statistical testing procedure for small to moderate sample size, while the latter is only an approximation which does not possess the correct asymptotic α level under the null. To bridge these gaps, a novel permutation approach is proposed which can be seen as a flexible generalization of the Kruskal–Wallis test to all kinds of factorial designs with independent observations. It is proven that the permutation principle is asymptotically correct while keeping its finite exactness property when data are exchangeable. The results of extensive simulation studies foster these theoretical findings. A real data set exemplifies its applicability.     

On the limitations of fixed‐step‐size adaptive methods with response confidence

The family of (non‐parametric, fixed‐step‐size) adaptive methods, also known as ‘up–down’ or ‘staircase’ methods, has been used extensively in psychophysical studies for threshold estimation. Extensions of adaptive methods to non‐binary responses have also been proposed. An example is the three‐category weighted up–down (WUD) method (Kaernbach, 2001) and its four‐category extension (Klein, 2001). Such an extension, however, is somewhat restricted, and in this paper we discuss its limitations. To facilitate the discussion, we characterize the extension of WUD by an algorithm that incorporates response confidence into a family of adaptive methods. This algorithm can also be applied to two other adaptive methods, namely Derman's up–down method and the biased‐coin design, which are suitable for estimating any threshold quantiles. We then discuss via simulations of the above three methods the limitations of the algorithm. To illustrate, we conduct a small scale of experiment using the extended WUD under different response confidence formats to evaluate the consistency of threshold estimation.     

Recommendations for reporting multiple‐baseline designs across participants

Two recommendations are offered for reporting multiple‐baseline designs across participants. These recommendations will better enable readers to (a) distinguish concurrent from nonconcurrent multiple‐baseline designs and (b) determine the temporal order in which sessions were conducted in concurrent multiple‐baseline designs. Copyright © 2005 John Wiley & Sons, Ltd.     

Beta‐binomial ANOVA for multivariate randomized response data

There is much empirical evidence that randomized response methods improve the cooperation of the respondents when asking sensitive questions. The traditional methods for analysing randomized response data are restricted to univariate data and only allow inferences at the group level due to the randomized response sampling design. Here, a novel beta‐binomial model is proposed for analysing multivariate individual count data observed via a randomized response sampling design. This new model allows for the estimation of individual response probabilities (response rates) for multivariate randomized response data utilizing an empirical Bayes approach. A common beta prior specifies that individuals in a group are tied together and the beta prior parameters are allowed to be cluster‐dependent. A Bayes factor is proposed to test for group differences in response rates. An analysis of a cheating study, where 10 items measure cheating or academic dishonesty, is used to illustrate application of the proposed model.     

Leniency for out‐group offenders

The present research investigates leniency for out‐group offenders and differentiates it from the black sheep effect. The authors assume that leniency for out‐group offenders can be used by in‐group members to protect their group's image by displaying that they are not prejudiced. Thus, leniency should disappear when in‐group members have otherwise shown that they are not prejudiced (i.e., moral credentials). In two experiments, offenders' group membership and participants' opportunity to establish moral credentials were manipulated. Results showed that out‐group offenders received the lowest punishment severity ratings (Studies 1 and 2). However, this leniency effect vanished when participants had established moral credentials by either endorsing the participation of out‐group members in lobby groups (Study 1) or writing about a positive experience with an out‐group member (Study 2). These findings suggest that lenient punishments for out‐group offenders may sometimes reflect a relatively easy strategy to display the in‐group as being unprejudiced. Copyright © 2012 John Wiley & Sons, Ltd.     

Effects of in‐group and out‐group ethnicity on children's attitudes towards members of the in‐group and out‐group

This study tested predictions drawn from social identity development theory (SIDT) (Nesdale, 1999a) concerning the development of young children's ethnic attitudes. 5‐7‐ and 9‐year‐old children (N = 159) participated in a minimal group study in which they were assigned to a team which had higher drawing ability than a competitor team. The team members were revealed to be of the same (Anglo‐Australian) vs. different (Pacific Islander) ethnicity (in‐group ethnicity). The ethnicity of the competitor team was varied in the same way. The children subsequently rated their liking for, and similarity to, the in‐group and the out‐group, and the extent to which they wished to change groups. Children liked in‐group members more than out‐group members. Liking for in‐group members was unaffected by the ethnic composition of the groups, but liking for outgroup members was reduced when the ethnic composition of the out‐group differed from that of the in‐group. Children felt most similar to same‐group, same‐ethnicity members and least similar to different ethnicity out‐group members. The desire to change teams increased with age but there was no intention to align with same ethnicity individuals. The extent to which the findings provide support for SIDT is discussed.     

Properties of bootstrap tests for N‐of‐1 studies

N‐of‐1 study designs involve the collection and analysis of repeated measures data from an individual not using an intervention and using an intervention. This study explores the use of semi‐parametric and parametric bootstrap tests in the analysis of N‐of‐1 studies under a single time series framework in the presence of autocorrelation. When the Type I error rates of bootstrap tests are compared to Wald tests, our results show that the bootstrap tests have more desirable properties. We compare the results for normally distributed errors with those for contaminated normally distributed errors and find that, except when there is relatively large autocorrelation, there is little difference between the power of the parametric and semi‐parametric bootstrap tests. We also experiment with two intervention designs: ABAB and AB, and show the ABAB design has more power. The results provide guidelines for designing N‐of‐1 studies, in the sense of how many observations and how many intervention changes are needed to achieve a certain level of power and which test should be performed.     

Effect of non-normality on test statistics for one-way independent groups designs

The data obtained from one‐way independent groups designs is typically non‐normal in form and rarely equally variable across treatment populations (i.e. population variances are heterogeneous). Consequently, the classical test statistic that is used to assess statistical significance (i.e. the analysis of variance F test) typically provides invalid results (e.g. too many Type I errors, reduced power). For this reason, there has been considerable interest in finding a test statistic that is appropriate under conditions of non‐normality and variance heterogeneity. Previously recommended procedures for analysing such data include the James test, the Welch test applied either to the usual least squares estimators of central tendency and variability, or the Welch test with robust estimators (i.e. trimmed means and Winsorized variances). A new statistic proposed by Krishnamoorthy, Lu, and Mathew, intended to deal with heterogeneous variances, though not non‐normality, uses a parametric bootstrap procedure. In their investigation of the parametric bootstrap test, the authors examined its operating characteristics under limited conditions and did not compare it to the Welch test based on robust estimators. Thus, we investigated how the parametric bootstrap procedure and a modified parametric bootstrap procedure based on trimmed means perform relative to previously recommended procedures when data are non‐normal and heterogeneous. The results indicated that the tests based on trimmed means offer the best Type I error control and power when variances are unequal and at least some of the distribution shapes are non‐normal.     

Robust techniques for testing heterogeneity of variance effects in factorial designs

Several ways of using the traditional analysis of variance to test heterogeneity of spread in factorial designs with equal or unequaln are compared using both theoretical and Monte Carlo results. Two types of spread variables, (1) the jackknife pseudovalues ofs ² and (2) the absolute deviations from the cell median, are shown to be robust and relatively powerful. These variables seem to be generally superior to the Z-variance and Box-Scheffé procedures.This research was sponsored by Public Health Service Training Grant MH-08258 from the National Institute of Mental Health. The author thanks Mark I. Appelbaum, Elliot M. Cramer, and Scott E. Maxwell for their helpful criticisms of this paper. An earlier version of this work was presented at the Annual Meeting of the Psychometric Society, Murray Hill, New Jersey, April, 1976.     

Breaking down a barrier: increasing perceived out‐group knowledge reduces negative expectancies about intergroup interaction

Although intergroup contact is an effective way of reducing prejudice, negative expectancies about interacting with out‐group members often create a barrier to intergroup contact. The current study investigated cognitive appraisals by which negative expectancies may arise. Specifically, we examined whether increasing Anglo Australians' appraisals of their knowledge about Muslims would reduce their negative expectancies about an (ostensible) upcoming interaction with a Muslim Australian. Participants (89 Anglo Australians) completed a test that provided positive feedback either on their knowledge about Muslims or on their general knowledge (control). As predicted, Anglo Australians who received positive feedback on their knowledge about Muslims had a lower threat appraisal and expected to feel less anxious during the intergroup interaction compared with those who were in the control condition. This provides support for the precursory role out‐group knowledge may have as a resource that is appraised upon the prospect of an intergroup interaction.     

Two experimental tests of trust in in‐group strangers: The moderating role of common knowledge of group membership

The role that shared group membership plays in decisions to trust others is now well established within social psychology. A close reading of this literature, however, shows that this process is often moderated by other variables. Currently, we examined one potential moderator of this process. In particular, we evaluated the role that common knowledge of a shared social group membership between self and a to‐be‐trusted stranger provides as a basis for trusting this stranger. This common knowledge emerges when the truster knows the group membership of the to‐be‐trusted other, and believes that this other also knows the group membership of the truster. In two experiments, using pre‐existing and minimal groups, we show that people are more likely to trust an in‐group member over an out‐group member under conditions of common group‐membership knowledge rather than private group‐membership knowledge (i.e. other does not know truster's group), even when they could choose not to trust anyone. The manner in which these data add to current understandings of group‐based trust in strangers is discussed. Copyright © 2011 John Wiley & Sons, Ltd.     

Randomization tests for ABAB designs: comparing-data-division- specific and common distributions

Monte Carlo simulations were used to generate data for ABAB designs of different lengths. The points of change in phase are randomly determined before gathering behaviour measurements, which allows the use of a randomization test as an analytic technique. Data simulation and analysis can be based either on data-division-specific or on common distributions. Following one method or another affects the results obtained after the randomization test has been applied. Therefore, the goal of the study was to examine these effects in more detail. The discrepancies in these approaches are obvious when data with zero treatment effect are considered and such approaches have implications for statistical power studies. Data-division-specific distributions provide more detailed information about the performance of the statistical technique.     

Hypothesis tests for population heterogeneity in meta‐analysis

Choice of the appropriate model in meta‐analysis is often treated as an empirical question which is answered by examining the amount of variability in the effect sizes. When all of the observed variability in the effect sizes can be accounted for based on sampling error alone, a set of effect sizes is said to be homogeneous and a fixed‐effects model is typically adopted. Whether a set of effect sizes is homogeneous or not is usually tested with the so‐called Q test. In this paper, a variety of alternative homogeneity tests – the likelihood ratio, Wald and score tests – are compared with the Q test in terms of their Type I error rate and power for four different effect size measures. Monte Carlo simulations show that the Q test kept the tightest control of the Type I error rate, although the results emphasize the importance of large sample sizes within the set of studies. The results also suggest under what conditions the power of the tests can be considered adequate.     

Correction for faking in self‐report personality tests

Faking is a common problem in testing with self‐report personality tests, especially in high‐stakes situations. A possible way to correct for it is statistical control on the basis of social desirability scales. Two such scales were developed and applied in the present paper. It was stressed that the statistical models of faking need to be adapted to different properties of the personality scales, since such scales correlate with faking to different extents. In four empirical studies of self‐report personality tests, correction for faking was investigated. One of the studies was experimental, and asked participants to fake or to be honest. In the other studies, job or school applicants were investigated. It was found that the approach to correct for effects of faking in self‐report personality tests advocated in the paper removed a large share of the effects, about 90%. It was found in one study that faking varied as a function of degree of how important the consequences of test results could be expected to be, more high‐stakes situations being associated with more faking. The latter finding is incompatible with the claim that social desirability scales measure a general personality trait. It is concluded that faking can be measured and that correction for faking, based on such measures, can be expected to remove about 90% of its effects.     

In‐group representation and social value affect the use of in‐group identification for maintaining and enhancing self‐evaluation

Individuals with independent self‐construals identify with their in‐group in order to buffer threats to self‐esteem to a greater degree than do those with interdependent self‐construals. This study focused on the effect of the in‐group's social value and representation on this identification process among individuals who reported various real‐world threats to their self‐esteem. We conducted a questionnaire survey for university students and we examined how self‐construal, the in‐group social value, and the in‐group representation either moderate or mediate the relationship between threat to self‐esteem and identification with their academic departments. The results of multiple‐group analyses of structural equation modelling showed that independents who reported more threats to their self‐esteem identified with the in‐group when it was high in social value. In contrast, they did not exhibit in‐group identification when it was low in social value. Interdependents showed less in‐group identification regardless of the in‐group's social value. The representation of the in‐group as a common identity group mediated the relationship between threat to self‐esteem and in‐group identification for independents, whereas the representation of a common bond group played the same role for interdependents. These findings suggest the importance of considering the role of self‐construal and the in‐group social value, along with the representation of in‐group, in understanding in‐group identification in situations of threat to self‐esteem.     

Estimation of maximal reliability for multiple‐component instruments in multilevel designs

A procedure for point and interval estimation of maximal reliability of multiple‐component measuring instruments in multi‐level settings is outlined. The approach is applicable to hierarchical designs in which individuals are nested within higher‐order units and exhibit possibly related performance on components of a given homogeneous scale. The method is developed within the framework of multi‐level factor analysis. The proposed procedure is illustrated with an empirical example.