Robust tests for multivariate factorial designs under heteroscedasticity

The question of how to analyze several multivariate normal mean vectors when normality and covariance homogeneity assumptions are violated is considered in this article. For the two-way MANOVA layout, we address this problem adapting results presented by Brunner, Dette, and Munk (BDM; 1997) and Vallejo and Ato (modified Brown-Forsythe [MBF]; 2006) in the context of univariate factorial and split-plot designs and a multivariate version of the linear model (MLM) to accommodate heterogeneous data. Furthermore, we compare these procedures with the Welch-James (WJ) approximate degrees of freedom multivariate statistics based on ordinary least squares via Monte Carlo simulation. Our numerical studies show that of the methods evaluated, only the modified versions of the BDM and MBF procedures were robust to violations of underlying assumptions. The MLM approach was only occasionally liberal, and then by only a small amount, whereas the WJ procedure was often liberal if the interactive effects were involved in the design, particularly when the number of dependent variables increased and total sample size was small. On the other hand, it was also found that the MLM procedure was uniformly more powerful than its most direct competitors. The overall success rate was 22.4% for the BDM, 36.3% for the MBF, and 45.0% for the MLM.     

Robust tests of equivalence for k independent groups

A common question of interest to researchers in psychology is the equivalence of two or more groups. Failure to reject the null hypothesis of traditional hypothesis tests such as the ANOVA F‐test (i.e., H₀: μ₁ = … = μ_k) does not imply the equivalence of the population means. Researchers interested in determining the equivalence of k independent groups should apply a one‐way test of equivalence (e.g., Wellek, 2003). The goals of this study were to investigate the robustness of the one‐way Wellek test of equivalence to violations of homogeneity of variance assumption, and compare the Type I error rates and power of the Wellek test with a heteroscedastic version which was based on the logic of the one‐way Welch (1951) F‐test. The results indicate that the proposed Wellek–Welch test was insensitive to violations of the homogeneity of variance assumption, whereas the original Wellek test was not appropriate when the population variances were not equal.     

On the limitations of fixed‐step‐size adaptive methods with response confidence

The family of (non‐parametric, fixed‐step‐size) adaptive methods, also known as ‘up–down’ or ‘staircase’ methods, has been used extensively in psychophysical studies for threshold estimation. Extensions of adaptive methods to non‐binary responses have also been proposed. An example is the three‐category weighted up–down (WUD) method (Kaernbach, 2001) and its four‐category extension (Klein, 2001). Such an extension, however, is somewhat restricted, and in this paper we discuss its limitations. To facilitate the discussion, we characterize the extension of WUD by an algorithm that incorporates response confidence into a family of adaptive methods. This algorithm can also be applied to two other adaptive methods, namely Derman's up–down method and the biased‐coin design, which are suitable for estimating any threshold quantiles. We then discuss via simulations of the above three methods the limitations of the algorithm. To illustrate, we conduct a small scale of experiment using the extended WUD under different response confidence formats to evaluate the consistency of threshold estimation.     

Some approximate tests for repeated measurement designs

Four approximate tests are considered for repeated measurement designs in which observations are multivariate normal with arbitrary covariance matrices. In these tests traditional within-subject mean square ratios are compared with critical values derived fromF distributions with adjusted degrees of freedom. Two of them—the approximate and the improved general approximate (IGA) tests—behave adequately in terms of Type I error. Generally, the IGA test functions better than the approximate test, however the latter involves less computations. In regards to power, the IGA test may compete with one multivariate procedure when the assumptions of the latter are tenable.The author wishes to thank Garrett K. Mandeville for his careful reading of the final version of the paper.     

Beta‐binomial ANOVA for multivariate randomized response data

There is much empirical evidence that randomized response methods improve the cooperation of the respondents when asking sensitive questions. The traditional methods for analysing randomized response data are restricted to univariate data and only allow inferences at the group level due to the randomized response sampling design. Here, a novel beta‐binomial model is proposed for analysing multivariate individual count data observed via a randomized response sampling design. This new model allows for the estimation of individual response probabilities (response rates) for multivariate randomized response data utilizing an empirical Bayes approach. A common beta prior specifies that individuals in a group are tied together and the beta prior parameters are allowed to be cluster‐dependent. A Bayes factor is proposed to test for group differences in response rates. An analysis of a cheating study, where 10 items measure cheating or academic dishonesty, is used to illustrate application of the proposed model.     

Effects of in‐group and out‐group ethnicity on children's attitudes towards members of the in‐group and out‐group

This study tested predictions drawn from social identity development theory (SIDT) (Nesdale, 1999a) concerning the development of young children's ethnic attitudes. 5‐7‐ and 9‐year‐old children (N = 159) participated in a minimal group study in which they were assigned to a team which had higher drawing ability than a competitor team. The team members were revealed to be of the same (Anglo‐Australian) vs. different (Pacific Islander) ethnicity (in‐group ethnicity). The ethnicity of the competitor team was varied in the same way. The children subsequently rated their liking for, and similarity to, the in‐group and the out‐group, and the extent to which they wished to change groups. Children liked in‐group members more than out‐group members. Liking for in‐group members was unaffected by the ethnic composition of the groups, but liking for outgroup members was reduced when the ethnic composition of the out‐group differed from that of the in‐group. Children felt most similar to same‐group, same‐ethnicity members and least similar to different ethnicity out‐group members. The desire to change teams increased with age but there was no intention to align with same ethnicity individuals. The extent to which the findings provide support for SIDT is discussed.     

Robust techniques for testing heterogeneity of variance effects in factorial designs

Several ways of using the traditional analysis of variance to test heterogeneity of spread in factorial designs with equal or unequaln are compared using both theoretical and Monte Carlo results. Two types of spread variables, (1) the jackknife pseudovalues ofs ² and (2) the absolute deviations from the cell median, are shown to be robust and relatively powerful. These variables seem to be generally superior to the Z-variance and Box-Scheffé procedures.This research was sponsored by Public Health Service Training Grant MH-08258 from the National Institute of Mental Health. The author thanks Mark I. Appelbaum, Elliot M. Cramer, and Scott E. Maxwell for their helpful criticisms of this paper. An earlier version of this work was presented at the Annual Meeting of the Psychometric Society, Murray Hill, New Jersey, April, 1976.     

Hypothesis tests for population heterogeneity in meta‐analysis

Choice of the appropriate model in meta‐analysis is often treated as an empirical question which is answered by examining the amount of variability in the effect sizes. When all of the observed variability in the effect sizes can be accounted for based on sampling error alone, a set of effect sizes is said to be homogeneous and a fixed‐effects model is typically adopted. Whether a set of effect sizes is homogeneous or not is usually tested with the so‐called Q test. In this paper, a variety of alternative homogeneity tests – the likelihood ratio, Wald and score tests – are compared with the Q test in terms of their Type I error rate and power for four different effect size measures. Monte Carlo simulations show that the Q test kept the tightest control of the Type I error rate, although the results emphasize the importance of large sample sizes within the set of studies. The results also suggest under what conditions the power of the tests can be considered adequate.     

Effect of non-normality on test statistics for one-way independent groups designs

The data obtained from one‐way independent groups designs is typically non‐normal in form and rarely equally variable across treatment populations (i.e. population variances are heterogeneous). Consequently, the classical test statistic that is used to assess statistical significance (i.e. the analysis of variance F test) typically provides invalid results (e.g. too many Type I errors, reduced power). For this reason, there has been considerable interest in finding a test statistic that is appropriate under conditions of non‐normality and variance heterogeneity. Previously recommended procedures for analysing such data include the James test, the Welch test applied either to the usual least squares estimators of central tendency and variability, or the Welch test with robust estimators (i.e. trimmed means and Winsorized variances). A new statistic proposed by Krishnamoorthy, Lu, and Mathew, intended to deal with heterogeneous variances, though not non‐normality, uses a parametric bootstrap procedure. In their investigation of the parametric bootstrap test, the authors examined its operating characteristics under limited conditions and did not compare it to the Welch test based on robust estimators. Thus, we investigated how the parametric bootstrap procedure and a modified parametric bootstrap procedure based on trimmed means perform relative to previously recommended procedures when data are non‐normal and heterogeneous. The results indicated that the tests based on trimmed means offer the best Type I error control and power when variances are unequal and at least some of the distribution shapes are non‐normal.     

In‐group representation and social value affect the use of in‐group identification for maintaining and enhancing self‐evaluation

Individuals with independent self‐construals identify with their in‐group in order to buffer threats to self‐esteem to a greater degree than do those with interdependent self‐construals. This study focused on the effect of the in‐group's social value and representation on this identification process among individuals who reported various real‐world threats to their self‐esteem. We conducted a questionnaire survey for university students and we examined how self‐construal, the in‐group social value, and the in‐group representation either moderate or mediate the relationship between threat to self‐esteem and identification with their academic departments. The results of multiple‐group analyses of structural equation modelling showed that independents who reported more threats to their self‐esteem identified with the in‐group when it was high in social value. In contrast, they did not exhibit in‐group identification when it was low in social value. Interdependents showed less in‐group identification regardless of the in‐group's social value. The representation of the in‐group as a common identity group mediated the relationship between threat to self‐esteem and in‐group identification for independents, whereas the representation of a common bond group played the same role for interdependents. These findings suggest the importance of considering the role of self‐construal and the in‐group social value, along with the representation of in‐group, in understanding in‐group identification in situations of threat to self‐esteem.     

Effects of group membership,intergroup competition and out‐group ethnicity on children's ratings of in‐group and out‐group similarity and positivity

Based on self‐categorization theory (SCT; Turner, Hogg, Oakes, Reicher, & Wetherell, 1987 ), this study examined the extent to which 7‐ and 10‐year‐old children's perceptions of similarity to, and positivity towards, their in‐group would be increased by factors predicted to enhance the salience of in‐group–out‐group categorizations. In a minimal group study, participants met the in‐group before or after the out‐group (group timing), the out‐group had the same or different ethnicity as the in‐group (out‐group ethnicity), and there was or was not to be a competition between the in‐group and the out‐group (intergroup competition). Ratings of the in‐group similarity were influenced by the out‐group ethnicity, but not by group timing or intergroup competition. Consistent with SCT, participants rated themselves as more similar to the in‐group when the out‐group had different vs. the same ethnicity. SCT's predictions concerning in‐group positivity were not confirmed. Instead, participants rated the in‐group more positively than the out‐group and the in‐group was rated more positively, when participants met the in‐group before rather than after the out‐group. Older compared with younger participants were also more prepared to change groups when the out‐group had different ethnicity. The implications for SCT are discussed.     

Estimation of maximal reliability for multiple‐component instruments in multilevel designs

A procedure for point and interval estimation of maximal reliability of multiple‐component measuring instruments in multi‐level settings is outlined. The approach is applicable to hierarchical designs in which individuals are nested within higher‐order units and exhibit possibly related performance on components of a given homogeneous scale. The method is developed within the framework of multi‐level factor analysis. The proposed procedure is illustrated with an empirical example.     

Estimation of reliability for multiple‐component measuring instruments in test–retest designs

A covariance structure modelling method for the estimation of reliability for composites of congeneric measures in test–retest designs is outlined. The approach also allows an approximate standard error and confidence interval for scale reliability in such settings to be obtained. The procedure further permits measurement error components due to possible transient condition influences to be accounted for and evaluated, and is illustrated with a pair of examples.     

Randomization tests for ABAB designs: comparing-data-division- specific and common distributions

Monte Carlo simulations were used to generate data for ABAB designs of different lengths. The points of change in phase are randomly determined before gathering behaviour measurements, which allows the use of a randomization test as an analytic technique. Data simulation and analysis can be based either on data-division-specific or on common distributions. Following one method or another affects the results obtained after the randomization test has been applied. Therefore, the goal of the study was to examine these effects in more detail. The discrepancies in these approaches are obvious when data with zero treatment effect are considered and such approaches have implications for statistical power studies. Data-division-specific distributions provide more detailed information about the performance of the statistical technique.     

Effects of information about group members on young children's attitudes towards the in‐group and out‐group

Research shows that being a member of a group is sufficient to instigate more positive attitudes towards the in‐group than an out‐group in young children. The present study assessed whether children's intergroup attitudes during the middle childhood years are moderated by additional information about in‐group and out‐group members, as proposed by Aboud's (1988) socio‐cognitive theory (ST). To a minimal group 6‐, 8‐, and 10‐year‐old children (N=159) were assigned, and received information, or no information, about the interests and activities of the in‐group and out‐group members. Results indicated that the in‐group was always rated more positively than the out‐group, and that the in‐group's ratings were unaffected by either the in‐group or out‐group information. In contrast, out‐group ratings were affected by out‐group information, but only when there was no information available about the in‐group. The implications of the findings for ST, and for social identity development theory, are discussed.     

Randomization tests for multiple-baseline designs: An extension of the SCRT-R package

Multiple-baseline designs are an extension of the basic single-case AB phase designs, in which several of those AB designs are implemented simultaneously to different persons, behaviors, or settings, and the intervention is introduced in a staggered way to the different units. These designs are well-suited for research in the behavioral sciences. We discuss the advantages and limitations for valid inferences, and suggest a statistical technique—randomization tests—for use with multiple-baseline data, to complement visual analysis. In addition, we provide an extension of our SCRT-R package (which already contained means for conducting randomization tests on single-case phase and alternation designs), for multiple-baseline AB data.     

A comparison of severalk sample tests for ordered alternatives in completely randomized designs

The importance of appropriate test selection for a given research endeavor cannot be over-emphasized. Using samples drawn from eleven populations (differing in shape, peakedness, and density in the tails), this study investigates the small sample empirical powers of ninek-sample tests against ordered location alternatives under completely randomized designs. The results then are intended to aid the researcher in the selection of a particular procedure appropriate for a given endeavor. To highlight this an industrial psychology application involving work productivity is presented.Research was supported in part by the Scholastic Assistance Program, Baruch College. The author wishes to thank Professors Matthew Goldstein, Shulamith Gross, David Levine, and Edward Wolf for their helpful comments when writing this paper. In addition, the author wishes to thank the referees and editor for their useful suggestions for improving the paper.     

Ideal selves and self‐esteem in people with independent or interdependent self‐construal

Not living up to one's ideal self has been shown to coincide with decreased self‐esteem. In the present paper, this notion is applied to the differentiation between people with independent versus interdependent self‐construal. We suggest that the ideal self of independents differs in two respects from the one of interdependents: with respect to its contents (autonomous versus social self‐knowledge) and with respect to the degree of context‐dependency of the encoded knowledge (context‐independent versus context‐dependent self‐knowledge). In three studies, via a priming we either manipulated contents or degree of context‐dependency of what participants considered themselves to actually be like. On both explicit and implicit measures, participants with independent construal indicated higher self‐esteem after priming of autonomous and context‐independent knowledge than after priming of social and context‐dependent knowledge. The opposite pattern was observed in participants with interdependent construal. Results suggest that independent and interdependent construals mirror different ideals which are applied as a comparison standard when evaluating the self. Copyright © 2006 John Wiley & Sons, Ltd.