Robust step‐down tests for multivariate independent group designs

A composite step‐down procedure, in which a set of step‐down tests are summarized collectively with Fisher's combination statistic, was considered to test for multivariate mean equality in two‐group designs. An approximate degrees of freedom (ADF) composite procedure based on trimmed/Winsorized estimators and a non‐pooled estimate of error variance is proposed, and compared to a composite procedure based on trimmed/Winsorized estimators and a pooled estimate of error variance. The step‐down procedures were also compared to Hotelling's T² and Johansen's ADF global procedure based on trimmed estimators in a simulation study. Type I error rates of the pooled step‐down procedure were sensitive to covariance heterogeneity in unbalanced designs; error rates were similar to those of Hotelling's T² across all of the investigated conditions. Type I error rates of the ADF composite step‐down procedure were insensitive to covariance heterogeneity and less sensitive to the number of dependent variables when sample size was small than error rates of Johansen's test. The ADF composite step‐down procedure is recommended for testing hypotheses of mean equality in two‐group designs except when the data are sampled from populations with different degrees of multivariate skewness.     

Gaussian model‐based partitioning using iterated local search

The emergence of Gaussian model‐based partitioning as a viable alternative to K‐means clustering fosters a need for discrete optimization methods that can be efficiently implemented using model‐based criteria. A variety of alternative partitioning criteria have been proposed for more general data conditions that permit elliptical clusters, different spatial orientations for the clusters, and unequal cluster sizes. Unfortunately, many of these partitioning criteria are computationally demanding, which makes the multiple‐restart (multistart) approach commonly used for K‐means partitioning less effective as a heuristic solution strategy. As an alternative, we propose an approach based on iterated local search (ILS), which has proved effective in previous combinatorial data analysis contexts. We compared multistart, ILS and hybrid multistart–ILS procedures for minimizing a very general model‐based criterion that assumes no restrictions on cluster size or within‐group covariance structure. This comparison, which used 23 data sets from the classification literature, revealed that the ILS and hybrid heuristics generally provided better criterion function values than the multistart approach when all three methods were constrained to the same 10‐min time limit. In many instances, these differences in criterion function values reflected profound differences in the partitions obtained.     

Coloured Letters and Numbers (CLaN): A reliable factor-analysis based synaesthesia questionnaire

Synaesthesia is a heterogeneous phenomenon, even when considering one particular sub-type. The purpose of this study was to design a reliable and valid questionnaire for grapheme-colour synaesthesia that captures this heterogeneity. By the means of a large sample of 628 synaesthetes and a factor analysis, we created the Coloured Letters and Numbers (CLaN) questionnaire with 16 items loading on 4 different factors (i.e., localisation, automaticity/attention, deliberate use, and longitudinal changes). These factors were externally validated with tests which are widely used in the field of synaesthesia research. The questionnaire showed good test–retest reliability and construct validity (i.e., internally and externally). Our findings are discussed in the light of current theories and new ideas in synaesthesia research. More generally, the questionnaire is a useful tool which can be widely used in synaesthesia research to reveal the influence of individual differences on various performance measures and will be useful in generating new hypotheses.     

A clusterwise simultaneous component method for capturing within‐cluster differences in component variances and correlations

This paper presents a clusterwise simultaneous component analysis for tracing structural differences and similarities between data of different groups of subjects. This model partitions the groups into a number of clusters according to the covariance structure of the data of each group and performs a simultaneous component analysis with invariant pattern restrictions (SCA‐P) for each cluster. These restrictions imply that the model allows for between‐group differences in the variances and the correlations of the cluster‐specific components. As such, clusterwise SCA‐P is more flexible than the earlier proposed clusterwise SCA‐ECP model, which imposed equal average cross‐products constraints on the component scores of the groups that belong to the same cluster. Using clusterwise SCA‐P, a finer‐grained, yet parsimonious picture of the group differences and similarities can be obtained. An algorithm for fitting clusterwise SCA‐P solutions is presented and its performance is evaluated by means of a simulation study. The value of the model for empirical research is illustrated with data from psychiatric diagnosis research.     

Testing two variances for superiority/non-inferiority and equivalence: Using the exhaustion algorithm for sample size allocation with cost

The equality of two group variances is frequently tested in experiments. However, criticisms of null hypothesis statistical testing on means have recently arisen and there is interest in other types of statistical tests of hypotheses, such as superiority/non-inferiority and equivalence. Although these tests have become more common in psychology and social sciences, the corresponding sample size estimation for these tests is rarely discussed, especially when the sampling unit costs are unequal or group sizes are unequal for two groups. Thus, for finding optimal sample size, the present study derived an initial allocation by approximating the percentiles of an F distribution with the percentiles of the standard normal distribution and used the exhaustion algorithm to select the best combination of group sizes, thereby ensuring the resulting power reaches the designated level and is maximal with a minimal total cost. In this manner, optimization of sample size planning is achieved. The proposed sample size determination has a wide range of applications and is efficient in terms of Type I errors and statistical power in simulations. Finally, an illustrative example from a report by the Health Survey for England, 1995–1997, is presented using hypertension data. For ease of application, four R Shiny apps are provided and benchmarks for setting equivalence margins are suggested.     

GROUP DIFFERENCES IN PERSONALITY: META‐ANALYSES COMPARING FIVE U.S. RACIAL GROUPS

This research poses 2 applied questions: How large are racial group differences on personality scales and are these differences likely to cause adverse impact in personnel selection? We examined the extent to which racial groups differ across Big 5 personality factors and facets. Large‐scale, quantitative estimates based on over 700 effect sizes were meta‐analytically summarized. Multiple personality instruments and understudied racial groups, particularly Asian Americans and American Indians, were included in the meta‐analyses. Most group comparisons and personality scales yielded negligible differences and are not likely to cause adverse impact in selection. However, facet‐level analyses produced different d‐values, with some group comparisons showing moderate differences, suggesting that the use of personality measures in selection does not uniformly circumvent adverse impact concerns. The veracity of this conclusion depends on (a) the particular trait in question, (b) the composition of the applicant pool (i.e., which groups are to be compared with one another), (c) the effect size (i.e., d‐value), and (d) the selection ratio. For practitioners, we present a quantitatively informed summary of when and where to expect adverse impact to result from personality measures used in personnel selection.     

Bayesian model selection of informative hypotheses for repeated measurements

When analyzing repeated measurements data, researchers often have expectations about the relations between the measurement means. The expectations can often be formalized using equality and inequality constraints between (i) the measurement means over time, (ii) the measurement means between groups, (iii) the means adjusted for time-invariant covariates, and (iv) the means adjusted for time-varying covariates. The result is a set of informative hypotheses. In this paper, the Bayes factor is used to determine which hypothesis receives most support from the data. A pivotal element in the Bayesian framework is the specification of the prior. To avoid subjective prior specification, training data in combination with restrictions on the measurement means are used to obtain so-called constrained posterior priors. A simulation study and an empirical example from developmental psychology show that this prior results in Bayes factors with desirable properties.     

Part‐set cueing effects in a diagnostic setting with professional auditors

The aim of this research is to examine the generality of the part‐set cueing effect, a well‐known memory‐inhibition phenomenon in basic research, to professional problem diagnosis. Three experiments were conducted. Experiment 1 examines whether part‐set cueing affects the ability of auditors to recall diagnoses, and if so, whether memory inhibition is affected by the plausibility of the hypotheses. Experiment 2 examines whether the part‐set cueing effect is sensitive to changes in the symptoms that drive the diagnosis and whether differences in inhibition occur because of prompt plausibility or because of target plausibility. Experiment 3 extends the findings of the first two experiments to hypothesis generation and discriminates between competing accounts for the part‐set cueing effect. The results show that part‐set cueing effects occur during hypothesis generation and that they are robust to changes in the symptoms. However, the results from the experiments reveal an asymmetric inhibition effect. Inhibition is significant when prompts contain low plausibility hypotheses that are also high in severity. When prompts consist of high plausibility or low severity hypotheses, no inhibition is observed. These findings are considered to be consistent with an editing account of the part‐set cueing phenomenon. The implications of these findings to problem diagnosis and decision aids are discussed. Copyright © 1999 John Wiley & Sons, Ltd.     

Relations Between Narrative Coherence,Identity, and Psychological Well‐Being in Emerging Adulthood

The hypothesis that the ability to construct a coherent account of personal experience is reflective, or predictive, of psychological adjustment cuts across numerous domains of psychological science. It has been argued that coherent accounts of identity are especially adaptive. We tested these hypotheses by examining relations between narrative coherence of personally significant autobiographical memories and three psychological well‐being components (i.e., purpose and meaning, positive self‐view, positive relationships). We also examined the potential moderation of the relations between coherence and well‐being by assessing the identity content of each narrative. We collected two autobiographical narratives of personally significant events from 103 undergraduate students and coded them for coherence and identity content. Two additional narratives about generic/recurring events were also collected and coded for coherence. We confirmed the prediction that constructing coherent autobiographical narratives is related to psychological well‐being. Further, we found that this relation was moderated by the narratives' relevance to identity and that this moderation held after controlling for narrative ability more generally (i.e., coherence of generic/recurring events). These data lend strong support to the coherent narrative identity hypothesis and the prediction that unique events are a critical feature of identity construction in emerging adulthood.     

Evaluating expectations about negative emotional states of aggressive boys using Bayesian model selection

Researchers often have expectations about the research outcomes in regard to inequality constraints between, e.g., group means. Consider the example of researchers who investigated the effects of inducing a negative emotional state in aggressive boys. It was expected that highly aggressive boys would, on average, score higher on aggressive responses toward other peers than moderately aggressive boys, who would in turn score higher than nonaggressive boys. In most cases, null hypothesis testing is used to evaluate such hypotheses. We show, however, that hypotheses formulated using inequality constraints between the group means are generally not evaluated properly. The wrong hypotheses are tested, i.e.. the null hypothesis that group means are equal. In this article, we propose an innovative solution to these above-mentioned issues using Bayesian model selection, which we illustrate using a case study.     

Statistical criteria for parallel tests: A comparison of accuracy and power

Parallel tests are needed so that alternate forms can be applied to different groups or on different occasions, but also in the context of split-half reliability estimation for a given test. Statistically, parallelism holds beyond reasonable doubt when the null hypotheses of equality of observed means and variances across the two forms (or halves) are not rejected. Several statistical tests have been proposed for this purpose, but their performance has never been compared. This study assessed the relative performance (type I error rate and power) of the Student–Pitman–Morgan, Bradley–Blackwood, and Wilks tests of equality of means and variances in the typical conditions surrounding studies of parallelism—namely, integer-valued and bounded test scores with distributions that may not be bivariate normal. The results advise against the use of the Wilks test and support the use of the Bradley–Blackwood test because of its simplicity and its minimally better performance in comparison with the more cumbersome Student–Pitman–Morgan test.     

A TRICKLE‐DOWN MODEL OF ABUSIVE SUPERVISION

Much of the abusive supervision research has focused on the supervisor–subordinate dyad when examining the effects of abusive supervision on employee outcomes. Using data from a large multisource field study, we extend this research by testing a trickle‐down model of abusive supervision across 3 hierarchical levels (i.e., managers, supervisors, and employees). Drawing on social learning theory and social information processing theory, we find general support for the study hypotheses. Specifically, we find that abusive manager behavior is positively related to abusive supervisor behavior, which in turn is positively related to work group interpersonal deviance. In addition, hostile climate moderates the relationship between abusive supervisor behavior and work group interpersonal deviance such that the relationship is stronger when hostile climate is high. The results provide support for our trickle‐down model in that abusive manager behavior was not only related to abusive supervisor behavior but was also associated with employees’ behavior 2 hierarchical levels below the manager.     

Effect of non-normality on test statistics for one-way independent groups designs

The data obtained from one‐way independent groups designs is typically non‐normal in form and rarely equally variable across treatment populations (i.e. population variances are heterogeneous). Consequently, the classical test statistic that is used to assess statistical significance (i.e. the analysis of variance F test) typically provides invalid results (e.g. too many Type I errors, reduced power). For this reason, there has been considerable interest in finding a test statistic that is appropriate under conditions of non‐normality and variance heterogeneity. Previously recommended procedures for analysing such data include the James test, the Welch test applied either to the usual least squares estimators of central tendency and variability, or the Welch test with robust estimators (i.e. trimmed means and Winsorized variances). A new statistic proposed by Krishnamoorthy, Lu, and Mathew, intended to deal with heterogeneous variances, though not non‐normality, uses a parametric bootstrap procedure. In their investigation of the parametric bootstrap test, the authors examined its operating characteristics under limited conditions and did not compare it to the Welch test based on robust estimators. Thus, we investigated how the parametric bootstrap procedure and a modified parametric bootstrap procedure based on trimmed means perform relative to previously recommended procedures when data are non‐normal and heterogeneous. The results indicated that the tests based on trimmed means offer the best Type I error control and power when variances are unequal and at least some of the distribution shapes are non‐normal.     

Overcoming cognitive obstacles in learning fractions: Equal‐partitioning and equal‐whole1

Abstract: The present study was designed to test the hypothesis that acquisition of the concepts of equal‐partitioning and equal‐whole, which constitute implicit knowledge in fractions, promotes understanding of fractions. To test the hypothesis an experimental program based on both the equal‐partitioning and equal‐whole was developed for three third‐grade classrooms in a Japanese elementary school. As a control, conventional teaching based on a Japanese textbook was provided in two third‐grade classrooms. The present study tested the hypothesis that students given such an intervention program would understand order and magnitude as central characteristics of fractions better than those who had been instructed using a traditional textbook. Students in the experimental group showed significantly better understanding of the equal‐partitioning of fractions and the representation of the sizes of fractions than did those in the textbook group. There were, moreover, no differences between the experimental and textbook groups in the performance of routine tasks frequently found in the textbook. These results are discussed in view of the important instructional aims of having students understand fractions.     

A note on preliminary tests of equality of variances

Preliminary tests of equality of variances used before a test of location are no longer widely recommended by statisticians, although they persist in some textbooks and software packages. The present study extends the findings of previous studies and provides further reasons for discontinuing the use of preliminary tests. The study found Type I error rates of a two‐stage procedure, consisting of a preliminary Levene test on samples of different sizes with unequal variances, followed by either a Student pooled‐variances t test or a Welch separate‐variances t test. Simulations disclosed that the twostage procedure fails to protect the significance level and usually makes the situation worse. Earlier studies have shown that preliminary tests often adversely affect the size of the test, and also that the Welch test is superior to the t test when variances are unequal. The present simulations reveal that changes in Type I error rates are greater when sample sizes are smaller, when the difference in variances is slight rather than extreme, and when the significance level is more stringent. Furthermore, the validity of the Welch test deteriorates if it is used only on those occasions where a preliminary test indicates it is needed. Optimum protection is assured by using a separate‐variances test unconditionally whenever sample sizes are unequal.     

Consistently Inconsistent Working Memory Performance Among Children with ADHD: Evidence of Response Accuracy Variability (RAV)

Heterogeneity in cognitive performance, once regarded as noise, is now considered a causal mechanism or core deficit of ADHD and its related symptoms in most etiological models of the disorder. Previous research on cognitive performance variability has documented increased heterogeneity in response latencies using reaction time data. In contrast, variability in response accuracy remains understudied. The present study is the first to examine Response Accuracy Variability (RAV) among children with ADHD. Children with ADHD (N?=?54) and typically developing children (N?=?50) completed phonological working memory tasks with four set size conditions. RAV was calculated for each set size using the adjusted coefficient of variation (ζ). Results from a mixed model ANOVA indicated that children with ADHD evinced significantly greater variation in working memory performance relative to typically developing children when engaged in tasks within their cognitive capacity (i.e., set sizes 3 and 4), whereas all children exhibit similar, high levels of variability on tasks that exceeded their cognitive capacity (i.e., set sizes 5 and 6). Findings are aligned with the extant literature in documenting consistently inconsistent cognitive performance among children ADHD.

     

Sample size determinations for Welch's test in one‐way heteroscedastic ANOVA

For one‐way fixed effects ANOVA, it is well known that the conventional F test of the equality of means is not robust to unequal variances, and numerous methods have been proposed for dealing with heteroscedasticity. On the basis of extensive empirical evidence of Type I error control and power performance, Welch's procedure is frequently recommended as the major alternative to the ANOVA F test under variance heterogeneity. To enhance its practical usefulness, this paper considers an important aspect of Welch's method in determining the sample size necessary to achieve a given power. Simulation studies are conducted to compare two approximate power functions of Welch's test for their accuracy in sample size calculations over a wide variety of model configurations with heteroscedastic structures. The numerical investigations show that Levy's (1978a) approach is clearly more accurate than the formula of Luh and Guo (2011) for the range of model specifications considered here. Accordingly, computer programs are provided to implement the technique recommended by Levy for power calculation and sample size determination within the context of the one‐way heteroscedastic ANOVA model.     

Surveying the forest: A meta‐analysis,moderator investigation,and future‐oriented discussion of the antecedents of voluntary employee turnover

Recent narrative reviews (e.g., Hom, Mitchell, Lee, and Griffeth, 2012; Hom, Lee, Shaw, and Hausknecht, 2017) advise that it is timely to assess the progress made in research on voluntary employee turnover so as to guide future work. To provide this assessment, we employed a three‐step approach. First, we conducted a comprehensive meta‐analysis of turnover predictors, updating existing effect sizes and examining multiple new antecedents. Second, guided by theory, we developed and tested a set of substantive moderators, considering factors that might exacerbate or mitigate zero‐order meta‐analytic effects. Third, we examined the holistic pattern of results in order to highlight the most pressing needs for future turnover research. The results of Step 1 reveal multiple newer predictors and updated effect sizes of more traditional predictors, which have received substantially greater study. The results of Step 2 provide insight into the context‐dependent nature of many antecedent–turnover relationships. In Step 3, our discussion takes a bird's‐eye view of the turnover “forest” and considers the theoretical and practical implications of the results. We offer several research recommendations that break from the traditional turnover paradigm, as a means of guiding future study.     

An alternative approach to relapse analysis: Using Monte Carlo methods and proportional rates of response

Relapse is the recovery of a previously suppressed response. Animal models have been useful in examining the mechanisms underlying relapse (e.g., reinstatement, renewal, reacquisition, resurgence). However, there are several challenges to analyzing relapse data using traditional approaches. For example, null hypothesis significance testing is commonly used to determine whether relapse has occurred. However, this method requires several a priori assumptions about the data, as well as a large sample size for between‐subjects comparisons or repeated testing for within‐subjects comparisons. Monte Carlo methods may represent an improved analytic technique, because these methods require no prior assumptions, permit smaller sample sizes, and can be tailored to account for all of the data from an experiment instead of some limited set. In the present study, we conducted reanalyses of three studies of relapse (Berry, Sweeney, & Odum, 2014 ; Galizio et al., 2018 ; Odum & Shahan, 2004 ) using Monte Carlo techniques to determine if relapse occurred and if there were differences in rate of response based on relevant independent variables (such as group membership or schedule of reinforcement). These reanalyses supported the previous findings. Finally, we provide general recommendations for using Monte Carlo methods in studies of relapse.