Effect of non-normality on test statistics for one-way independent groups designs

The data obtained from one‐way independent groups designs is typically non‐normal in form and rarely equally variable across treatment populations (i.e. population variances are heterogeneous). Consequently, the classical test statistic that is used to assess statistical significance (i.e. the analysis of variance F test) typically provides invalid results (e.g. too many Type I errors, reduced power). For this reason, there has been considerable interest in finding a test statistic that is appropriate under conditions of non‐normality and variance heterogeneity. Previously recommended procedures for analysing such data include the James test, the Welch test applied either to the usual least squares estimators of central tendency and variability, or the Welch test with robust estimators (i.e. trimmed means and Winsorized variances). A new statistic proposed by Krishnamoorthy, Lu, and Mathew, intended to deal with heterogeneous variances, though not non‐normality, uses a parametric bootstrap procedure. In their investigation of the parametric bootstrap test, the authors examined its operating characteristics under limited conditions and did not compare it to the Welch test based on robust estimators. Thus, we investigated how the parametric bootstrap procedure and a modified parametric bootstrap procedure based on trimmed means perform relative to previously recommended procedures when data are non‐normal and heterogeneous. The results indicated that the tests based on trimmed means offer the best Type I error control and power when variances are unequal and at least some of the distribution shapes are non‐normal.     

Type I errors and power of the parametric bootstrap goodness‐of‐fit test: Full and limited information

In sparse tables for categorical data well‐known goodness‐of‐fit statistics are not chi‐square distributed. A consequence is that model selection becomes a problem. It has been suggested that a way out of this problem is the use of the parametric bootstrap. In this paper, the parametric bootstrap goodness‐of‐fit test is studied by means of an extensive simulation study; the Type I error rates and power of this test are studied under several conditions of sparseness. In the presence of sparseness, models were used that were likely to violate the regularity conditions. Besides bootstrapping the goodness‐of‐fit usually used (full information statistics), corrected versions of these statistics and a limited information statistic are bootstrapped. These bootstrap tests were also compared to an asymptotic test using limited information. Results indicate that bootstrapping the usual statistics fails because these tests are too liberal, and that bootstrapping or asymptotically testing the limited information statistic works better with respect to Type I error and outperforms the other statistics by far in terms of statistical power. The properties of all tests are illustrated using categorical Markov models.     

Randomization procedures in single‐case intervention research contexts: (Some of) “the rest of the story”

Following up on articles recently published in this journal, the present contribution tells (some of) “the rest of the story” about the value of randomization in single‐case intervention research investigations. Invoking principles of internal, statistical‐conclusion, and external validity, we begin by emphasizing the critical distinction between design randomization and analysis randomization, along with the necessary correspondence between the two. Four different types of single‐case design‐and‐analysis randomization are then discussed. The persistent negative influence of serially dependent single‐case outcome observations is highlighted, accompanied by examples of inappropriate applications of parametric and nonparametric tests that have appeared in the literature. We conclude by presenting valid applications of single‐case randomization procedures in various single‐case intervention contexts, with specific reference to a freely available Excel‐based software package that can be accessed to incorporate the present randomization schemes into a wide variety of single‐case intervention designs and analyses.     

Non‐parametric three‐way mixed ANOVA with aligned rank tests

Research problems that require a non‐parametric analysis of multifactor designs with repeated measures arise in the behavioural sciences. There is, however, a lack of available procedures in commonly used statistical packages. In the present study, a generalization of the aligned rank test for the two‐way interaction is proposed for the analysis of the typical sources of variation in a three‐way analysis of variance (ANOVA) with repeated measures. It can be implemented in the usual statistical packages. Its statistical properties are tested by using simulation methods with two sample sizes (n = 30 and n = 10) and three distributions (normal, exponential and double exponential). Results indicate substantial increases in power for non‐normal distributions in comparison with the usual parametric tests. Similar levels of Type I error for both parametric and aligned rank ANOVA were obtained with non‐normal distributions and large sample sizes. Degrees‐of‐freedom adjustments for Type I error control in small samples are proposed. The procedure is applied to a case study with 30 participants per group where it detects gender differences in linguistic abilities in blind children not shown previously by other methods.     

The effect of counselling programmes involving expressive activities with semi‐structured groups on self‐concealment levels of adolescents

The aim of this study was to investigate the effects of counselling programmes based on expressive activities with semi‐structured groups, on the self‐concealment levels of adolescents. The research was carried out using a pretest–posttest control group design. The dependent variable of the present research, is the psychological counselling programme conducted with a semi‐structured group based on expressive activities. The research was carried out with 18 6th and 7th grade secondary school students (10 female and 8 male). The Self‐Concealment Scale was used as data collection tool. In this research, the use of nonparametric tests was preferred, as the sample was smaller than 30. Therefore, it was determined whether there is a statistically significant difference between the arithmetic means of the experimental and control groups’ pretest, posttest and trail test scores by use of Wilcoxon signed‐rank test, to determine the effects of a psychological counselling programme with semi‐structured group based on expressive activities. Also, it was determined whether there was a statistically significant relationship between the pretest, posttest and trail intervention scores received by the intervention and control group students from the Self‐Concealment Scale, using Mann–Whitney U test. The research results revealed that the “psychological counselling programme based on expressive activities with semi‐structured groups” was significantly effective in alleviation of self‐concealment levels, and this effect was also maintained in follow‐up evaluations. The obtained results are discussed in the light of related literature findings and recommendations are proposed accordingly.     

Randomization tests as alternative analysis methods for behavior‐analytic data

Randomization statistics offer alternatives to many of the statistical methods commonly used in behavior analysis and the psychological sciences, more generally. These methods are more flexible than conventional parametric and nonparametric statistical techniques in that they make no assumptions about the underlying distribution of outcome variables, are relatively robust when applied to small‐n data sets, and are generally applicable to between‐groups, within‐subjects, mixed, and single‐case research designs. In the present article, we first will provide a historical overview of randomization methods. Next, we will discuss the properties of randomization statistics that may make them particularly well suited for analysis of behavior‐analytic data. We will introduce readers to the major assumptions that undergird randomization methods, as well as some practical and computational considerations for their application. Finally, we will demonstrate how randomization statistics may be calculated for mixed and single‐case research designs. Throughout, we will direct readers toward resources that they may find useful in developing randomization tests for their own data.     

More powerful tests of predictor subsets in regression analysis under nonnormality

It is well-known that for normally distributed errors parametric tests are optimal statistically, but perhaps less well-known is that when normality does not hold, nonparametric tests frequently possess greater statistical power than parametric tests, while controlling Type I error rate. However, the use of nonparametric procedures has been limited by the absence of easily performed tests for complex experimental designs and analyses and by limited information about their statistical behavior for realistic conditions. A Monte Carlo study of tests of predictor subsets in multiple regression analysis indicates that various nonparametric tests show greater power than the F test for skewed and heavy-tailed data. These nonparametric tests can be computed with available software.     

Personalizing Behavioral Interventions Through Single‐Patient (N‐of‐1) Trials

Behavioral interventions are typically studied with the use of a conventional between‐subject randomized controlled trial (RCT) design. In this design, the effect of an intervention on one group of patients is compared with the effect of a control condition on another group of patients, such that a between‐subject change is tested. A between‐subject design has an underlying assumption that there is a homogenous treatment effect for a behavioral intervention, drug, or psychotherapy and that the way the intervention operates in the study will tend to operate in the same way in many other patients. We review some of the philosophical and practical problems with the use of this design when a clinician is attempting to decide on a course of behavioral treatment aimed at within‐subject change in patients who are likely to have heterogeneous or unique responses to behavioral treatment. We also review the biases inherent in our current clinical practice model, which does not use any empirical data collection or design for testing if a treatment is useful, and also in the conventional between‐subject personalized medicine RCT designs. We propose increased use of single‐patient (also known as N‐of‐1) trials that employ within‐subject designs, in cases where treatment response is heterogeneous – as is the case for most psychological and behavioral treatments. Limitations of such designs include that they can only be used when the treatment is potentially reversible, the patient can act as their own control, and the outcome can be measured repeatedly. Increased use of within‐subject trials may address in many more instances the more clinically relevant question of how a specific patient will respond to a specific treatment and could introduce a more harmonious scientific approach into the way we treat our patients. We have incorporated a case presentation that illustrates the complexities of applying evidence drawn from these different designs to selecting and evaluating treatments for the behavioral issues commonly faced by clinicians and patients.     

Prejudice‐reduction in Culturally Mixed Classrooms: The Development and Assessment of a Theory‐driven Intervention Among Majority and Minority Youth in Finland

The aim of this field experiment was to develop and assess an intervention promoting positive intergroup relations in culturally diverse schools. The intervention was based on extended contact and social learning and utilized behavioural journalism as its method. Intervention effects were assessed on out‐group attitudes, perceived importance of future contact, perceived peer norms and intergroup anxiety among ethnic majority (N = 583) and minority (N = 214) youth in grades 7–9 in Finnish secondary schools (total N_experimental = 388; total N_control = 409). As a result of the intervention, both groups showed a tendency to perceive future intergroup contact as more important, and this effect was most notable for younger participants and girls. Prototypicality of in‐group and out‐group peer models contributed positively to intervention effects. However, unexpectedly, the intervention also increased experiences of intergroup anxiety among the oldest participants. The results are discussed taking into account the developmental stage of the youth studied. Besides critically assessing the effectiveness of the intervention, recommendations for improving theory‐driven prejudice‐reduction and for the development of future interventions in culturally diverse contexts are given. Copyright © 2013 John Wiley & Sons, Ltd.     

Single‐Case Experimental Designs for the Evaluation of Treatments for Self‐Injurious and Suicidal Behaviors

Single‐case experimental designs (SCEDs) provide a time‐ and cost‐effective alternative to randomized clinical trials and offer significant advantages in terms of internal and external validity. A brief history and primer on SCEDs is provided, specifically for use in suicide intervention research. Various SCED methodologies, such as AB, ABAB, multiple baseline, and changing criterion designs, are described. Advantages and disadvantages of their use specifically for intervention research for suicidal and self‐injurious behaviors are detailed.     

CAMP: A community‐based approach to promoting safe sex behaviour in adolescence

The primary goal of this study was to describe the development and assessment of the Community Awareness Motivation Partnership (CAMP) Teen Theatre intervention. Based on the behavioural ecological model, CAMP addressed the role of contraceptive use in safe sex behaviour through a brief informative, entertaining, and culturally responsive dramatization programme. Adolescents (N = 1613) between the ages of 11 and 18 (M = 13.31, SD = 0.76) received the intervention, and 961 participants completed both pre‐ and post‐tests. The majority of participants were Latino (78.5%, n = 1039), many of whom were first generation immigrants. Results indicated that the intervention increased participants' reported intentions to delay sex and use contraceptives. The intervention produced short‐term changes in safe‐sex behavioural intentions and knowledge. Suggestions for producing long‐term changes were discussed. Copyright © 2003 John Wiley & Sons, Ltd.     

Delivering phonological and phonics training within whole‐class teaching

Background Early, intensive phonological awareness and phonics training is widely held to be beneficial for children with poor phonological awareness. However, most studies have delivered this training separately from children's normal whole‐class reading lessons. Aims We examined whether integrating this training into whole class, mixed‐ability reading lessons could impact on children with poor phonological awareness, whilst also benefiting normally developing readers. Sample Teachers delivered the training within a broad reading programme to whole classes of children from Reception to the end of Year 1 (N = 251). A comparison group of children received standard teaching methods (N = 213). Method Children's literacy was assessed at the beginning of Reception, and then at the end of each year until 1 year post‐intervention. Results The strategy significantly impacted on reading performance for normally developing readers and those with poor phonological awareness, vastly reducing the incidence of reading difficulties from 20% in comparison schools to 5% in intervention schools. Conclusions Phonological and phonics training is highly effective for children with poor phonological awareness, even when incorporated into whole‐class teaching.     

Reducing Alcohol‐Related Consequences Among High School Seniors: Efficacy of a Brief,Web‐Based Intervention

The authors examined the efficacy of a brief, web‐based personalized feedback intervention on reducing alcohol‐related consequences among high school seniors (N = 105) using a group‐randomized controlled design. Results of repeated measures mixed‐models analyses indicated significant intervention effects over time for alcohol‐related consequences at 30‐day and 6‐month follow‐up assessments. Drinking risk status moderated intervention effects such that results were only significant for high‐risk drinkers (i.e., students reporting initiation of heavy episodic drinking at baseline).     

A note on consistency of non-parametric rank tests and related rank transformations

The extent to which rank transformations result in the same statistical decisions as their non‐parametric counterparts is investigated. Simulations are presented using the Wilcoxon–Mann–Whitney test, the Wilcoxon signed‐rank test and the Kruskal–Wallis test, together with the rank transformations and t and F tests corresponding to each of those non‐parametric methods. In addition to Type I errors and power over all simulations, the study examines the consistency of the outcomes of the two methods on each individual sample. The results show how acceptance or rejection of the null hypothesis and differences in p‐values of the test statistics depend in a regular and predictable way on sample size, significance level, and differences between means, for normal and various non‐normal distributions.     

Working adults’ well‐being: An online self‐help goal‐based intervention

An online self‐help goal‐setting and planning (GAP) intervention to improve working adults’ well‐being was tested using a longitudinal, randomized crossover design. The study sought to (1) test the effectiveness of the intervention relative to wait‐list controls; and (2) test the stability of effects over a 3‐month follow‐up period. Participants were recruited from the UK Civil Service and were randomized to either a GAP intervention or a wait‐list control condition. Wait‐list participants then crossed‐over to receive GAP. Relative to wait‐list controls (N = 149), GAP participants (N = 158) reported significantly higher levels of positive affect (PA) and flourishing, but similar levels of negative affect (NA) and life satisfaction immediately after the intervention. Longitudinal data were analysed for the whole sample (N = 307). Compared to the start of the intervention, participants reported an increase in PA and flourishing directly after the intervention and 3 months later. NA and life satisfaction showed no change by the end of the intervention, but had improved by 3‐month follow‐up. Completing more modules predicted post‐intervention improvements in well‐being, accounting for pre‐intervention well‐being levels. The online self‐help format allowed the intervention to be offered with minimal therapeutic support, enabling convenient access by a large group of employees. The study provides an example of a successful adaptation of a clinically proven well‐being intervention to make it accessible to working adults.

Practitioner points

• Well‐being interventions proven in clinical settings can be effectively adapted for use in workplace settings with only minor alterations.
• Brief, online self‐help interventions can improve working adults’ well‐being.
• Goal‐based interventions can improve working adults’ well‐being when focused towards goals that are aligned with personal values and have been chosen by the individual.

     

Natural reference: A phylo‐ and ontogenetic perspective on the comprehension of iconic gestures and vocalizations

The recognition of iconic correspondence between signal and referent has been argued to bootstrap the acquisition and emergence of language. Here, we study the ontogeny, and to some extent the phylogeny, of the ability to spontaneously relate iconic signals, gestures, and/or vocalizations, to previous experience. Children at 18, 24, and 36 months of age (N = 216) and great apes (N = 13) interacted with two apparatuses, each comprising a distinct action and sound. Subsequently, an experimenter mimicked either the action, the sound, or both in combination to refer to one of the apparatuses. Experiments 1 and 2 found no spontaneous comprehension in great apes and in 18‐month‐old children. At 24 months of age, children were successful with a composite vocalization‐gesture signal but not with either vocalization or gesture alone. At 36 months, children succeeded both with a composite vocalization‐gesture signal and with gesture alone, but not with vocalization alone. In general, gestures were understood better compared to vocalizations. Experiment 4 showed that gestures were understood irrespective of how children learned about the corresponding action (through observation or self‐experience). This pattern of results demonstrates that iconic signals can be a powerful way to establish reference in the absence of language, but they are not trivial for children to comprehend and not all iconic signals are created equal.     

Identifying and changing the normative beliefs about aggression which lead young Muslim adults to join extremist anti‐Semitic groups in Pakistan

Two studies investigated the role of beliefs about the acceptability of aggression (“normative beliefs”) against Jews in determining who would join an extremist group. In Study 1, students in a university in Pakistan (N=144) completed self‐report attitude measures, and were subsequently approached by a confederate who asked whether they wanted to join an extremist anti‐Semitic organization. Normative beliefs about aggression against Jews were very strong predictors of whether participants agreed to join. In Study 2, participants (N=92) were experimentally assigned to either a brief educational intervention, designed to improve inter‐group relations, or to a control group. They also filled in self‐report attitude measures pre and post intervention. Participants in the intervention group were much less likely to agree to join the extremist group, and this effect of the intervention on joining was mediated by changes in normative beliefs about aggression against Jews. The results have implications for theories of inter‐group aggression and interventions to prevent people from being recruited into extremist groups. Aggr. Behav. 35:514–519, 2009. © 2009 Wiley‐Liss, Inc.     

Applying mixed‐effects modeling to single‐subject designs: An introduction

Behavior analysis and statistical inference have shared a conflicted relationship for over fifty years. However, a significant portion of this conflict is directed toward statistical tests (e.g., t‐tests, ANOVA) that aggregate group and/or temporal variability into means and standard deviations and as a result remove much of the data important to behavior analysts. Mixed‐effects modeling, a more recently developed statistical test, addresses many of the limitations of more basic tests by incorporating random effects. Random effects quantify individual subject variability without eliminating it from the model, hence producing a model that can predict both group and individual behavior. We present the results of a generalized linear mixed‐effects model applied to single‐subject data taken from Ackerlund Brandt, Dozier, Juanico, Laudont, & Mick, 2015, in which children chose from one of three reinforcers for completing a task. Results of the mixed‐effects modeling are consistent with visual analyses and importantly provide a statistical framework to predict individual behavior without requiring aggregation. We conclude by discussing the implications of these results and provide recommendations for further integration of mixed‐effects models in the analyses of single‐subject designs.     

Comparing two independent groups: A test based on a one‐step M‐estimator and bootstrap‐t

A new test is proposed for the problem of comparing two independent groups in terms of some measure of location. The proposed test () uses a one‐step M‐estimator and a bootstrap‐t method with the procedure proposed by Özdemir and Kurt (2006) . Eight methods were compared in terms of actual Type I error and power when the underlying distributions differ in skewness and kurtosis under heterogeneity of variances. For the 21 theoretical distributions, the Yuen test with the bootstrap‐t method was the most favourable, followed by test. For the five real data sets, the proposed test and percentile bootstrap method with the one‐step M‐estimator performed best.     

Bootstrap scree tests: A Monte Carlo simulation and applications to published data

A non‐parametric procedure for Cattell's scree test is proposed, using the bootstrap method. Bentler and Yuan developed parametric tests for the linear trend of scree eigenvalues in principal component analysis. The proposed method is for cases where parametric assumptions are not realistic. We define the break in the scree trend in several ways, based on linear slopes defined with two or three consecutive eigenvalues, or all eigenvalues after the k largest. The resulting scree test statistics are evaluated under various data conditions, among which Gorsuch and Nelson's bootstrap CNG performs best and is reasonably consistent and efficient under leptokurtic and skewed conditions. We also examine the bias‐corrected and accelerated bootstrap method for these statistics, and the bias correction is found to be too unstable to be useful. Using seven published data sets which Bentler and Yuan analysed, we compare the bootstrap approach to the scree test with the parametric linear trend test.