Repeated measures one‐way ANOVA based on a modified one‐step M‐estimator

Wilcox, Keselman, Muska and Cribbie (2000) found a method for comparing the trimmed means of dependent groups that performed well in simulations, in terms of Type I errors, with a sample size as small as 21. Theory and simulations indicate that little power is lost under normality when using trimmed means rather than untrimmed means, and trimmed means can result in substantially higher power when sampling from a heavy‐tailed distribution. However, trimmed means suffer from two practical concerns described in this paper. Replacing trimmed means with a robust M‐estimator addresses these concerns, but control over the probability of a Type I error can be unsatisfactory when the sample size is small. Methods based on a simple modification of a one‐step M‐estimator that address the problems with trimmed means are examined. Several omnibus tests are compared, one of which performed well in simulations, even with a sample size of 11.     

Comparing groups in a before–after design: When t test and ANCOVA produce different results

Background. Researchers often test people before and after some treatment and compare these scores with a control group. Sometimes it is not possible to allocate people into conditions randomly, which means the initial scores for the two groups may differ. There are two main approaches: t test on the gain scores and ANCOVA partialling out the initial scores. Lord (1967) showed that these can lead to different conclusions. This is an often‐discussed paradox in psychology and education. Aims. The reasons why these approaches can lead to different conclusions, the assumptions that each approach makes and how the approaches relate to group allocation, are discussed Methods. Three sets of simulations are reported that investigate the relationships among effect size, group allocation, measurement error and Lord's paradox. Conclusions. Recommendations are given that stress careful examination of the research questions, sampling and allocation of participants and graphing the data. ANCOVA is appropriate when allocation is based on the initial scores, t test can be appropriate if allocation is associated non‐causally with the initial scores, but often neither approach provides adequate results.     

Reaction time and intelligence: Comparing associations based on two response modes

People who score highly on intelligence tests also tend to have faster and less variable reaction times. Effect size estimates for the reaction time–intelligence association are larger in samples that are more representative of the population. However, such samples have often been tested on a reaction time device that requires reading a number and processing its association with a specific response location (Cox, Huppert, & Whichelow, 1993). Here, we use this device and another reaction time device (Dykiert et al., 2010) that is similar, except that the responses require less processing; subjects simply press a button that is adjacent to the stimulus light. We focus on the possibility that lights as stimuli require less higher-order cognitive engagement than numbers, and then test whether parameters from these two tasks are highly correlated and similarly associated with age and higher cognitive abilities. Both tasks measured simple and choice reaction times and their intra-individual variation across trials. The parameters of the two tasks were very highly correlated and parameters from both tasks were similarly associated with age, social factors, and differences in higher cognitive abilities. The respective choice reaction time parameters from either task accounted for much of the age- and higher cognitive ability-associations of the other task's parameters. These findings are important in establishing that the effect sizes of higher cognitive ability associations with processing speed measures may be found when the processing demands are minimal.     

Helping Teams to Help Themselves: Comparing two Team‐Led Debriefing Methods

Team‐based structures have become more widely used in organizations. Therefore, it is important for team members to perform well in their current team and to build skills and enthusiasm for working on future teams. This study examined team debriefing, an intervention in which team members reflect on recent experiences to prepare for subsequent tasks. Prior researchers have shown that facilitated team debriefs work, but they have not examined how to enable teams to conduct their own debriefs or studied how debriefs affect individual level outcomes. Therefore, we compared 2 team‐led debriefing techniques: (a) an unguided debrief and (b) a guided debrief designed to incorporate lessons learned from prior debriefs. We collected data from 174 business students who were members of 35 teams from 9 sections of a Strategic Management course. Class sections were randomly assigned to one of the debriefing conditions, and teams completed 4 business cases over 10 weeks. A multilevel design was employed and a multistage model building approach was used to test the hypotheses using hierarchical linear modeling techniques. Results of this cluster randomized, quasi‐experimental design suggest that the team‐led guided debrief intervention resulted in superior team processes as compared to the unguided debriefing method. Team processes, in turn, related significantly to greater team performance and increased individual readiness for teamwork and enthusiasm for teaming. Implications for future research and practice are discussed.     

A new quantile estimator with weights based on a subsampling approach

Quantiles are widely used in both theoretical and applied statistics, and it is important to be able to deploy appropriate quantile estimators. To improve performance in the lower and upper quantiles, especially with small sample sizes, a new quantile estimator is introduced which is a weighted average of all order statistics. The new estimator, denoted NO, has desirable asymptotic properties. Moreover, it offers practical advantages over four estimators in terms of efficiency in most experimental settings. The Harrell–Davis quantile estimator, the default quantile estimator of the R programming language, the Sfakianakis–Verginis SV2 quantile estimator and a kernel quantile estimator. The NO quantile estimator is also utilized in comparing two independent groups with a percentile bootstrap method and, as expected, it is more successful than other estimators in controlling Type I error rates.     

The Role of Self‐focused Attention and Negative Self‐thought in Interview Anxiety: A test of two interventions

Job interview anxiety is an unpleasant experience for many candidates that can cause them significant distress in the days leading up to the interview. As a result, many candidates are motivated to overcome their feelings of anxiety, however, few empirical studies have examined strategies aimed at lowering nervous jitters. This study looked at the relation between interview anxiety and ‘self‐focused attention’ (the tendency to focus on oneself during the interview), and also negative self‐thoughts during the interview. We tested two training interventions that are grounded in the social anxiety literature, positive imagery (fostering positive self‐images), and field perspective taking (focusing attention externally), which were designed to reduce self‐focused attention and negative thought. Both interventions decreased interview anxiety and increased interview self‐efficacy, but did not affect interview performance scores.     

Addressing selective attrition in the enhanced response time‐based concealed information test: A within‐subject replication

The response time‐based concealed information test can reveal when a person recognizes a relevant item among other, irrelevant items, based on comparatively slower responding. Thereby, if a person is concealing the knowledge about the relevance of this item (e.g., recognizing it as a murder weapon), this deception can be revealed. A recent study, conducted online and using a between‐subject design, introduced a significantly enhanced version by including additional items in the task. While this modified version outperformed the original version, it also resulted in a much higher rate of participant dropouts (i.e., participants leaving the experiment's website without completing the task). The grave implication is that the perceived enhancement is perhaps merely due to selective attrition. Therefore, the current experiment replicates the original one, but using a within‐subject design. The results show that there is a large enhancement even when selective attrition is prevented.     

Attachment‐based intervention for substance‐using mothers: A preliminary test of the proposed mechanisms of change

Although randomized controlled trials examining the efficacy of attachment‐based interventions have been increasing in recent years, adequate measurement of treatment integrity, integrity–outcome associations, and mechanisms of change has been rare. The aim of this investigation was to conduct a rigorous test of proposed mechanisms of change in the Mothers and Toddlers Program (MTP) treatment model, a 12‐session, attachment‐based individual therapy for substance‐using mothers of children birth to 3 years of age. The MTP aims to improve maternal reflective functioning (RF) and representation quality (RQ) to bring about second‐order change in maternal caregiving behavior. Following guidelines from M.K. Nock ( 2007 ), it was hypothesized that (a) therapist adherence to unique MTP treatment components would uniquely predict improvement in RF and RQ and that (b) improvement in RF and RQ would function as unique mechanisms of change (when compared with other potential mechanisms—reduction in depression and increase in abstinence from drug use) in the improvement of caregiving behavior. Findings supported each hypothesis, confirming the proposed mechanisms of the treatment model. However, improvement in maternal depression also uniquely predicted improvement in caregiving behavior. Results underscore the potential value of attachment‐based parenting interventions for improving mother–child relations and the importance of providing these interventions in clinic settings where mothers have access to comprehensive care (e.g., psychiatric services).     

Putting evidence‐based claims to the test: A multi‐site classroom study of retrieval practice and spaced practice

Retrieval practice and spacing out studying help increase learning. Introductory Psychology students at nine colleges/universities took quizzes with more (RP+) or less (RP?) retrieval practice and more (SP+) or less (SP?) spacing between quizzes. We compared (N = 351) scores on class exams and on a standardized test. We also measured key student variables (e.g., depth of processing). GPA predicted a significant portion of variance in exam scores, R² = 0.05, F(1, 204) = 10.84, p = 0.001. We found a main effect for retrieval practice and a significant retrieval by spacing practice interaction, R² = 0.28, F(3, 201) = 27.38, p < 0.001. Students using SP+, as well as those with SP? and RP?, scored higher than students using SP? and RP+. In practice, closer attention needs to be paid to quiz and exam content overlap and the duration within which repetition is used.     

Connection‐based and object‐based grouping in multiple‐object tracking: A developmental study

Developmental research on Gestalt laws has previously revealed that, even as young as infancy, we are bound to group visual elements into unitary structures in accordance with a variety of organizational principles. Here, we focus on the developmental trajectory of both connection‐based and object‐based grouping, and investigate their impact on object formation in participants, aged 9–21 years old (N = 113), using a multiple‐object tracking paradigm. Results reveal a main effect of both age and grouping type, indicating that 9‐ to 21‐year‐olds are sensitive to both connection‐based and object‐based grouping interference, and tracking ability increases with age. In addition to its importance for typical development, these results provide an informative baseline to understand clinical aberrations in this regard.

Statement of contribution

What is already known on this subject?

• The origin of the Gestalt principles is still an ongoing debate: Are they innate, learned over time, or both?
• Developmental research has revealed how each Gestalt principle has its own trajectory and unique relationship to visual experience.
• Both connectedness and object‐based grouping play an important role in object formation during childhood.

What does this study add?

• The study identifies how sensitivity to connectedness and object‐based grouping evolves in individuals, aged 9–21 years old.
• Using multiple‐object tracking, results reveal that the ability to track multiple objects increases with age.
• These results provide an informative baseline to understand clinical aberrations in different types of grouping.

     

A meta‐mood model of rumination and depression: Preliminary test in a non‐clinical population

Depressive rumination and trait meta‐mood (emotional attention, emotional clarity and emotional repair) have been suggested as vulnerability factors leading to depression, but less is known about the associations among them. In this study, we examined the relationships between trait meta‐mood, rumination and depressive symptomatology. Using structural equation analysis in a large sample of a non‐clinical population we found a preliminary test of the role of trait meta‐mood dimensions in rumination and depressive symptomatology. Results indicated that attention to feelings has two pathways in its relation with rumination and depressive mood. On the one hand, emotional attention was associated with emotional clarity, and emotional clarity with emotional repair, which was related to lower depressive symptomatology, in part, by reducing rumination. On the other hand, emotional attention was directly associated with ruminative thoughts which, in turn, were related to higher depressive mood. Findings are discussed in terms of the implications of beliefs about emotions in the treatment of depression.     

The Validity of Conscientiousness for Predicting Job Performance: A meta‐analytic test of two hypotheses

This study examined two hypotheses regarding the moderating effects of job characteristics on the validity of personality. Using meta‐analytic techniques, the authors explored the extent to which the structural characteristics and cognitive ability requirements of jobs influence the role of conscientiousness in predicting performance. The results suggest that conscientiousness is a stronger predictor of performance in jobs that are highly routinized, and a weaker predictor of performance in jobs with high levels of cognitive ability requirements. Implications for theory and future research are discussed.     

The role of women's self‐injury support‐groups: a grounded theory

Research evidence suggests that services are struggling to adequately address the increasing incidence of self‐injury and the needs of women who self‐injure, while national self‐injury support‐groups across the UK appear to be growing in number. Despite their reported value, evidence regarding the role of self‐injury support‐groups in women's management of their self‐injury is lacking although government policy and official guidelines are advocating the incorporation of support‐groups into self‐injury services. Seven semi‐structured interviews were conducted and analysed using Grounded Theory to investigate the role of three UK self‐injury support‐groups in women's management of self‐injury and associated difficulties. Empowerment‐as‐process emerged as the core theme of self‐injury support‐groups, mediated through experiences of belonging, sharing, autonomy, positive feeling and change. Findings are discussed in relation to relevant theory and research, followed by critical evaluation and implications of the study. Copyright © 2006 John Wiley & Sons, Ltd.     

Comparing computer‐based training and lecture formats to teach visual analysis of baseline‐treatment graphs

Accurate analysis of data is vital to the validation of interventions. As such, there has been a recent increase in studies evaluating visual analysis training procedures. However, past investigations have not evaluated direct and indirect visual analysis training methods with matched instructional content that was systematically designed. Furthermore, training has rarely included assessment of generalization and maintenance of visual analysis skills. The purpose of the current dissertation study was to compare the effectiveness and efficiency of (a) computer‐based training, (b) lecture formats with and (c) without the opportunity to pause, and (d) a no‐training group to teach visual analysis of AB graphs to university students. To make these formats directly comparable, the instructional content was equated by ensuring information and examples were identical across the three training procedures. Eighty‐three students were randomly assigned to one of the four groups. Results showed that all three training formats produced increases in accurate responding compared to the no‐training group. Visual analysis skills generalized to novel graphs and maintained approximately 2 weeks following all trainings. These results suggest that structured approaches that are carefully designed to train visual analysis are effective and lead to gains that generalize and maintain in the absence of training.     

Comparing Personality Test Formats and Warnings: Effects on criterion‐related validity and test‐taker reactions

The potential for applicant response distortion on personality measures remains a major concern in high‐stakes testing situations. Many approaches to understanding response distortion are too transparent (e.g., instructed faking studies) – or are too subtle (e.g., correlations with social desirability measures as indices of faking). Recent research reveals more promising approaches in two methods: using forced‐choice (FC) personality test items and warning against faking. The present study examined effects of these two methods on criterion‐related validity and test‐taker reactions. Results supported incremental validity for an FC and Likert‐scale measure in warning and no‐warning conditions, above and beyond cognitive ability. No clear differences emerged between the FC vs Likert measures or warning vs no‐warning conditions in terms of validity. However, some evidence suggested that FC measures and warnings may produce negative test‐taker reactions. We conclude with implications for implementation in selection settings.     

Surface‐ and deep‐level dissimilarity effects on social integration and individual effectiveness related outcomes in work groups: A meta‐analytic integration

Prior research linking demographic (e.g., age, ethnicity/race, gender, and tenure) and underlying psychological (e.g., personality, attitudes, and values) dissimilarity variables to individual group member's work‐related outcomes produced mixed and contradictory results. To account for these findings, this study develops a contingency framework and tests it using meta‐analytic and structural equation modelling techniques. In line with this framework, results showed different effects of surface‐level (i.e., demographic) dissimilarity and deep‐level (i.e., underlying psychological) dissimilarity on social integration, and ultimately on individual effectiveness related outcomes (i.e., turnover, task, and contextual performance). Specifically, surface‐level dissimilarity had a negative effect on social integration under low but not under high team interdependence. In return, social integration fully mediated the negative relationship between surface‐level dissimilarity and individual effectiveness related outcomes under low interdependence. In contrast, deep‐level dissimilarity had a negative effect on social integration, which was stronger under high and weaker under low team interdependence. Contrary to our predictions, social integration did not mediate the negative relationship between deep‐level dissimilarity and individual effectiveness related outcomes but suppressed positive direct effects of deep‐level dissimilarity on individual effectiveness related outcomes. Possible explanations for these counterintuitive findings are discussed.     

An alternative to Cohen's standardized mean difference effect size: a robust parameter and confidence interval in the two independent groups case

The authors argue that a robust version of Cohen's effect size constructed by replacing population means with 20% trimmed means and the population standard deviation with the square root of a 20% Winsorized variance is a better measure of population separation than is Cohen's effect size. The authors investigated coverage probability for confidence intervals for the new effect size measure. The confidence intervals were constructed by using the noncentral t distribution and the percentile bootstrap. Over the range of distributions and effect sizes investigated in the study, coverage probability was better for the percentile bootstrap confidence interval.