The biasing effects of selection and attrition on estimating the mean

Organizational and validation researchers often work with data that has been subjected to selection on the predictor and attrition on the criterion. These researchers often use the data observed under these conditions to estimate either the predictor or criterion's restricted population means. We show that the restricted means due to direct or indirect selection are a function of the population means plus the selection ratios. Thus, any difference between selected mean groups reflects the population difference plus the selection ratio difference. When there is also attrition on the criterion, the estimation of group differences becomes even more complicated. The effect of selection and attrition induces measurement bias when estimating the restricted population mean of either the predictor or criterion. A sample mean observed under selection and attrition does not estimate either the population mean or the restricted population mean. We propose several procedures under normality that yield unbiased estimates of the mean. The procedures focus on correcting the effects of selection and attrition. Each procedure was evaluated with a Monte Carlo simulation to ascertain its strengths and weaknesses. Given appropriate sample size and conditions, we show that these procedures yield unbiased estimators of the restricted and unrestricted population means for both predictor and criterion. We also show how our findings have implications for replicating selected group differences.     

Age group differences and longitudinal changes in approach–avoidance sensitivity: Findings from an 8-year longitudinal study

An increasing number of studies point to normative changes in personality occurring during adulthood. We examined age group differences and longitudinal changes in approach–avoidance sensitivities across adulthood using a population-based sample of younger, midlife, and older adults (N = 7468) assessed on three occasions over an 8 year interval. Younger adults reported higher approach sensitivity (assessed using BAS-Drive and BAS-Reward subscales) relative to midlife and older adults. Approach sensitivity tended to decline between Times 1 and 3 for all age groups. Cross-sectional age differences were not evident for avoidance sensitivity at Time 1 (assessed using the BIS); however there was evidence for different patterns of change in avoidance over time across the age groups. Results are discussed in the context of lifespan developmental perspectives on self-regulation of emotion and behaviour.     

Modeling competence development in the presence of selection bias

A major challenge for representative longitudinal studies is panel attrition, because some respondents refuse to continue participating across all measurement waves. Depending on the nature of this selection process, statistical inferences based on the observed sample can be biased. Therefore, statistical analyses need to consider a missing-data mechanism. Because each missing-data model hinges on frequently untestable assumptions, sensitivity analyses are indispensable to gauging the robustness of statistical inferences. This article highlights contemporary approaches for applied researchers to acknowledge missing data in longitudinal, multilevel modeling and shows how sensitivity analyses can guide their interpretation. Using a representative sample of N = 13,417 German students, the development of mathematical competence across three years was examined by contrasting seven missing-data models, including listwise deletion, full-information maximum likelihood estimation, inverse probability weighting, multiple imputation, selection models, and pattern mixture models. These analyses identified strong selection effects related to various individual and context factors. Comparative analyses revealed that inverse probability weighting performed rather poorly in growth curve modeling. Moreover, school-specific effects should be acknowledged in missing-data models for educational data. Finally, we demonstrated how sensitivity analyses can be used to gauge the robustness of the identified effects.     

„Intention to treat“ oder „treatment as received“ – Umgang mit Abbrechern in der Forschung zur Straftäterbehandlung

Lack of internal validity is a frequent problem in evaluation studies of offender treatment, as randomized controlled trials are difficult to implement and comparison groups in quasi-experimental studies are often not completely equivalent. Special matching procedures or statistical control of selection bias by using potentially interfering variables as covariates can provide a solution to some of these problems; however, a high dropout rate from offender treatment programs presents another problem, as the variables that cause attrition are often risk factors for reoffending. It is argued that a treatment-as-received (TR) approach which compares treatment completers with the comparison group may confound genuine treatment effects with selection effects. Particularly high recidivism rates in dropouts may be due to negative effects of attrition (e.g. stigmatization) but may more likely reflect a higher a priori risk for reoffending. Using data from some recent treatment studies, the authors show that alleged positive effects often disappear when a more adequate intention-to-treat (ITT) approach is used in which dropouts are retained in the treatment sample.     

Studies involving fathers: Subject refusal, attrition and sampling bias

The reported rates of subject refusal and attrition are reviewed in those studies where data were collected directly from fathers. Only 34 per cent of such studies report on subject refusal rates and 36 per cent on attrition rates. Few studies comment on or give information that would allow an assessment to be made of sampling bias arising from subject refusal and attrition. Nevertheless, studies reporting subject refusal and attrition suggest that fathers are no more difficult to recruit than mothers and that a variety of factors, individually and in combination, may have some impact on recruitment. These include the type of commitment required, the techniques of recruitment used and whether studies are hospital, home or laboratory based. However, if biases due to subject refusal and attrition are to be identified and avoided, studies need to be more open in providing information about the characteristics of refusing and participating subjects. To encourage this, some new data are presented, suggesting a number of trends: (1) extended personal contacts with potential subjects may minimize subject refusal and attrition; (2) attrition may be somewhat higher in middle-class families and families with sons.     

School‐based humanistic counselling for psychological distress in young people: A practice research network to address the attrition problem

Aims: School‐based humanistic counselling (SBHC) is a common psychological intervention for young people, particularly in the UK. However, studies have tended to suffer from high attrition rates, such that effects may have been over‐estimated. This paper describes a low budget ‘star topology’ practice research network (PRN) outcome evaluation of SBHC in a sample where attrition rates were minimised. Design: A practice‐based longitudinal study in a small PRN compared levels of psychological distress at first and last session from session‐by‐session data. Multilevel regression modelling was used to identify predictors of outcomes. Methods: Eight counsellors working across 11 schools agreed to use session‐by‐session self‐rating on the Young Person's CORE (YP‐CORE) yielding data from 256 young people aged 11 to 17. Predictors of outcomes were analysed using multilevel regression analysis. Results: Mean levels of distress on the YP‐CORE reduced from 18.29 (SD = 7.32) at baseline to 9.10 (SD = 6.19) at endpoint, giving a baseline to endpoint effect size of 1.26 (95% CI = 1.06–1.46). Lower levels of psychological distress at endpoint were associated with male clients, younger age groups, greater rates of attendance at counselling, and bereavement as a presenting problem. Conclusions: This ‘star topology’ PRN focused on a single study and demonstrated that such a system can provide relatively low cost, high quality data. The data showed that SBHC is associated with large reductions in psychological distress, and that this cannot be attributed to the high attrition rates of previous datasets.     

The fate of cognition in very old age: six-year longitudinal findings in the Berlin Aging Study (BASE)

The authors report full-information longitudinal age gradients in 4 intellectual abilities on the basis of 6-year longitudinal changes in 132 individuals (mean age at T1 = 78.27, age range = 70-100) from the Berlin Aging Study. Relative to the cross-sectional parent sample (N = 516, mean age at T1 = 84.92 years), this sample was positively selected because of differential mortality and experimental attrition. Perceptual speed, memory, and fluency declined with age. In contrast, knowledge remained stable up to age 90, with evidence for decline thereafter. Age gradients were more negative in old old (n = 66, mean age at T1 = 83.04) than in old (n = 66, mean age at T1 = 73.77) participants. Rates of decline did not differ reliably between men and women or between participants with high versus low life-history status. They conclude that intellectual development after age 70 varies by distance to death, age, and intellectual ability domain.     

Stability of deferred imitation in 12- to 18-month-old infants: A closer look into developmental dynamics

Deferred imitation is used to assess declarative memory in infants. Although a lot of studies show that infants from 6 months onwards are able to re-enact actions following a delay, only a few studies describe early declarative memory longitudinally. From a variable-centred approach, these studies found a modest relationship between measurement occasions that increases with age. However, no studies have analysed the differential aspect of memory development from a person-centred perspective. The present study analyses memory stabilities both from a variable-centred and person-centred approach within a sample of N = 87 infants of a 2-wave longitudinal design (12- and 18-month-olds). From a variable-centred perspective, the results indicate that, first, there was a significant increase in infants' memory performance and, second, that although reliable, the stability of infants' memory performance was relatively low. From a person-centred perspective two vs. three groups of infants were differentiated showing different developmental growth trajectories and stability correlations but no differences in language, cognitive and motor development. The implications of those results in terms of further test development are discussed. Furthermore, important methodological expansions for the analysis of infants' memory data are presented and discussed.     

Are apparent sex differences in mean IQ scores created in part by sample restriction and increased male variance?

This study investigated the possibility that apparent sex differences in IQ are at least partly created by the degree of sample restriction from the baseline population. We used a nationally representative sample, the 1970 British Cohort Study. Sample sizes varied from 6518 to 11,389 between data-collection sweeps. Principal components analysis of scores obtained on four cognitive tests administered at age 10 was used to obtain estimates that we name ‘IQ’. These age-10 scores were then used to estimate the sex differences at age 10, and also among participants in the two later waves, at age 26 and 30. At age 10, there was a small but significant advantage for boys (Cohen’s d = 0.081). Boys had greater variability in these IQ scores. We then investigated how this very small male advantage at 10 changed with sample restriction. We used the same IQs obtained at age 10, but considered only those subjects who returned for data-collection sweeps at ages 26 and 30 years. Subjects returning at age 26 and 30 were more likely to be females and to have higher age-10 IQ scores. Attrition at age 30 was 28% and the male advantage in IQ scores increased by 15%. Attrition at age 26 was 43% and the male advantage in IQ scores increased by 48%. The findings underline the importance of monitoring attrition in longitudinal studies, as well as emphasising the need for representative samples in studying sex differences in intelligence. A proportion of the apparent male advantage in general cognitive ability that has been reported by some researchers might be attributable to the combination of greater male variance in general cognitive ability and sample restriction, though this remains to be tested in a sample with an appropriate mental test battery.     

The association between dispositional self-control and longitudinal changes in eating behaviors,diet quality,and BMI

In a random sample from the general population (N = 2781, 46% males), a longitudinal survey was conducted. The association between dispositional self-control and changes in eating behaviours and diet quality was analysed between the first wave (2010) and the last wave (2014). Results show that the higher the dispositional self-control, the lower the increase in overeating behaviours (emotional eating, external eating, ambivalence towards palatable food and overeating), and BMI and the larger the improvement in healthy diet over time. Self-control was not associated with changes in dietary restraint. This is one of the first studies suggesting that dispositional self-control is associated with changes in eating behaviours and healthy food intake over time.     

Intelligence and selective attrition in a nationally representative and longitudinal sample of Americans

Selective attrition is a potentially serious problem that can bias the findings that are generated from longitudinal samples. Although the extent to which IQ might be involved in selective attrition has been studied, the results from these studies have been decidedly mixed. The current study adds to the literature examining the link between IQ and selective attrition by examining four waves of data from the National Longitudinal Study of Adolescent Health. Analysis of these data revealed that IQ is related to attrition, with lower IQ respondents more likely to drop out of the study when compared against higher IQ respondents. The evidence suggests that dropouts score about 4.5 IQ points lower than subjects who did not drop out. Limitations of the study are noted and directions for future research are discussed.     

Cross-Sectional versus Longitudinal Age Gradients of Tower of Hanoi Performance: The Role of Practice Effects and Cohort Differences in Education

ABSTRACT

We examined 5-year longitudinal changes in Tower of Hanoi (TOH) performance in a population-based sample of adults (35–85 years initially; n?=?1480). An age-matched sample (n?=?433) was included to estimate practice effects. The longitudinal age gradients differed substantially from the cross-sectional age gradients. This was the case even when practice effects, that were substantial in magnitude across the young/middle-aged groups, were controlled for. Instead of a continuous age-related deficit in performance from 35 and onwards, longitudinal data showed slowing of performance and increases of illegal moves past age 65. Cohort-related differences in educational attainment did not account for this discrepancy. Further analyses revealed a positive relation between practice-related gains and explicit memory of having performed the task at the first test occasion and a positive association between latent changes in TOH and Block Design, in line with cross-sectional findings. In conclusion, the results demonstrate a pattern of age-related changes indicating a late-onset decline of TOH performance and underscore the need to control for retest effects in longitudinal aging research.     

The effects of attrition on baseline comparability in randomized experiments in education: a meta-analysis

Using meta-analysis, randomized experiments in education that either clearly did or clearly did not experience student attrition were examined for the baseline comparability of groups. Results from 35 studies suggested that after attrition, the observed measures of baseline comparability of groups did not differ more than would be expected given sampling error. The degree of either overall or differential attrition did not relate to baseline comparability, a finding that held under sensitivity analyses. Also, both overall and differential attrition rates were unrelated to posttest effect sizes. All of these analyses, however, lacked sufficient statistical power to detect small but potentially meaningful effects. Results suggest caution is warranted when applying quality scales and other blanket rules pertaining to attrition that are meant to either serve as inclusion-exclusion criteria or in scoring study quality. Much greater attention is needed to both the reporting of attrition in primary studies and to the development of conceptual and empirical models of the attrition process. These developments would aid further investigation of the relation between attrition and study outcomes.     

Parent predictors of changes in child behavior problems

The present study examined whether ineffective discipline, single parent status, social support, parent involvement, and parent depression predicted changes in preschoolers' (N = 129) behavior problems. This study also evaluated whether child sex and ethnicity moderated the relations between these variables and changes in problem behavior. Parents completed questionnaires at the beginning of the study, and parent, teacher, and observational ratings of children's behavior problems were collected twice during the school year. Parents' own social support predicted improvement for boys and parent depression was associated with worsening symptoms for girls. Single parenthood and parent involvement predicted changes in behavior problems for the sample as a whole. Several significant ethnic differences emerged, highlighting the importance of considering cultural context in studies of parenting and child externalizing behavior.     

Gender and Teachers' Attrition: The Occupational Destination of Former Teachers

Based on the resources–rewards model and on the sex segregation perspective, the study was designed to examine gender differences in teachers' attrition by comparing three groups: teachers who remained in the educational system, teachers who left the labor market, and teachers who left teaching and entered other occupations. The study also addressed the type of occupations into which former teachers moved. The sample was 2,108 teachers who participated in the censuses of 1983 and 1995 and are represented in the inter-generation dataset of the Israeli Central Bureau of Statistics. The findings indicated that one-third of the teachers did not continue teaching. Most of them moved to other occupations. Gender was found to be related to teachers' attrition and to their occupational destination. The data support the sex segregation perspective and suggest that working in a female-dominated occupation has different implications for women's and men's attrition and patterns of changing occupations, which contribute to maintaining the sex segregation structure in the labor market.     

Unravelling the Interplay Between Genetic and Environmental Contributions in the Unfolding of Personality Differences from Early Adolescence to Young Adulthood

In two studies, we examined the genetic and environmental sources of the unfolding of personality trait differences from childhood to emerging adulthood. Using self‐reports from over 3000 representative German twin pairs of three birth cohorts, we could replicate previous findings on the primary role of genetic sources accounting for the unfolding of inter‐individual differences in personality traits and stabilizing trait differences during adolescence. More specifically, the genetic variance increased between early (ages 10–12 years) and late (ages 16–18 years) adolescence and stabilized between late adolescence and young adulthood (ages 21–25 years). This trend could be confirmed in a second three‐wave longitudinal study of adolescents' personality self‐reports and parent ratings from about 1400 Norwegian twin families (average ages between 15 and 20 years). Moreover, the longitudinal study provided evidence for increasing genetic differences being primarily due to accumulation of novel genetic influences instead of an amplification of initial genetic variation. This is in line with cumulative interaction effects between twins' correlated genetic makeups and environmental circumstances shared by adolescent twins reared together. In other words, nature × nurture interactions rather than transactions can account for increases in genetic variance and thus personality variance during adolescence. © 2019 European Association of Personality Psychology     

Are synesthetes exceptional beyond their synesthetic associations? A systematic comparison of creativity,personality, cognition,and mental imagery in synesthetes and controls

Synesthesia has historically been linked with enhanced creativity, but this had never been demonstrated in a systematically recruited sample. The current study offers a broad examination of creativity, personality, cognition, and mental imagery in a small sample of systematically recruited synesthetes and controls (n = 65). Synesthetes scored higher on some measures of creativity, personality traits of absorption and openness, and cognitive abilities of verbal comprehension and mental imagery. The differences were smaller than those reported in the literature, indicating that previous studies may have overestimated group differences, perhaps due to biased recruitment procedures. Nonetheless, most of our results replicated literature findings, yielding two possibilities: (1) our study was influenced by similar biases, or (2) differences between synesthetes and controls, though modest, are robust across recruitment methods. The covariance among our measures warrants interpretation of these differences as a pattern of associations with synesthesia, leaving open the possibility that this pattern could be explained by differences on a single measured trait, or even a hidden, untested trait. More generally, this study highlights the difficulty of comparing groups of people in psychology, not to mention neuropsychology and neuroimaging studies. The requirements discussed here – systematic recruitment procedures, large battery of tests, and large cohorts – are best fulfilled through collaborative efforts and cumulative science.     

Mortality-Related Differences and Changes in Episodic Memory Among the Oldest Old: Evidence From a Population-Based Sample of Nonagenarians

We examined cross-sectional differences and longitudinal changes in episodic memory performance related to impending death among a group of very old people, aged 90–101 years. Participants were assessed at 3 measurement points across a 6-year interval. Three groups were identified: those who survived the entire follow-up period (n =40), those who died before the first follow-up (n =44), and those who died after the first follow-up (n =14). Participants completed a battery of episodic memory tasks consisting of face recognition, word recognition, word recall, and object recall with selective reminding. Those who survived performed better than those who were going to die in object recall at baseline. A Cox regression analysis, controlling age, revealed that object recall performance was significantly related to subsequent mortality status. Longitudinal analyses demonstrated significant 3-year decline for both face recognition and object recall, but no evidence of differential decline as a function of mortality group. Thus, longitudinal changes in memory preceding death were not as pronounced as the corresponding cross-sectional differences in this very old sample. In general, the results suggest that mortality-related memory deficits are present in extreme old age, although these deficits are relatively small and task-specific.     

Research design issues for the use of magnetic resonance imaging machines in brain studies of psychological/psychiatric variables

The magnetic resonance imaging (MRI) machine itself has an impact on the likelihood of obtaining successful measurements of brain size in certain groups of subjects. The differential selection and attrition in both cross-sectional and longitudinal designs, therefore, indicate that the MRI coincidentally serves as a screen for the anatomical structure of the brains that are successfully scanned. This screening effect introduces confounds in experiments whose very hypotheses are focused on comparing anatomical differences in subjects who differ, for example, in their reactions to anxiety-inducing situations. Here, behavioral interventions and possible statistical models are presented in order to reduce attrition and other effects of the confounds introduced by the MRI measurement process in research. Child and adolescent research—particularly in the attention-deficit/hyperactivity disorder research area—is used as an example to clarify and delineate the general research principles presented in the present article.