Statistical power in randomized intervention studies with noncompliance

This study examined various factors that affect statistical power in randomized intervention studies with noncompliance. On the basis of Monte Carlo simulations, this study demonstrates how statistical power changes depending on compliance rate, study design, outcome distributions, and covariate information. It also examines how these factors influence power in different methods of estimating intervention effects. Intent-to-treat analysis and complier average causal effect estimation are compared as 2 alternative ways of estimating intervention effects under noncompliance. The results of this investigation provide practical implications in designing and evaluating intervention studies taking into account noncompliance.     

Efficacy of Cognitive Rehabilitation Therapies for Mild Cognitive Impairment (MCI) in Older Adults: Working Toward a Theoretical Model and Evidence-Based Interventions

To evaluate the efficacy of cognitive rehabilitation therapies (CRTs) for mild cognitive impairment (MCI). Our review revealed a need for evidence-based treatments for MCI and a lack of a theoretical rehabilitation model to guide the development and evaluation of these interventions. We have thus proposed a theoretical rehabilitation model of MCI that yields key intervention targets–cognitive compromise, functional compromise, neuropsychiatric symptoms, and modifiable risk and protective factors known to be associated with MCI and dementia. Our model additionally defines specific cognitive rehabilitation approaches that may directly or indirectly target key outcomes–restorative cognitive training, compensatory cognitive training, lifestyle interventions, and psychotherapeutic techniques. Fourteen randomized controlled trials met inclusion criteria and were reviewed. Studies markedly varied in terms of intervention approaches and selected outcome measures and were frequently hampered by design limitations. The bulk of the evidence suggested that CRTs can change targeted behaviors in individuals with MCI and that CRTs are associated with improvements in objective cognitive performance, but the pattern of effects on specific cognitive domains was inconsistent across studies. Other important outcomes (i.e., daily functioning, quality of life, neuropsychiatric symptom severity) were infrequently assessed across studies. Few studies evaluated long-term outcomes or the impact of CRTs on conversion rates from MCI to dementia or normal cognition. Overall, results from trials are promising but inconclusive. Additional well-designed and adequately powered trials are warranted and required before CRTs for MCI can be considered evidence-based.     

Using Multilevel Mixtures to Evaluate Intervention Effects in Group Randomized Trials

There is evidence to suggest that the effects of behavioral interventions may be limited to specific types of individuals, but methods for evaluating such outcomes have not been fully developed. This study proposes the use of finite mixture models to evaluate whether interventions, and, specifically, group randomized trials, impact participants with certain characteristics or levels of problem behaviors. This study uses latent classes defined by clustering of individuals based on the targeted behaviors and illustrates the model by testing whether a preventive intervention aimed at reducing problem behaviors affects experimental users of illicit substances differently than problematic substance users or those individuals engaged in more serious problem behaviors. An illustrative example is used to demonstrate the identification of latent classes, specification of random effects in a multilevel mixture model, independent validation of latent classes, and the estimation of power for the proposed models to detect intervention effects. This study proposes specific steps for the estimation of multilevel mixture models and their power and suggests that this model can be applied more broadly to understand the effectiveness of interventions.     

The Impact of Covariates on Statistical Power in Cluster Randomized Designs: Which Level Matters More?

Field experiments with nested structures are becoming increasingly common, especially designs that assign randomly entire clusters such as schools to a treatment and a control group. In such large-scale cluster randomized studies the challenge is to obtain sufficient power of the test of the treatment effect. The objective is to maximize power without adding many clusters that make the study much more expensive. In this article I discuss how power estimates of tests of treatment effects in balanced cluster randomized designs are affected by covariates at different levels. I use third-grade data from Project STAR, a field experiment about class size, to demonstrate how covariates that explain a considerable proportion of variance in outcomes increase power significantly. When lower level covariates are group-mean centered and clustering effects are larger, top-level covariates increase power more than lower level covariates. In contrast, when clustering effects are smaller and lower level covariates are grand-mean centered or uncentered, lower level covariates increase power more than top-level covariates.     

Evaluating Group-Based Interventions When Control Participants Are Ungrouped

Individually randomized treatments are often administered within a group setting. As a consequence, outcomes for treated individuals may be correlated due to provider effects, common experiences within the group, and/or informal processes of socialization. In contrast, it is often reasonable to regard outcomes for control participants as independent, given that these individuals are not placed into groups. Although this kind of design is common in intervention research, the statistical models applied to evaluate the treatment effects are usually inconsistent with the resulting data structure, potentially leading to biased inferences. This article presents an alternative model that explicitly accounts for the fact that only treated participants are grouped. In addition to providing a useful test of the overall treatment effect, this approach also permits one to formally determine the extent to which treatment effects vary over treatment groups and whether there is evidence that individuals within treatment groups become similar to one another. This strategy is demonstrated with data from the Reconnecting Youth program for high school students at risk of school failure and behavioral disorders.     

An Application of the Complier Average Causal Effect Analysis to Examine the Effects of a Family Intervention in Reducing Illicit Drug Use among High‐Risk Hispanic Adolescents

The Complier Average Causal Effect (CACE) method has been increasingly used in prevention research to provide more accurate causal intervention effect estimates in the presence of noncompliance. The purpose of this study was to provide an applied demonstration of the CACE analytic approach to evaluate the relative effects of a family‐based prevention intervention, Familias Unidas, in preventing/reducing illicit drug use for those participants who received the intended dosage. This study is a secondary data analysis of a randomized controlled trial designed to evaluate the relative efficacy of Familias Unidas with high‐risk Hispanic youth. A total of 242 high‐risk Hispanic youth aged 12–17 years and their primary caregivers were randomized to either Familias Unidas or Community Practice and assessed at baseline, 6 months and 12 months postbaseline. CACE models were estimated with a finite growth mixture model. Predictors of engagement were included in the CACE model. Findings indicate that, relative to the intent‐to‐treat (ITT) analytic approach, the CACE analytic approach yielded stronger intervention effects among both initially engaged and overall engaged participants. The CACE analytic approach may be particularly helpful for studies involving parent/family‐centered interventions given that participants may not receive the intended dosage. Future studies should consider implementing the CACE analysis in addition to ITT analysis when examining the effects of family‐based prevention programs to determine whether, and the extent to which, the CACE analysis has more power to uncover intervention effects.     

Matching methods for selection of subjects for follow-up

This work examines ways to make the best use of limited resources when selecting individuals to follow up in a longitudinal study estimating causal effects. In the setting under consideration, covariate information is available for all individuals but outcomes have not yet been collected and may be expensive to gather, and thus only a subset of the comparison subjects will be followed. Expressions in Rubin and Thomas (1996, 2000) show the benefits that can be obtained, in terms of reduced bias and variance of the estimated treatment effect, of selecting comparison individuals well-matched to those in the treated group, as compared to a random sample of comparison individuals. We primarily consider non-experimental settings but also consider implications for randomized trials. The methods are illustrated using data from the Johns Hopkins University Baltimore Prevention Program, which included data collection from age 6 to young adulthood of participants in an evaluation of two early elementary-school based universal prevention programs.     

Calibration of a color monitor for visual psychophysics

It has become common for stimuli used in visual psychophysical experiments to be presented on high-resolution color cathode-ray tubes (CRTs) such as the Barco CDCT 6551. These enable a flexibility of color, spatial-frequency content, temporal-frequency content, duration, size, and position that is not provided by most other media. CRTs are, however, not perfect; they suffer from the effects of temporal instability, spatial variability, lack of phosphor constancy, gun interdependence, and gun nonlinearity. This paper describes methods of assessing these aspects of monitor performance with respect to how significant each may be in psychometric terms. Although every application of CRT use in visual psychophysics is different, some general rules can be formulated to help ensure that unwanted effects are kept to a minimum. For the CRT used in this study (Barco CDCT 6551), a warm-up time of 30-45 min is necessary before chromatic and luminous stability ensues. Restriction of individual gun outputs to within 10%-90% of the possible range ensures that the effects of gun interdependence and lack of phosphor constancy are negligible. Calibration methods dealing with the linearization of gun output are also discussed.     

There are stable individual differences in performance variability, both from moment to moment and from day to day

Individual differences in decision speed have been regarded as direct reflections of a "primitive" functional neurophysiological characteristic, which affects performance on all cognitive tasks and so may be regarded as the "biological basis of intelligence", or of age-related changes in mental abilities. More detailed analyses show that variability within an experimental session (WSV) is a stable individual difference characteristic and that mean choice reaction times (CRTs) are gross summary statistics that reflect variability, rather than maximum speed of performance. A total of 98 people aged from 60 to 80 years completed 36 weekly sessions on six different letter categorization tasks. After effects of practice and of circadian variability had been eliminated, individuals with lower scores on the Cattell Culture Fair intelligence test had slower CRTs and greater WSV on all tasks. A simulation study showed that the greater WSVs of low Cattell scorers led directly to the significantly greater variability of their mean CRTs from session to session. However because CRTs on tasks co-varied from session to session it was apparent that, besides being affected by WSV, individuals' between-session variabilities (BSVs) also vary because of state changes that affect their performance from day to day. It seems that both variability in performance from trial to trial during a session and variability in average performance from day to day are correlated, stable, individual difference characteristics that vary inversely with intelligence test performance. Methodological consequences of these results for interpretations of age-related cognitive changes, for variability between as well as within individuals, for individual differences in decision speed, and for circadian variability in performance are discussed.     

Polytomous multilevel testlet models for testlet‐based assessments with complex sampling designs

A pplications of standard item response theory models assume local independence of items and persons. This paper presents polytomous multilevel testlet models for dual dependence due to item and person clustering in testlet‐based assessments with clustered samples. Simulation and survey data were analysed with a multilevel partial credit testlet model. This model was compared with three alternative models – a testlet partial credit model (PCM), multilevel PCM, and PCM – in terms of model parameter estimation. The results indicated that the deviance information criterion was the fit index that always correctly identified the true multilevel testlet model based on the quantified evidence in model selection, while the Akaike and Bayesian information criteria could not identify the true model. In general, the estimation model and the magnitude of item and person clustering impacted the estimation accuracy of ability parameters, while only the estimation model and the magnitude of item clustering affected the item parameter estimation accuracy. Furthermore, ignoring item clustering effects produced higher total errors in item parameter estimates but did not have much impact on the accuracy of ability parameter estimates, while ignoring person clustering effects yielded higher total errors in ability parameter estimates but did not have much effect on the accuracy of item parameter estimates. When both clustering effects were ignored in the PCM, item and ability parameter estimation accuracy was reduced.     

Estimating the causal effect of randomization versus treatment preference in a doubly randomized preference trial

Although randomized studies have high internal validity, generalizability of the estimated causal effect from randomized clinical trials to real-world clinical or educational practice may be limited. We consider the implication of randomized assignment to treatment, as compared with choice of preferred treatment as it occurs in real-world conditions. Compliance, engagement, or motivation may be better with a preferred treatment, and this can complicate the generalizability of results from randomized trials. The doubly randomized preference trial (DRPT) is a hybrid randomized and nonrandomized design that allows for estimation of the causal effect of randomization versus treatment preference. In the DRPT, individuals are first randomized to either randomized assignment or choice assignment. Those in the randomized assignment group are then randomized to treatment or control, and those in the choice group receive their preference of treatment versus control. Using the potential outcomes framework, we apply the algebra of conditional independence to show how the DRPT can be used to derive an unbiased estimate of the causal effect of randomization versus preference for each of the treatment and comparison conditions. Also, we show how these results can be implemented using full matching on the propensity score. The methodology is illustrated with a DRPT of introductory psychology students who were randomized to randomized assignment or preference of mathematics versus vocabulary training. We found a small to moderate benefit of preference versus randomization with respect to the mathematics outcome for those who received mathematics training.     

心理学实验的可重复性

严格意义的实验可重复性指的是实验控制条件不变的情况下其结果的可重复性,置信区间是表示这种可重复性的恰当方法,可重复性的提高可通过在实验设计和数据分析中将影响因变量的随机额外变量作为协变量引入来实现;另一种意义的可重复性指的是实验结果的可迁移性,它涉及当控制条件变化时因控制变量和自变量的交互作用而导致的实验结果的变化。在这两种意义下,心理学实验较低的可重复性都源于它的额外变量的庞杂。     

Who enrolls in prevention trials? Discordance in perception of risk by professionals and participants

Internal and external validity problems permeate all intervention studies but are accentuated in primary preventive intervention research, particularly when studies target or recruit individuals based on their risk for psychopathology. Since many people who are at risk do not yet experience distress, they may not perceive the need for intervention. Recruitment tactics based on explaining extent of risk are unlikely to be persuasive and may have negative consequences. If respondents are not motivated to participate, a small or biased subset of the target population will participate in the intervention. Bias is of special concern when those enrolled represent only part of the continuum of risk. Selective enrollment may compromise both internal validity (the interpretation of the research results) and external validity (the generalizability of the findings) of intervention trials in primary prevention. This article discusses the effects of partial enrollment and the resultant bias. It suggests several strategies for increasing the enrollment of the target population and examines some of their ethical ramifications. It also stresses the importance of collecting systematic data documenting how the participants in the intervention differ from the target group as a whole.     

Alcohol-Induced Amnesia and Personalized Drinking Feedback: Blackouts Predict Intervention Response

Alcohol-induced amnesia (“blackout”) is a reliable predictor of alcohol-related harm. Given its association with other negative consequences, experience of alcohol-induced amnesia may serve as a teachable moment, after which individuals are more likely to respond to intervention. To test this hypothesis, alcohol-induced amnesia was evaluated as a moderator of brief intervention effect on (a) alcohol-related consequences and (b) the proposed intervention mediators, protective behavioral strategies and peak blood alcohol concentration (BAC). Baseline alcohol risk measured using the Alcohol Use Disorders Identification Test (AUDIT) was also evaluated as a moderator to rule out the possibility that amnesia is simply an indicator of more general alcohol risk. College students (N = 198) reporting alcohol use in a typical week completed assessments at baseline and 1-month follow-up as part of a larger intervention trial. Participants were randomized to assessment only (AO; n = 58) or personalized feedback intervention (PFI; n = 140). Hierarchical regression was used to examine direct and indirect intervention effects. A significant group-by-amnesia interaction revealed that only PFI participants who had experienced alcohol-induced amnesia in the past month reported decreases in alcohol consequences at 1-month follow-up. The PFI reduced alcohol-related consequences indirectly through changes in peak BAC, but only among those who had experienced amnesia at baseline. In contrast, baseline alcohol risk (AUDIT) did not moderate intervention effects, and use of protective behavioral strategies did not statistically mediate intervention effects. Findings suggest that loss of memory for drinking events is a unique determinant of young adult response to brief alcohol intervention. Normative feedback interventions may be particularly effective for individuals who have experienced alcohol-induced amnesia in the past 30 days.     

Character Strengths Interventions: Building on What We Know for Improved Outcomes

For this review strengths intervention studies were located using online searches and collegial networks and included if they explicitly sought to teach or use a strengths classification to enhance well-being, and used pre- and post-intervention measures and a comparison group. Eight studies met the criteria and have been summarised by this review. To date, the effect sizes achieved by character strengths interventions have been small to moderate. An understanding of how these interventions work may facilitate development of more effective interventions, while expanding the field of character strengths interventions to include a broader range of activities and approaches may also offer benefits. Research examining individual factors, such as strengths use, psychological need satisfaction, goal-setting and goal-striving provides promising leads to explain how strengths interventions work. However, the effect on intervention efficacy of relational or contextual factors, such as intervention environment or facilitator attitude to strengths, has not yet been explored. Implications for interventions in school settings are considered.     

Understanding Change in Recycling and Littering Behavior Across a School Social Network

Understanding how communities change requires examining how individuals’ beliefs and behaviors are shaped by those around them. This paper investigates behavior change across a large social network following a recycling intervention in a New Zealand high school community. We used a mixed methods design, combining focus group data with social network analysis from two waves of a questionnaire that measured friendship networks; recycling and littering behaviors; perceived behavioral norms; and teacher, friend, and parent encouragement for these behaviors. Recycling behavior increased significantly over the course of our study. Supporting the importance of social networks in this context, both littering and recycling behavior showed clear social clustering. Further, the degree of change in an individuals’ littering and recycling behavior across time was predicted by friends’ prior behavior. Focus group data provided insight into students’ perceptions of social interactions and how these contributed to littering and recycling behavior.     

“Hit Me Baby”: From Britney Spears to the Socialization of Sexual Objectification of Girls in a Middle School Drama Program

This language socialization study integrates ethnographic and intertextual methods of data collection and analysis to examine how one middle school drama class’s performance of Britney Spears’s first hit song, originally titled “Hit Me Baby One More Time,” exemplifies not only how sexually charged media can contribute to the normalization of sexist, abusive, and thus also violent behavior toward women, but also how local caretaking adults can contribute to these socialization practices even within the context of official educational activities. Prior studies related to the socialization of gender equality and sexual abuse prevention in educational institutions have focused on whether and how adult intervention may prevent or stop gender and thus also sexually related abuse. This study indicates that further research into adult complicity and the need for intervention into adult behavior may also be called for. The ethnographic fieldwork for this paper was conducted during a larger language socialization study at a middle school in the western United States. This included the videotaping of rehearsals and performances by middle school students of popular songs. The intertextual data chosen for analysis is based on these ethnographic observations. The conclusion that some adults are actively socializing female sexual objectification and male dominance during school-based activities is based on observations of these locally occurring interactions.     

Survey Satisficing Biases the Estimation of Moderation Effects

Survey satisficing in online data collection biases the estimation of treatment effects in many ways. Extending the findings of a previous study, which demonstrated that satisficing biased the estimation of main treatment effects, this study also shows that satisficing distorts the estimation of moderation effects. Targeting Japanese adults’ attitudes toward food, this study tests how the effect of country of production (Japan vs. China) is moderated by preexisting ethnocentric attitudes. The results show that while nonsatisficers predictably adjust their attitude toward food based on their preexisting ethnocentric attitude, satisficers stick to their initial stereotypical response. That is, the theoretically predicted moderation effect was observed among nonsatisficers, but not among satisficers, which indicates that satisficing biases not only the estimation of a treatment effect but also that of a moderation effect.     

When necessity is the mother of invention: Mood and problem solving

Happy moods foster the ability to think about problems in new ways, but little is known about how sad moods affect this process. This paper investigates the hypothesis that individuals in sad moods adhere to the data and might not think about problems in new ways until they receive data indicating that it is appropriate to do so. To investigate this hypothesis, participants in happy, sad, and neutral moods completed a classic mental set task (Luchins, 1942). All mood groups were able to break the mental set and to think about the problems in a new way, but mood affected when they did so. Consistent with the idea that individuals in sad moods adhere to the data, they relied on the mental set until they received evidence that it may be problematic. In contrast, individuals in happier moods were more likely to abandon the mental set on their own, rather than wait for evidence of its inadequacy. In three follow-up experiments, when the information provided by participants’ sad feelings was rendered uninformative, the mood effect disappeared. These findings are consistent with the claim that affect influences processing only when it provides information about how to proceed.     

任务框架和序列趋势对趋势阻尼的影响

以往研究发现,下降序列的趋势阻尼大于上升序列,但该结论可能掺杂了任务框架这一额外变量。基于此实验1选取可控事件,实验2选取不可控事件作为背景,探讨了任务框架和序列趋势对趋势阻尼的影响。研究发现：不管在可控事件还是不可控事件中,序列趋势对趋势阻尼的影响都受到任务框架的调节,收益框架中下降序列的阻尼大于上升序列,损失框架中两者差异不显著。