Estimating the causal effect of randomization versus treatment preference in a doubly randomized preference trial

Although randomized studies have high internal validity, generalizability of the estimated causal effect from randomized clinical trials to real-world clinical or educational practice may be limited. We consider the implication of randomized assignment to treatment, as compared with choice of preferred treatment as it occurs in real-world conditions. Compliance, engagement, or motivation may be better with a preferred treatment, and this can complicate the generalizability of results from randomized trials. The doubly randomized preference trial (DRPT) is a hybrid randomized and nonrandomized design that allows for estimation of the causal effect of randomization versus treatment preference. In the DRPT, individuals are first randomized to either randomized assignment or choice assignment. Those in the randomized assignment group are then randomized to treatment or control, and those in the choice group receive their preference of treatment versus control. Using the potential outcomes framework, we apply the algebra of conditional independence to show how the DRPT can be used to derive an unbiased estimate of the causal effect of randomization versus preference for each of the treatment and comparison conditions. Also, we show how these results can be implemented using full matching on the propensity score. The methodology is illustrated with a DRPT of introductory psychology students who were randomized to randomized assignment or preference of mathematics versus vocabulary training. We found a small to moderate benefit of preference versus randomization with respect to the mathematics outcome for those who received mathematics training.     

Rehabilitation of executive disorders after brain injury: Are interventions effective?

In this paper, the effectiveness of interventions for executive disorders was reviewed. The objective was to evaluate the internal and external validity of intervention studies. A total of 46 papers, describing 54 studies, conducted in the last two decades meeting several preset inclusion criteria, was included in this review. The studies were categorized into three treatment approaches in order to enhance comparability. The overall results show that many interventions yield positive outcomes and seem to be effective in reducing executive problems in brain‐injured subjects. However, several studies have only an explorative intent and are based on less sophisticated experimental designs. The verification of their results is generally more tenuous. The internal validity, or the set‐up of experimental conditions necessary to draw valid conclusions about treatment effectiveness, including the choice of well‐matched control groups, or the randomization of patients over treatment and control conditions, is not always strong. The same conclusion can be drawn for the external validity of a number of the intervention studies; often evidence of generalization to real‐life situations, long‐term follow‐up, and transfer to non‐trained situations, were (partially) lacking in the studies under review. The authors are aware that the design of proper randomized controlled trials for the investigation of the treatment effectiveness of executive disorders is cumbersome and time consuming. Nonetheless, the provisional results of several well‐designed studies described in this review make the effort worthwhile.     

Watching your troubles away: television viewing as a stimulus for subjective self-awareness

Three studies explored the role of television viewing in eliciting subjective self-awareness and positive self-feelings. Study 1 assessed the effects of self-awareness manipulations via exposure to a neutral television program on actual-ideal discrepancies. Those who watched television showed significantly smaller self-discrepancies than those who did not, independent of mood. Study 2 demonstrated the ecological validity of this finding by replicating it with people watching television in their own homes. Study 3 investigated whether manipulations of self-feelings affected television watching. Results indicated that those who received failure feedback watched television longer than those in a control condition who likewise watched television longer than those who received success feedback. Television appears to be an effective stimulus to direct the focus away from oneself and to render people less aware of how they are falling short of their standards.     

Different randomized multiple-baseline models for different situations: A practical guide for single-case intervention researchers

Single-case researchers often implement multiple-baseline designs as their preferred methodology for intervention evaluations. Recent writings and empirical investigations have argued in favor of incorporating various forms of randomization into such designs for the purpose of elevating the intervention study's internal validity and scientific credibility. In this article, we consider a variety of randomized multiple-baseline designs and associated randomization statistical tests, along with their potential strengths and limitations. In what amounts to a practical guide, we refer school psychology researchers to these versatile randomization procedures for planning and executing their intervention studies.     

An Evaluation of the Cross-Cultural Validity of Holland's Theory: Career Choices by Workers in India

The current study evaluated the cross-cultural validity of Holland's (1985) theory through internal and external analyses based on the responses of 172 natives of India who completed the Vocational Preference Inventory (VPI) and measures of job and occupational satisfaction. Internal consistency estimates for Holland dimensions, intercorrelations among participants' vocational interest scales, and the results of a randomization test (Tracey, 1997) all provide evidence for the internal structure of the VPI with Indians. However, the VPI did not exhibit high external validity. Further, multiple regression results indicated that congruence, consistency, and differentiation did not predict job or occupational satisfaction, suggesting cross-cultural boundaries on Holland's theory itself. Finally, we assessed and found limited conceptual and linguistic equivalence of measurement of the VPI in India. The assumptions of Holland's theory in light of the current findings are discussed.     

Measurement of Sense of Community Within the Sphere of City1

Based on a deductive process, several objective items were developed to measure people's sense of community with their city of residence. An item analysis produced a 17-item scale, which was then evaluated for homogeneity and external validity in three studies using telephone interviews on random samples in Alabama and South Carolina. Of seven hypotheses that were tested, six received support. Results described the scale as internally reliable and unidimensional, and the scale differentiated between people who differed in terms of demographics, home ownership, and civic contributions. Contrary to prediction, the scale did not relate to how long people had lived in their city. Explanations for this were offered and then conclusions were drawn about the potential usefulness of the scale.     

Handling the Stress of Looking for a Job in Law School: The Relationship between Intrinsic Motivation,Internal Attributions,Relations with Others,and Happiness

A study was conducted to examine how law students cope with the stress of looking for a job. We argue that effective coping is exhibited by the capacity to keep stress in one domain of life from spreading to and contaminating other, unrelated areas of life. Certain situations may be undeniably stressful; however, individuals who can maintain positive feelings about their life as a whole despite this stress may cope well. We hypothesized that three social psychological variables might be associated with this phenomenon: motivation (intrinsic versus extrinsic), attributions (internal versus external), and relations with others. The results indicated that students with intrinsic motives for being a lawyer were happier with their lives in general than were students with extrinsic motives. Likewise, students who attributed the ability to find a job to internal attributes were happier than were those who did not. Relations with others did not relate to happiness. Implications of these findings for career counseling are presented.     

Do Students with Different Characteristics Take Part in Psychology Experiments at Different Times of the Semester?

ABSTRACT In this study we examined 15 personality and demographic differences between psychology students who took part in experiments at different times of the semester and different times of the day. In general, few significant relations were found and those that did emerge were small in magnitude. Females and freshmen signed up for experiments earlier in the semester. Females took part in their first experiment earlier in the semester, as did students who were Protestant and who came from smaller home towns. Students low in self-consciousness also tended to begin participating earlier. Extraverts participated earlier in the day than introverts. Results generally were consistent with previous findings of differences between research volunteers and non-volunteers. However, it was concluded that because of their small magnitude and inconsistency from site to site, with a few precautions experimenters probably need not be concerned that timing effects threaten the external validity of psychological research outcomes.     

Evidence for the external validity of a social simulation measure of social skills

Two studies were conducted in order to obtain evidence regarding the external validity of a social simulation measure of social skills. The first study employed a contrasted group design, with the groups consisting of National Guardsmen and psychiatric patients. As predicted, trained judges rated the performances of the National Guardsmen in the simulations as more skillful and appearing less anxious than the psychiatric patients. In the second study, trained judges' ratings of psychiatric patients' performances in the simulations were found to be significantly related to ratings made by interviewers who had conducted a structured social history interview with the patients, nurses' ratings based on their observations of patients after 3 days on the psychiatric unit, a research assistant's ratings based on contact with the patient during the experiment, and finally, the subject's own self-ratings. In discussing the findings, it is hypothesized that one reason why support was found for the external validity of the simulation strategy in this study, in contrast to some previously reported studies, was the use of molar ratings in this study as opposed to the more molecular rating procedures used in those previous studies.These studies were funded in part by a research grant from the Veterans Administration.     

Does jury deliberation really improve jurors' memories?

The courts assume that jury deliberation corrects errors in jurors' memories, so that the verdict is based on accurate memory for the trial. We evaluated the validity of this assumption by examining jurors' memories and verdicts both before and after deliberation. Unlike previous studies, we tracked how event memories changed as a function of how they were discussed in deliberation. Overall, deliberation resulted in only a slight memory improvement. Deliberation corrected errors and did not introduce distortions. Reasons for such slight memory improvement are that jurors did not think they had memory gaps and thus did not use the deliberation process to improve their memory, and jurors who controlled deliberation were not always the most accurate in their memories. Finally, those most likely to change their verdict as a result of deliberation were not those who had the least accurate memories, but rather those who had the least confidence in their memories. Copyright © 2002 John Wiley & Sons, Ltd.     

An R package for single-case randomization tests

Randomization tests are nonparametric statistical tests that obtain their validity by computationally mimicking the random assignment procedure that was used in the design phase of a study. Because randomization tests do not rely on a random sampling assumption, they can provide a better alternative than parametric statistical tests for analyzing data from single-case designs. In this article, an R package is described for use in designing single-case phase (AB, ABA, and ABAB) and alternation (completely randomized, alternating treatments, and randomized block) experiments, as well as for conducting statistical analyses on data gathered by means of such designs. The R code is presented in a step-by-step way, which at the same time clarifies the rationale behind single-case randomization tests.     

Psychoeducational Intervention Program about Posttraumatic Growth for Japanese High School Students

This study evaluates the effect of a psychoeducational intervention about posttraumatic growth (PTG) among Japanese adolescents. Study 1 examined whether those who learned about changes related to stress would report higher growth than those who did not. Study 2 examined whether those who learned about PTG perceived more growth than those who learned about negative changes or those who did not learn about potential changes. Findings from both studies indicated that PTG was higher in those who received the intervention program than those who did not. These studies demonstrate the potential effectiveness of a psychoeducational intervention program in fostering adolescents’ perceptions and understanding of their personal growth following stressful experiences. Implications are that the knowledge about not only stress reactions or coping but the possibility of PTG can be integrated into the health education.     

Telephone counseling increases cessation rates among young adult smokers.

During June 2000-May 2001, the American Cancer Society conducted a randomized trial of telephone counseling among more than 3,500 current smokers who called to seek assistance in quitting. All eligible callers were randomized to receive either self-help booklets through the mail or booklets and up to 5 sessions of telephone counseling. Approximately 12% (420/3,522) of study participants were 18-25 years of age. Using intent to treat analyses, 3- and 6-month quit rates among both younger and older smokers were significantly higher among those who received telephone counseling than among those who received self-help booklets only. Three-month rates were 20% versus 9% for 18-25 year olds and 15% versus 10% for older adults. Results indicate that younger smokers can benefit from telephone counseling.     

Socially motivated projection: Need to belong increases perceived opinion consensus on important issues

In three studies, we tested whether the need to belong would motivate people to perceive consensus for their opinions on important social issues. In Study 1, a nationally representative telephone survey, participants with a high dispositional need to belong perceived greater consensus for their opinions on immigrant naturalization than did those with a low need to belong. However, this relationship was strongest among participants who reported that the issue was personally important to them. In Study 2, participants primed with rejection‐related (versus acceptance‐related) words, and who reported high levels of issue importance, demonstrated greater false consensus for their opinions on a proposed alcohol tax increase. In Study 3, participants who received random feedback that they held a common (versus uncommon) opinion had a lower subsequent need to belong when the issue was important to them, suggesting that consensus perceptions can in fact mitigate belongingness needs. Copyright © 2011 John Wiley & Sons, Ltd.     

Sudden Gains in Two Trauma-Focused Treatments for Posttraumatic Stress Disorder

In the current study, we examined the degree to which sudden gains (large, rapid, and stable symptom reduction in a one-session interval) predicted treatment outcome in adults randomized to two different trauma-focused treatments. Adults diagnosed with PTSD were randomized to either written exposure therapy (WET; n = 63), a brief, exposure-based treatment for posttraumatic stress disorder (PTSD), or the more time-intensive Cognitive Processing Therapy (CPT; n = 63). Findings showed that 20.6% of participants who received WET and 17.5% of participants who received CPT experienced sudden gains. Sudden gains occurred earlier in WET (M session = 2.69, SD = 0.75) than in CPT (M session = 5.64, SD = 3.01). However, there were no treatment condition differences in the magnitude of the sudden gains. Treatment outcomes were significantly better for those who experienced sudden gains compared with those who did not, regardless of treatment assignment. Exploratory analyses of participants’ trauma narratives revealed that expressing more negative emotion predicted the occurrence of sudden gains in both treatment conditions. Negative beliefs about the self and others did not predict sudden gains. The findings are discussed in terms of how they may help identify individual early response patterns that predict outcomes in trauma-focused treatments.     

Who enrolls in prevention trials? Discordance in perception of risk by professionals and participants

Internal and external validity problems permeate all intervention studies but are accentuated in primary preventive intervention research, particularly when studies target or recruit individuals based on their risk for psychopathology. Since many people who are at risk do not yet experience distress, they may not perceive the need for intervention. Recruitment tactics based on explaining extent of risk are unlikely to be persuasive and may have negative consequences. If respondents are not motivated to participate, a small or biased subset of the target population will participate in the intervention. Bias is of special concern when those enrolled represent only part of the continuum of risk. Selective enrollment may compromise both internal validity (the interpretation of the research results) and external validity (the generalizability of the findings) of intervention trials in primary prevention. This article discusses the effects of partial enrollment and the resultant bias. It suggests several strategies for increasing the enrollment of the target population and examines some of their ethical ramifications. It also stresses the importance of collecting systematic data documenting how the participants in the intervention differ from the target group as a whole.     

The Nature,Timing, and Symptom Trajectories of Dropout From Transdiagnostic and Single-Diagnosis Cognitive-Behavioral Therapy for Anxiety Disorders

Dropout from psychotherapy is common and can have negative effects for patients, providers, and researchers. A better understanding of when and why patients stop treatment early, as well as actionable factors contributing to dropout, has the potential to prevent it. Here, we examined dropout from a large randomized controlled trial of transdiagnostic versus single-diagnosis cognitive-behavioral treatment (CBT) for patients with anxiety disorders (n = 179; Barlow et al., 2017). We aimed to characterize the timing of and reasons for dropout and test whether participants who dropped out had different symptom trajectories than those who completed treatment. Results indicated that overall, the greatest risk of dropout was prior to the first treatment session. In single-diagnosis CBT, dropout risk was particularly elevated before the first session and after other early sessions, whereas in transdiagnostic CBT, dropout risk was low and stable before and during treatment. Participants most often dropped out due to failure to comply with study procedures or dissatisfaction with or desiring alternative treatment. Results from multilevel models showed that trajectories of anxiety symptoms did not significantly differ between dropouts and completers. These findings suggest that there may be specific time windows for targeted and timely interventions to prevent dropout from CBT.     

Parent-child relations in offenders who commit violent sexual crimes against children

One hundred eighty-one men were compared on parent–child relations, using the Clarke Parent–Child Relations Questionnaire. The sample included 66 heterosexual pedophiles, 29 homosexual pedophiles, 36 incest offenders and 50 controls. The offender groups were further divided into those who used force (20%) versus those who did not; into those sexually victimized as children themselves (53.2%) versus those not victimized; and into those physically abused as children (47.5%) versus those not abused. Results showed that the offender groups had more disturbances than controls in father–son relations, contrary to earlier findings showing disturbances in mother–son relations among individuals who interact sexually with minors. The offenders who used force in their offenses differed only in being more aggressive to their fathers as children, but otherwise did not differ from offenders who did not use force in their offenses. Offenders both sexually and physically abused as children showed more disturbances in father relationships than offenders who were not abused during their childhoods. Disturbed parent-child relations undoubtedly leave residual difficulties in sex offenders in later life, but in the present sample, they did not play the expected critical role in explaining the use of force by perpetrators who coerce children into sex.     

Outcome predictors in guided and unguided self-help for social anxiety disorder

Internet-based self-help with therapist guidance has shown promise as an effective treatment and may increase access to evidence-based psychological treatment for social anxiety disorder (SAD). Although unguided self-help has been suggested primarily as a population-based preventive intervention, some studies indicate that patients with SAD may profit from unguided self-help. Gaining knowledge about predictors of outcome in guided and unguided self-help for SAD is important to ensure that these interventions can be offered to those who are most likely to respond. Utilizing a sample of 245 patients who received either guided or unguided self-help for SAD, the present study examined pre-treatment symptoms and program factors as predictors of treatment adherence and outcome. The results were in line with previous findings from the face-to-face treatment literature: namely, the intensity of baseline SAD symptoms, but not depressive symptoms, predicted treatment outcomes in both unguided and guided self-help groups. Outcomes were unrelated to whether a participant has generalized versus specific SAD. Furthermore, for the unguided self-help group, higher credibility ratings of the treatment program were associated with increased treatment adherence. The findings suggest that guided and unguided self-help may increase access to SAD treatment in a population that is more heterogeneous than previously assumed.