Clinically significant but impractical? A response to Hageman and Arrindell

Hageman, and Arrindell [Hageman, W.J., & Arrindell, W.A. (in press). Establishing clinically significant change: increment of precision and the distinction between individual and group level of analysis. Behavior Research and Therapy.] suggest adaptations to the traditional clinical significance model originally developed by Jacobson, Follette, and Revenstorf [Jacobson, N.S., Follette, W.C., & Revenstorf, D. (1984). Toward a standard definition of clinically significant change. Behavior Therapy, 17, 308-311.]. They observe that one must distinguish between analysis at the individual and group level and based upon an alternative decision-making strategy have formulated different procedures for assessing clinically significant change that incorporate the unreliability inherent in testing measures. A comparison of the traditional method with Hageman and Arrindell's suggested approach is conducted utilizing data originally presented by Jacobson and Truax [Jacobson, N.S., & Truax, P., Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.] and implications of this comparison for the method developed by Hageman and Arrindell's method are discussed. Although this revised method has much to recommend it, it seems to yield results at the individual level that are quite similar to those derived from the traditional method. Given the complexity of the revised method, the traditional model developed by Jacobson, Follette, and Revenstorf (1984) still seems to be preferable.     

Discussion

Some essential adaptations to the method for determining clinically significant change originally introduced by Jacobson, Follette and Revenstorf [Jacobson, N. S., Follette, W. C. & Revenstorf, D. (1984a). Psychotherapy outcome research: methods for reporting variability and evaluating clinical significance. Behavior Therapy, 15, 336-352.] are presented. One adaptation deals with the failure in the original method to distinguish between analysis at the individual versus analysis at the group level. A second adaptation entails the provision of a closer approximation of the underlying true scores. This refinement represents an enhancement in precision. Specific aspects of this refinement may be understood in terms of a correction for error-based regression to the mean. Taking into account these adaptations, new procedures are described for determining (clinically significant) change. Some guidelines for the publication of outcome findings are also presented.     

Establishing clinically significant change: increment of precision and the distinction between individual and group level of analysis

Some essential adaptations to the method for determining clinically significant change originally introduced by Jacobson, Follette and Revenstorf [Jacobson, N. S., Follette, W. C. & Revenstorf, D. (1984a). Psychotherapy outcome research: methods for reporting variability and evaluating clinical significance. Behavior Therapy, 15, 336-352.] are presented. One adaptation deals with the failure in the original method to distinguish between analysis at the individual versus analysis at the group level. A second adaptation entails the provision of a closer approximation of the underlying true scores. This refinement represents an enhancement in precision. Specific aspects of this refinement may be understood in terms of a correction for error-based regression to the mean. Taking into account these adaptations, new procedures are described for determining (clinically significant) change. Some guidelines for the publication of outcome findings are also presented.     

Clinical significance methods: a comparison of statistical techniques

Clinically significant change refers to meaningful change in individual patient functioning during psychotherapy. Following the operational definition of clinically significant change offered by Jacobson, Follette, and Revenstorf (1984), several alternatives have been proposed because they were thought to be either more accurate or more sensitive to detecting meaningful change. In this study, we compared five methods using a sample of 386 outpatients who underwent treatment in routine clinical practice. Differences were found between methods, suggesting that the statistical method used to calculate clinical significance has an effect on estimates of meaningful change. The Jacobson method (Jacobson & Truax, 1991) provided a moderate estimate of treatment effects and was recommended for use in outcome studies and research on clinically significant change, but future research is needed to validate this statistical method.     

Clinically significant and practical! Enhancing precision does make a difference. Reply to McGlinchey and Jacobson, Hsu, and Speer

Based on a secondary analysis of the Jacobson and Truax [Jacobson, N.S. & Truax, P. (1991). Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.] data using both their own traditional approach and the refined method advanced by Hageman and Arrindell [Hageman, W.J.J.M., & Arrindell, W.A. (1999). Establishing clinically significant change: increment of precision and the distinction between individual and group level of analysis. Behaviour Research and Therapy, 37, 1169-1193], McGlinchey and Jacobson [McGlinchey, J. B., & Jacobson, N. S. (1999). Clinically significant but impractical? A response to Hageman and Arrindell. Behaviour Research and Therapy, 37, 1211-1217.] reported practically identical findings on reliable and clinically significant change across the two approaches. This led McGlinchey and Jacobson to conclude that there is little practical gain in utilizing the refined method over the traditional approach. Close inspection of the data used by McGlinchey and Jacobson however revealed a serious mistake with respect to the value of the standard error of measurement that was employed in their calculations. When the proper index value was utilised, further re-analysis by the present authors disclosed clear differences (i.e. different classifications of S's) across the two approaches. Importantly, these differences followed exactly the same pattern as depicted in Table 2 in Hageman and Arrindell (1999). The theoretical advantages of the refined method, i.e. enhanced precision, appropriate distinction between analysis at the individual and group levels, and maximal comparability of findings across studies, exceed those of the traditional method. Application of the refined method may be carried out within approximately half an hour, which not only supports its practical manageability, but also challenges the suggestion of McGlinchey and Jacobson (1999) that the relevant method would be too complex (impractical) for the average scientist. The reader is offered the opportunity of obtaining an SPSS setup in the form of an ASCII text file by means of which the relevant calculations can be carried out. The ways in which the valuable commentaries by Hsu [Hsu, L. M. (1999). A comparison of three methods of identifying reliable and clinically significant client changes: commentary on Hageman and Arrindell. Behaviour Research and Therapy, 37, 1195-1202.] and Speer [Speer, D. C. (1999). What is the role of two-wave designs in clinical research? Comment on Hageman and Arrindell. Behaviour Research and Therapy, 37, 1203-1210.) contribute to a better understanding of the technical/statistical backgrounds of the traditional and refined methods were also discussed.     

The effect of uncertainty of item parameter estimation on ability estimates

The conventional method of measuring ability, which is based on items with assumed true parameter values obtained from a pretest, is compared to a Bayesian method that deals with the uncertainties of such items. Computational expressions are presented for approximating the posterior mean and variance of ability under the three-parameter logistic (3PL) model. A 1987 American College Testing Program (ACT) math test is used to demonstrate that the standard practice of using maximum likelihood or empirical Bayes techniques may seriously underestimate the uncertainty in estimated ability when the pretest sample is only moderately large.This work was partially supported under contract No. N00014-85-K-0113, NR150-535, from the Cognitive Science Program, Office of Naval Research. The authors wish to thank Mark D. Reckase for providing the ACT data used in the illustration and two referees, Asociate Editor and Editor for helpful suggestions.     

The development of an instrument for measuring leadership behaviors in therapy groups

Research on leadership behavior in therapy groups emphasizes the need for precise theories and clinically meaningful instruments to study leadership variables. This paper reviews the development of the Group Leader Behavior Instrument (GLBI) and investigates its relationship to an earlier measure, the Group Leadership Questionnaire (GTQ-C). Both instruments were completed by 40 practicing group therapists. Analyses confirm a moderate degree of similarity between the leadership profiles obtained from the GLBI and GTQ-C. Differences due to the construction of the two scales are also examined. The authors conclude with a discussion of the potential uses of the GLBI to assess leadership behaviors in ongoing groups, to train group therapists, and to further progress in process and outcome research on group leadership.The authors thank Dr. Mary Fox, Director of the Group Therapy Program, Department of Psychiatry, University of Rochester, for her continued support in this work.     

Acceptance and Commitment Therapy for Posttraumatic Stress Disorder in Early Psychosis: A Case Series

Persons with psychosis often report high levels of posttraumatic stress disorder (PTSD) symptoms, which render them more vulnerable to relapse, symptom exacerbation, and reduced well-being. However, less is known about how to adequately accommodate the needs of persons recovering from a first episode of psychosis, presenting with PTSD. Further, the existing evidence-based interventions for PTSD seem less equipped to deal with serious mental disorder and comorbid conditions. This study aimed to assess the efficacy, acceptability, and safety of Acceptance and Commitment Therapy (ACT) for persons suffering from PTSD with comorbid trauma and psychosis. Three consecutively referred participants meeting ICD-10 criteria for PTSD and a first-episode nonaffective psychotic disorder were treated in an outpatient service within a case-series analysis. A manual-guided ACT intervention of 12 sessions showed clinically relevant improvement on self-report measures of PTSD symptoms and emotional distress. These initial findings are promising and appear to justify a more controlled evaluation of this brief intervention.     

Book Reviews

Object Relations Therapy: Using The Relationship by Sheldon Cashdan New York: Norton, 1988

Facets of Emotion: Recent Research edited by Klaus R. Scherer Hillsdale, NJ: Erlbaum, 1988

The Cognitive Structure of Emotions by Andrew Ortony, Gerald L. Clore, & Allan Collins Cambridge, England: Cambridge University Press, 1988     

A randomized clinical trial of group and individual Cognitive-Behavioral Therapy approaches for Social Anxiety Disorder

To compare the effectiveness of two Cognitive-Behavioral Therapy (CBT) interventions—an individual and a group intervention—in Social Anxiety Disorder therapy. We compared the two treatment groups against a waitlist condition in a randomized clinical trial with 86 young adults. The individual CBT intervention was Trial-Based Cognitive Therapy (TBCT) developed by De-Oliveira, a novel technique in which the therapist engages the patient in a simulated judicial trial with the goal of identifying and changing core dysfunctional beliefs. The group intervention consisted of exposition therapy based on the Hofmann and Otto protocol (Group CBT) to restructure negative and dysfunctional cognitions regarding social situations. Both interventions reduced psychiatric symptoms from pre- to post-test and primary social anxiety and depression symptoms relative to waitlist controls. The interventions were recently introduced in Brazil, and this is the first randomized control trial to compare TBCT and this Group CBT, which were effective in assessing changes in social anxiety symptoms as well as co-occurring psychiatric symptoms.     

Application of Problem Solving Therapy for Late-Life Anxiety

Cognitive behavioral therapy for anxiety has demonstrated lower efficacy in older compared with younger adults. Yet, few other evidence-based options for late-life anxiety have been examined. This case series aimed to demonstrate the application of Problem Solving Therapy (PST) to older adults with anxiety disorders building on PST’s strong empirical support for treating late-life depression. PST was implemented to treat three older primary-care patients diagnosed with anxiety disorders. We present treatment outcomes and discuss the feasibility and acceptability of using PST to treat these patients. Implications and lessons learned from these patients are discussed to inform further development of PST to better meet the needs of older patients suffering from late-life anxiety.     

On the Derivation of Instructional Applications From Cognitive Theories: Commentary on Chandler and Sweller

In "Cognitive Load Theory and the Format of Instruction," Chandler and Sweller (1991) report a series of experiments that focus on presentation formats that optimize learning from diagrams accompanied by ancillary text. This series of studies continues a line of work in which Sweller and his colleagues (Cooper & Sweller, 1987; Sweller, 1988; Sweller & Cooper, 1985; Tarmizi & Sweller, 1988; Ward & Sweller, 1990) have shown that students who study worked examples perform better than students who actually work the problems. Sweller (1988) proposed cognitive load theory as an explanation of these results, indicating that the critical feature of worked examples is that they appropriately direct attention and impose a relatively light cognitive load. Similarly, other presentation formats that accomplish these goals should also facilitate learning.     

Adaptability of Family Therapy Modalities in the Treatment of Lesbian and Gay Clients with Bulimia Nervosa

Bulimia nervosa (BN) has becoming an increasingly common form of eating disorder, particularly with gay and lesbian populations. The lack of current literature on this particular topic, as well as the limited resources for recommendations of treatment adaptations for this population is evident by the disproportionate number of sexual minorities that present for treatment. This paper discusses the possible reasons for the lack of literature and resources, and provides clinicians with the adaptations of Emotion Focused Therapy, Cognitive Behavioral Therapy, Structural Family Therapy, and Internal Family Systems when working with lesbian and gay populations who are suffering from BN. This paper aims to contribute more to the limited research in the area of sexual minorities in hopes to broaden the understanding of the treatment of individuals, as well as family functioning. We attempt to adapt these therapy modalities to be more inclusive of these nuanced presenting concerns, and to provide clinicians from all scopes of practice a thorough look into the treatment of a sexual minority population.     

Outcomes of specific interpersonal problems for binge eating disorder: comparing group psychodynamic interpersonal psychotherapy and group cognitive behavioral therapy

We assessed whether an attachment-based treatment, Group Psychodynamic Interpersonal Psychotherapy (GPIP) had a greater impact compared to Group Cognitive Behavioral Therapy (GCBT) on Cold/Distant and Intrusive/Needy interpersonal problems. Ninety-five individuals with Binge Eating Disorder (BED) were randomized to GPIP or GCBT and assessed at pre-, post-, and six months post-treatment. Both therapies resulted in a significant decrease in all eight interpersonal problem subscales except the Nonassertive subscale. GPIP resulted in a greater reduction in the Cold/Distant subscale compared to GCBT, but no differences were found for changes in the Intrusive/Needy subscale. GPIP may be most relevant for those with BED who have Cold/Distant interpersonal problems and attachment avoidance.     

Mindfulness and Acceptance-Based Group Therapy for Social Anxiety Disorder: An Open Trial

Mindfulness and Acceptance-Based Group Therapy (MAGT) for Social Anxiety Disorder (SAD) is based largely on Acceptance and Commitment Therapy (ACT; Hayes et al., 1999), with enhanced mindfulness mostly from Mindfulness-Based Cognitive Therapy (MBCT; Segal et al., 2002). The purpose of this study was to assess the feasibility and initial effectiveness of MAGT for the treatment of SAD. Forty-two SAD patients were invited to take part in an open trial of MAGT. Participants completed measures of social anxiety, mindfulness and acceptance, depression, and rumination at pretreatment, midtreatment (6 weeks), posttreatment (12 weeks), and at a 3-month follow-up session. Twenty-nine participants completed the treatment and these participants reported that the treatment was helpful. Effect sizes for treatment completers ranged from 1.00 to 1.17 for the social anxiety symptom measures at follow-up. Intent-to-treat analyses revealed significant reductions in social anxiety, depression, and rumination and significant increases in mindfulness and acceptance, with effect sizes ranging from .65 to .76 on the social anxiety measures. This study demonstrates that MAGT is feasible and acceptable to SAD patients and provides further support for the use of mindfulness and acceptance-based interventions for the treatment of SAD.     

Cognitive-Behavioral Therapy: The Truth Behind the Allegations

In response to several pejorative statements about Cognitive Behavioral Therapy in a recent issue of Psychotherapy (Silverman, 1999), a point by point counter argument to these criticisms are presented. Among other contentions, Silverman (1999) disparaged empirically validated methods as being simplistic and scientifically impoverished. The validity of these statements are challenged and empirical evidence is presented to support each counter argument. The efficacy of Cognitive Behavioral Therapy for a broad range of disturbances and presenting problems is presented. The future of Cognitive Behavioral Therapy and other empirically validated protocols are discussed, as well as recommendations for their use.     

Book reviews

Eclecticism comes of age John C. Norcross (ed.): Handbook of Eclectic Psychotherapy. New York: Brunner/Mazel, 1987. $65.00.

Eclecticism comes of age John C. Norcross (ed.): Casebook of Eclectic Psychotherapy. New York: Brunner/Mazel, 1987. $61.50.

Assessment in action Jennifer M. Kidd: Assessment in Action: a Manual for Those Involved in the Educational Guidance of Adults. Leicester: Unit for the Development of Adult Continuing Education, 1988. £4.50.

'A stormy adolescence' Debra Greenspoon Linesch: Adolescent Art Therapy. New York: Brunner/Mazel, 1988. $27.50.

Applying developmental theory to the practice of counselling Allen E. Ivey: Developmental Therapy; Theory into Practice. San Francisco: Jossey-Bass, 1986. £34.

Preaching to the unconverted Michael Reddy: The Manager's Guide to Counselling at Work. London: BPS and Methuen, 1987. £6.95 (£5.50 to members of the British Psychological Society).     

Abstracts edited by Neil Frude

F riedman , S. (1987) Technical considerations in the behaviour-marital treatment of agoraphobia. American Journal of Family Therapy,
P arker , T., H ill , J. W. and M iller , G. (1987) Multiple family therapy: evaluating a group experience for mentally retarded adolescents and their families. Family Therapy
B erger , M. and J urkovic , G. J. (1987) Family context in therapy. Journal of Family Psychology
L ebow , J. L. (1987) Training psychologists in family therapy in family institute settings. Journal of Family Pgcholosy
S loman , L., P erry , A. and F rankenburg , F. (1987) Family therapy with deaf member families. American Journal of Family Therapy
H oopes , M. H. (1987) Multigenerational systems: basic assumptions. American Journal of Family Therapy
M unton , A. G. and A ntaki , C. (1988) Causal beliefs amongst families in therapy: attributions at the group level. British Journal of Clinical Psychology     

Stop Thinking and Start Doing: Switching From Cognitive Therapy to Behavioral Activation in a Case of Chronic Treatment-Resistant Depression

Several recent investigations have demonstrated that Cognitive Therapy (CT) and Behavioral Activation (BA) are both efficacious treatments for depression (Butler, Chapman, Forman, & Beck, 2006; Dimidjian et al., 2006; Dobson, 1989; Gloaguen, Cottraux, Cucherat, & Blackburn, 1998; Hollon, Thase, & Markowitz, 2002; Jacobson et al., 1996). This investigation focuses on the treatment of Mr. X, a 62-year-old man with chronic treatment-resistant depression and comorbid personality pathology. After a course of 21 sessions of CT, treatment was switched to BA. We present daily mood data from 147 days of each treatment. Generalized least squares analyses of these time series data demonstrated that BA was associated with improved mood and these results were supported by clinical observation of improved functioning during the course of BA. We hypothesize that efforts at cognitive restructuring had an iatrogenic effect on this client and paradoxically triggered depressive rumination; these failed efforts provided further evidence in support of his underlying defectiveness schema. We also speculate that this client's dependent personality ran counter to CT's emphasis on autonomous homework assignments. Future randomized clinical trials are needed to investigate if nonresponders to CT benefit by switching to BA.