Observation procedure, observer gender, and behavior valence as determinants of sampling error in a behavior assessment analogue

Several factors thought to influence the representativeness of behavioral assessment data were examined in an analogue study using a multifactorial design. Systematic and unsystematic methods of observing group behavior were investigated using 18 male and 18 female observers. Additionally, valence properties of the observed behaviors were inspected. Observers' assessments of a videotape were compared to a criterion code that defined the population of behaviors. Results indicated that systematic observation procedures were more accurate than unsystematic procedures, though this factor interacted with gender of observer and valence of behavior. Additionally, males tended to sample more representatively than females. A third finding indicated that the negatively valenced behavior was overestimated, whereas the neutral and positively valenced behaviors were accurately assessed.     

Embarrassment of riches in the measurement of humility: A critical review of 22 measures

Abstract

Less than ten years ago, humility science seemed stuck with intractable measurement problems. Due to theoretical innovations, measures have proliferated in recent years. Humility science now faces a critical task of reconciling definitions and measures. We reviewed 22 measures of humility, including (a) survey measures of general humility, (b) survey measures of humility subdomains, (c) indirect measures of humility, and (d) state measures of humility. We coded each item of each measure into a humility content domain and compared the various content areas covered by each measure. Then, we described the scale structure and evidence pertaining to reliability and validity. Finally, we identified the relatively stronger measures of humility and recommended a consolidated definition of humility.     

Cognitive behavioral therapy in practice: therapist perceptions of techniques,outcome measures,practitioner qualifications,and relation to research

Cognitive behavioral therapy (CBT) has a strong evidence base for several psychiatric disorders, however, it may be argued that currently there is no overall agreement on what counts as ‘CBT’. One reason is that CBT is commonly perceived as encompassing a broad range of treatments, from purely cognitive to purely behavioral, making it difficult to arrive at a clear definition. The purpose of the present study was to explore practicing therapists’ perceptions of CBT. Three hundred fifty members of two multi-disciplinary interest groups for CBT in Sweden participated. Mean age was 46 years, 68% were females, 63% psychologists and mean number of years of professional experience was 12 years. Participants completed a web-based survey including items covering various aspects of CBT practice. Overall, therapist perceptions of the extent to which different treatment techniques and procedures were consistent with CBT were in line with current evidence-based CBT protocols and practice guidelines, as were therapists’ application of the techniques and procedures in their own practice. A majority of participants (78%) agreed that quality of life or level of functioning were the most important outcome measures for evaluating treatment success. Eighty percent of therapists believed that training in CBT at a basic level was a requirement for practicing CBT. There was a medium size Spearman correlation of r_s=.46 between the perceived importance of research to practice and the extent to which participants kept themselves updated on research. Implications for training, quality assurance, and the effectiveness of CBT in clinical practice are discussed.     

Assessing the Behavior of Girls: What We See and What We Miss

Three standardized measures of child and adolescent behavior are reviewed for their attention to issues salient in the experiences of girls in late childhood and adolescence (ages 12 to 18; American; American and Brown & Gilligan 1992). The Achenbach Child Behavior Checklist and Teacher Report Form (Achenbach and Achenbach), the Behavior Assessment System for Children (Reynolds & Kamphaus, 1992), and the Social Skills Rating System (Gresham & Elliott, 1990) are analyzed. Challenges to psychologists in schools for including the experiences of girls in the knowledge base grounding their practice and research are articulated.     

Behavioral measures of assertiveness: A comparison of audio and audiovisual coding of structured interactions

This study assessed the relative utility of audio and audiovisually recorded versions of structured interactions from the Behavioral Assertiveness Test-Revised. The subjects were 28 adult male psychiatric patients who participated in an assertion training program. The systems were evaluated on a number of dimensions including reliability, comparability, sensitivity to change, concurrent and discriminant validity, and cost. The two systems were found to be approximately equally reliable, to produce comparable results on measures that were assessed by both systems, and to be approximately equally sensitive to change. The main advantage of the audiovisual system was the inclusion of a measure of eye contact — which proved to be one of the most valid component behaviors assessed. Potential users need to weigh the additional information obtained from this measure against the substantially greater cost of the audiovisual system. It was concluded that both systems could be simplified, and limitations of the study were noted.     

Repurposing process measures to train psychotherapists: Training outcomes using a new approach

Aims: First, this paper presents the rationale for a novel approach to training counsellors in which measures for psychotherapy process research are taught to students before moving on to teaching basic empathic reflections and interventions. The rationale for this is that client process measures can be re‐purposed to help orient and sensitise trainees to key in‐session moments. Second, we present a training outcome study that assesses the effectiveness of this approach. Method: Using an experiential‐integrative therapy approach, a 13‐week training program was used to teach psychotherapy skills and process research measures to22 clinical graduate students taken from two cohorts. As part of the course, trainees conducted several single sessions with volunteer clients on four separate occasions. Training outcomes were measured using both trainee and client reports. Results: Compared to baseline, therapists reported significant and steady gains (all p's<.05) in session management, reducing their anxious self‐awareness, and in improved sense of self‐efficacy, with the latter having the largest effect (partial Eta Sq.=.381). Discussion: While the findings provide some support for a new training strategy, a dismantling design is needed next to more closely examine the process‐measure approach to training.     

Change process in systemic therapy: A qualitative investigation

This study addresses the process of therapeutic change in individuals who received systemic therapy. This study is an exploratory qualitative study based on the client's perceptions and therapeutic experiences. The sample included 10 clients who had completed their therapy with systemic therapists. The method used for the collection of the data was semi‐structured interviews (Change Interview; Elliott, Slatick, & Urman, 2001). The interviews were analysed using grounded theory analysis. The results showed that both specific and common factors in therapy function in a co‐occurring mode, hence suggesting a synergy effect between common factors and specific techniques or therapist factors in psychotherapy. The core category ‘experience of therapeutic change, within the secure frame of therapy, is a process of deconstructing and reconstructing the house you live in: Yourself’ emerged based on self‐exploration and self‐discovery and is then discussed in a theoretical constructivist framework. Implications for research on the issue of the process of therapeutic change are discussed.     

Comparability of peer-assessment measures: A multitrait-multimethod and selection analytic approach

This research attempted to assess the relationship among the Pupil Evaluation Inventory (PEI), sociometric nominations, and sociometric ratings in terms of (1) a multitrait-multimethod analysis, (2) selection agreement indices of extreme individuals, and (3) stability over an 8-week period. Subjects were members of four second-grade classrooms at a regional school serving a diverse population of children. While stability estimates for measures were adequate and evidence for convergent validity was reasonably good, evidence for divergent validity was weaker. Selection agreement analyses yielded highly variable results, indicating that different samples of children tend to be selected by different instruments despite the similarity in dimensions being tapped. Methodological and theoretical implications of these findings are discussed.This work was based on the first author's M.S. thesis under the direction of the second author.     

Change mechanisms in cognitive therapy of a simple phobia: Logical analysis and empirical hypothesis testing

The effectiveness of two hypothesized change mechanisms in cognitive therapy was investigated: logical analysis and empirical hypothesis testing. Thirty-eight spider phobics, as determined by performance on a behavioral avoidance test, were randomly assigned to either one of these two conditions or to a no-treatment control condition. Subjects participated in three group sessions. Outcome phobia questionnaire data suggested that both mechanisms produced desirable changes in a short period of time, with stronger evidence that logical analysis was superior to the control. Outcome from the behavior avoidance test and self-efficacy ratings failed to reach statistical significance but the trends were in the direction of positive change. Results are discussed in terms of the tripartite response dessynchrony hypothesis. Suggestions for future process research in cognitive therapy are provided.William O'Donohue, Ph.D., is an assistant professor of psychology at Northern Illinois University.Jeff Szymanski is a graduate student in clinical psychology at Northern Illinois University.The authors would like to thank Christine Casselles, Melissa McKelvie, Thomas M. Brown, Jill C. Rudman, Bonnie Schrieber, Amy Ray, Anne Valle, Lisa Herold, Jacqueline Ryan, Heather Barta, and Angela Leek for their assistance in this project. Moreover, the authors are grateful to Sol Feldman and Jane Fisher for their comments on an earlier version of this paper.     

Measuring depressive symptomatology of patients referred to a behavioral medicine clinic: Concordance of self-report measures

We measured depressive symptomatology of 134 outpatient veterans treated within a Behavioral Medicine Clinic utilizing three reliable instruments with depression subscales: Minnesota Multiphasic Personality Inventory (MMPI), Derogatis Stress Profile (DSP), and Symptom Checklist 90—Revised (SCL-90-R). Results indicate that although the instruments correlate significantly, they differ in sensitivity to depression and are clinically discordant. The MMPI classified a substantially higher number of patients as depressed than either the DSP or SCL-90-R. The present study argues that the discrepancy in clinical concordance and sensitivity of the three depression scales has considerable implications for psychologists concerned with the assessment and treatment of depression.     

Examining the item response process to personality measures in high-stakes situations: Issues of measurement validity and predictive validity

We conducted two experimental studies with between-subjects and within-subjects designs to investigate the item response process for personality measures administered in high- versus low-stakes situations. Apart from assessing measurement validity of the item response process, we examined predictive validity; that is, whether or not different response models entail differential selection outcomes. We found that ideal point response models fit slightly better than dominance response models across high- versus low-stakes situations in both studies. Additionally, fitting ideal point models to the data led to fewer items displaying differential item functioning compared to fitting dominance models. We also identified several items that functioned as intermediate items in both the faking and honest conditions when ideal point models were fitted, suggesting that ideal point model is “theoretically” more suitable across these contexts for personality inventories. However, the use of different response models (dominance vs. ideal point) did not have any substantial impact on the validity of personality measures in high-stakes situations, or the effectiveness of selection decisions such as mean performance or percent of fakers selected. These findings are significant in that although prior research supports the importance and use of ideal point models for measuring personality, we find that in the case of personality faking, though ideal point models seem to have slightly better measurement validity, the use of dominance models may be adequate with no loss to predictive validity.     

On the validity of implicit association measures of personality traits

Participants (N = 200) completed self-report and implicit association (IAT) measures of Extraversion, Conscientiousness, and Emotional Stability. Peer trait ratings and supervisor ratings of job performance were also collected. Results suggested low convergence between IAT and self-report trait measures. Cognitive ability and self-positive bias both explained significant portions of the method variance expressed by IAT measures. Correlations between IAT trait measures and job performance were statistically non-significant, of similar magnitudes to their self-report counterparts, and of magnitudes consistent with past research on criterion validities of self-report trait measures. The results offer only very limited support for the use of IAT trait measures in applied assessment settings.     

The Student Guided Functional Assessment Interview: An Analysis of Student and Teacher Agreement

Functional assessment strategies are used to guide the treatment of problem behavior with individuals who have severe disabilities. Also, researchers have extended functional assessment applications to individuals with emotional and behavioral disabilities. One such extension is the use of students as informants in the functional assessment process. The present study assessed an interview tool designed to elicit information directly from students regarding their problem behaviors. We examined agreement between students and their teachers. Results showed high agreement on the causes and functions of problem behavior with mixed agreement on support plan recommendations.     

Assessing motor proficiency in young adults: The Bruininks Oseretsky Test-2 Short Form and the McCarron Assessment of Neuromuscular Development

Currently, only two motor tests have norms extending into young adulthood – the McCarron Assessment of Neuromuscular Development (MAND, McCarron 1997) and the Bruininks Oseretsky Test of Motor Proficiency-2 (BOT-2, Bruininks & Bruininks, 2005). Research into the motor difficulties in early adulthood and health outcomes has been impeded because there is no agreed gold standard motor test for this group. The purposes of this study were to compare the discrimination accuracy, classification agreement, and predictive values, and gender distribution and prevalence of each test in identifying motor impairment (MI) in relation to DSM-V diagnostic criteria for DCD. Ninety-one young, healthy adults (M = 21.4 years, SD = 3.3) were recruited. Those classified as MI by each test scored at one standard deviation or more below the overall mean standard score. Small, statistically significant correlations were found between the MAND and BOT-2 SF tests for score rank (r = 0.370, p = 0.01) and standard score values (r = 0.404; p = 0.01). The overall decision agreement for non-MI cases was relatively high at 85% but very low for MI cases (4.4%). Unexpectedly, gender was balanced in MI cases. BOT-2 SF identified twice as many MI cases than MAND (13.2% vs 6.6%), yet overall comparative test specificity was high (89%). Predictive values for MAND, compared against BOT-2 SF as the standard, indicated broad independence between these tests and overall, the decision statistics indicated that the two tests identified different adult cohorts with MI. Objective classification of adult motor proficiency using a gold standard assessment tool including complex and ecologically valid tasks is still elusive.     

Mediators in psychological treatment of social anxiety disorder: Individual cognitive therapy compared to cognitive behavioral group therapy

According to cognitive-behavioral models of social anxiety disorder (SAD), four of the important maintaining mechanisms are avoidance, self-focused attention, anticipatory processing and post-event cognitive processing. Individual cognitive therapy (ICT) and cognitive behavioral group therapy (CBGT) both have substantial empirical support. However, it is unclear whether they achieve their effects by similar or different mechanisms. The aim of this study was to investigate whether changes in the four maintenance processes mediate clinical improvement in ICT and CBGT for SAD. We analyzed data from participants (N = 94) who received either ICT or CBGT in two separate RCTs. The results showed that ICT had larger effects than CBGT on social anxiety and each of the four potential mediators. More pertinently, moderated mediation analyses revealed significant between-treatment differences. Whereas improvement in ICT was mainly mediated by reductions in avoidance and self-focused attention, improvement in CBGT was mediated by changes in self-focused attention and in anticipatory and post-event processing. These results support the importance of the putative mediators, but suggest that their relative weights are moderated by treatment type.     

Cognitive behavioral therapy and acceptance and commitment therapy for insomnia: Exploring the potential benefit of psychological flexibility and self-compassion combined with behavioral strategies

Chronic insomnia is a prevalent sleep disorder with serious consequences on wellbeing and health that largely extend into daily functioning. Cognitive-Behavioral Therapy for Insomnia (CBT-I), an efficacious intervention for insomnia with solid empirical support, is the recommended first-line treatment. Given the complexity of factors and mechanisms involved in its aetiology and maintenance, advances in treatment protocols and modules are important. We will review the current knowledge on insomnia and examine how advancements in behavioral sleep medicine and third-wave therapies may apply to treatment. Specifically, we will outline how a treatment protocol based on Acceptance and Commitment Therapy (ACT), adapted to include insomnia-specific behavioral strategies and with an explicit focus on self-compassion, could be a potentially effective treatment. We believe that broadening treatment focus to target hyperarousal, metacognitions, dysfunctional though control strategies and provide self-compassion training may benefit treatment outcomes, increase sleep quality, reduce daytime symptoms, and improve quality of life.     

Assessing the Verbal Behavior of Conduct Problem Children During Mother-Child Interactions:

In this study we compared the verbal behavior of 4 to 8-year-old conduct problem and normal children in interactions with their mothers in the Child-Directed Interaction phase of the Dyadic Parent-Child Interaction Coding System (Eyberg & Robinson, 1983). It was predicted that conduct problem children would use less praise, ask fewer questions, make more critical statements, and give more commands during interactions with their mothers than normal children. Results support two of these four predictions, with conduct problem children using fewer questions and less praise. Implications for the assessment and treatment of conduct problems as well as how these verbal behaviors may develop from problem parent-child interactions are discussed.     

A review and critique of the process approach in neuropsychological assessment

This paper surveys the process approach literature with an emphasis on higher level cognitive functions like attention and concentration, learning and memory, and problem solving and executive functioning. In particular, it discusses organizational strategies and kinds of errors found on a variety of tests as well as behavioral, situational, and interpersonal issues as they relate to test procedures. It critiques the process approach and suggests future directions.     

Individualized services and supports through the wraparound process: Philosophy and procedures

The wraparound process is emerging as an alternative to the traditional treatment planning processes inherent in categorical services for children and adolescents with emotional and behavioral disorders. We describe the current practices which have been developed in the field. Key elements of the value and philosophical base for the wraparound process are discussed. Proposed procedural steps are described to aid communities in the implementation of the wraparound process. Due to social and policy pressures, it is likely that wraparound process implementations will continue to develop.