Single‐Case Experimental Designs for the Evaluation of Treatments for Self‐Injurious and Suicidal Behaviors

Single‐case experimental designs (SCEDs) provide a time‐ and cost‐effective alternative to randomized clinical trials and offer significant advantages in terms of internal and external validity. A brief history and primer on SCEDs is provided, specifically for use in suicide intervention research. Various SCED methodologies, such as AB, ABAB, multiple baseline, and changing criterion designs, are described. Advantages and disadvantages of their use specifically for intervention research for suicidal and self‐injurious behaviors are detailed.     

Single-case experimental designs for behavioral neuroscience

Single-case experimental designs (SCEDs) are commonly used in behavior analytic research but rarely used in behavioral neuroscience research. The recent development of technologies that allow control of the timing of neurobiological events such as gene expression and neuronal firing enable the fruitful application of SCEDs for the study of brain–behavior relations. There are at least 3 benefits expected from applying SCEDs to study how neurobiological events affect behavior. First, SCEDs entail direct within- and across-subject assessments of reliability, likely increasing the probability of replication across studies and encouraging a search for the causes of replication failure when they occur. Second, SCEDs focus on behavior in individual organisms producing a body of knowledge that applies to individuals rather than population parameters. Finally, SCEDs require fewer animals, decreasing costs and effort and addressing the ethical obligation to reduce the number of animals used for research. Examples are provided using hypothetical data generated based on published research. Collaborations between behavior analysts and behavioral neuroscientists will bring the world within the skin under direct experimental control and broaden our understanding of the determinants of behavior.     

Experimental and Quasi-Experimental Designs in Visitor Studies: A Critical Reflection on Three Projects

Identifying causal relationships is an important aspect of research and evaluation in visitor studies, such as making claims about the learning outcomes of a program or exhibit. Experimental and quasi-experimental approaches are powerful tools for addressing these causal questions. However, these designs are arguably underused in visitor studies. In this article, we offer examples of the use of experimental and quasi-experimental designs in science museums to aide investigators interested in expanding their methods toolkit and increasing their ability to make strong causal claims about programmatic experiences or relationships among variables. Using three designs from recent research (fully randomized experiment, posttest only quasi-experimental design with comparison condition, and posttest with independent pretest design), we discuss challenges and tradeoffs related to feasibility, participant experience, alignment with research questions, and internal and external validity. We end the article with broader reflections on the role of experimental and quasi-experimental designs in visitor studies.     

Quantifying false positives in simulated events using partial interval recording and momentary time sampling with dual‐criteria methods

The dual‐criteria (DC) and conservative dual‐criteria (CDC) methods allow clinicians and researchers to quantify the occurrence of false‐positive outcomes within single‐case experimental designs. The purpose of the current study was to use these DC and CDC methods to measure the incidence of false positives with simulated data collected via discontinuous interval methods (i.e., momentary time sampling, partial‐interval recording) as a function of data series length. We generated event data to create 316,800 unique simulated data series for analysis. In Experiment 1, we evaluated how changes in relevant parameters (i.e., interval sizes, event durations, IRT‐to‐event‐run ratios) produced false positives with momentary time sampling procedures. We also assessed the degree that the CDC method produced fewer false positives than the DC method with simulated interval data. In Experiment 2, we used similar procedures to quantify the occurrence of false positives with partial‐interval recording data. We successfully replicated outcomes from previous research in the current study, though such results only highlight the generality of the procedures relating to false positive (and not false negative) outcomes. That is, these results indicate MTS and PIR may adequately control for false positives, but our conclusions are limited by a lack of data on power.     

The empirical status of empirically supported psychotherapies: assumptions, findings, and reporting in controlled clinical trials

This article provides a critical review of the assumptions and findings of studies used to establish psychotherapies as empirically supported. The attempt to identify empirically supported therapies (ESTs) imposes particular assumptions on the use of randomized controlled trial (RCT) methodology that appear to be valid for some disorders and treatments (notably exposure-based treatments of specific anxiety symptoms) but substantially violated for others. Meta-analytic studies support a more nuanced view of treatment efficacy than implied by a dichotomous judgment of supported versus unsupported. The authors recommend changes in reporting practices to maximize the clinical utility of RCTs, describe alternative methodologies that may be useful when the assumptions underlying EST methodology are violated, and suggest a shift from validating treatment packages to testing intervention strategies and theories of change that clinicians can integrate into empirically informed therapies.     

Multivariate multiple regression analyses: a permutation method for linear models

A multivariate extension of a univariate procedure for the analysis of experimental designs is presented. A Euclidean-distance permutation procedure is used to evaluate multivariate residuals obtained from a regression algorithm, also based on Euclidean distances. Applications include various completely randomized and randomized block experimental designs such as one-way, Latin square, factorial, nested, and split-plot designs, with and without covariates. Unlike parametric procedures, the only required assumption is the randomization of subjects to treatments.     

A Clinician’s Primer for Idiographic Research: Considerations and Recommendations

Theorists and clinicians have long noted the need for idiographic (i.e., individual-level) designs within clinical psychology. Results from idiographic work may provide a possible resolution of the therapist’s dilemma—the problem of treating an individual using information gathered via group-level research. Due to advances in data collection and time series methodology, there has been increasing interest in using idiographic designs to answer clinical questions. Although time series methods have been well-studied outside the field of clinical psychology, there is limited direction on how clinicians can use such models to inform their clinical practice. In this primer, we collate decades of published and word-of-mouth information on idiographic designs, measurement, and modeling. We aim to provide an initial guide on the theoretical and practical considerations that we urge interested clinicians to consider before conducting idiographic work of their own.     

Strategic and tactical limits of comparison studies

A comparison study is an experiment whose primary purpose is to compare directly (regardless of experimental design) at least two different procedures for changing behavior or two or more components of such a procedure. This paper argues that, in spite of their popularity, such studies typically lead to inappropriate inferences with poor generality based on improper evidence gathered in support of the wrong question, thus wasting the limited experimental resources. The discussion considers problems concerning the functions of comparison studies, the nature of the comparisons that are attempted, the generality of their findings, and the limited role that they can play in technological research.     

Number of events and reliability in fMRI

Relatively early in the history of fMRI, research focused on issues of power and reliability, with an important line concerning the establishment of optimal procedures for experimental design in order to maximize the various statistical properties of such designs. However, in recent years, tasks wherein events are defined only a posteriori, on the basis of behavior, have become increasingly common. Although these designs enable a much wider array of questions to be answered, they are not amenable to the tight control afforded by designs with events defined entirely a priori, and little work has assessed issues of power and reliability in such designs. We demonstrate how differences in numbers of events—as can occur with a posteriori event definition—affect reliability, both through simulation and in real data.     

The Mozart effect: questions about the seminal findings of Rauscher, Shaw, and colleagues

Seminal evidence for the Mozart effect was presented by Rauscher, Shaw, and colleagues in 1993 and 1994. A critical evaluation of their methodologies and interpretation of findings raises questions that must be answered before this evidence can be regarded as valid. We discuss issues such as their scoring of the Stanford-Binet Intelligence Scale, the experimental design used in their 1993 study, the validity of their 1993 IQ measure, the duration of the Mozart effect, their choice of experimental tasks in relation to predictions of the trion model of neural functioning, and the statistical analyses and interpretation of results in the 1993 and 1994 studies.     

Clinical Feedback About Empirically Supported Treatments for Obsessive‐Compulsive Disorder

Previous evidence for the treatment of obsessive‐compulsive disorder (OCD) has been derived principally from randomized controlled trials. As such, evidence about the treatment of OCD has unilaterally flowed from researchers to clinicians. Despite often having decades of experience treating OCD, clinicians’ feedback on their clinical observations in using these treatments has not been solicited. The current study contacted clinicians for their clinical observations on empirically supported treatments for OCD to identify commonly used cognitive-behavioral techniques and their limitations in their practices. One hundred eighty-one psychotherapists completed an online survey. The average participant practiced psychotherapy for 15 years, worked in private practice, held a doctorate, and treated an average of 25 clients with OCD in their lifetime. In regard to the most common techniques, behavioral strategies involving exposure to a feared outcome and prevention of a compulsive ritual were the most frequent group of interventions, followed by techniques that attempted to identify and challenge irrational thoughts. However, the majority of participants also reported incorporating mindfulness or acceptance-based methods. Based on therapists’ reports, the most common barriers to the efficacy of cognitive-behavioral interventions included limited premorbid functioning, chaotic lifestyles, controlling and critical families, OCD symptom severity, OCD symptom chronicity, and comorbidities. This study provides insight into common practices and limitations in clinical practice to inform future clinically relevant treatment research.     

Why Can't We Be More Idiographic in Our Research?

ABSTRACT— Most psychological scientists make inferences about the relations among variables of interest by comparing aggregated data from groups of individuals. Although this method is unarguably a useful one that will continue to yield scientific advances, important limitations exist regarding the efficiency and flexibility of such designs, as well as with the generality of obtained results. Idiographic research strategies, which focus on the intensive study of individual organisms over time, offer a proficient and flexible alternative to group comparison designs; however, they are rarely taught in graduate training programs and are seldom used by psychological scientists. We highlight some of the unique strengths of idiographic methods, such as single case experimental designs, and suggest that psychological science will progress most efficiently with an increased use of such methods in both laboratory and clinical settings.     

Challenges of Evaluating Multilevel Interventions

This article uses the Comprehensive Mixed-Methods Participatory Evaluation (CMMPE; Nastasi and Hitchcock Transforming school mental health services: Population-based approaches to promoting the competency and wellness of children, Thousand Oaks, CA: Corwin Press with National Association of School Psychologists 2008; Nastasi et al. School-based mental health services: creating comprehensive and culturally specific programs. Washington, DC: American Psychological Association 2004) model as a framework for addressing the multiplicity of evaluation decisions and complex nature of questions related to program success in multilevel interventions. CMMPE defines program success in terms of acceptability, integrity, social or cultural validity, outcomes (impact), sustainability and institutionalization, thus broadening the traditional notions of program outcomes. The authors use CMMPE and an example of a community-based multilevel sexual risk prevention program with multiple outcomes to discuss challenges of evaluating multilevel interventions. The sexual risk program exemplifies what Schensul and Tricket (this issue) characterize as multilevel intervention–multilevel evaluation (M–M), with both intervention and evaluation at community, health practitioner, and patient levels. The illustration provides the context for considering several challenges related to M–M designs: feasibility of randomized controlled trials within community-based multilevel intervention; acceptability and social or cultural validity of evaluation procedures; implementer, recipient, and contextual variations in program success; interactions among levels of the intervention; unanticipated changes or conditions; multiple indicators of program success; engaging multiple stakeholders in a participatory process; and evaluating sustainability and institutionalization. The complexity of multilevel intervention and evaluation designs challenges traditional notions of evaluation research and experimental designs. Overcoming these challenges is critical to effective translation of research to practice in psychology and related disciplines.     

Cross-sectional,longitudinal and composite longitudinal data on the Career Maturity Inventory Attitude Scale

Longitudinal data collection designs are frequently used in the study of developmental variables but typically yield patterns of results that are different from those obtained using cross-sectional designs. Recent research has focused on ways to clarify and reconcile the methodological distinctions between these two designs by utilizing alternative designs that incorporate features of both. This investigation compared the results obtained by the two traditional designs on the Career Maturity Inventory-Attitude Scale with the effects of an alternative design: constructing a composite longitudinal gradient. The two traditional methodologies yielded different results from each other for males, but not for females. However, the alternative design failed to eliminate the obtained differences for males, and introduced differences for females, casting doubt on the efficacy of the “solution.” The implications of these results for developmental research and career maturity were discussed.     

Imaging the developing brain with fMRI

Advancements in magnetic imaging techniques have revolutionized our ability to study the developing human brain in vivo. The ability to noninvasively image both anatomy and function in healthy volunteers, including young children, has already enhanced our understanding of brain and behavior relations. The application of these techniques to developmental research offers the opportunity to further explore these relationships and allows us to ask questions about where, when and how cognitive abilities develop in relation to changes in underlying brain systems. It is also possible to explore the contributions of maturation versus learning in the development of these abilities through cross-sectional and longitudinal research involving training and intervention procedures. Current imaging methodologies, in conjunction with new and rapidly evolving techniques, hold the promise of even greater insights into developmental issues in the near future. These methodologies and their application to development and learning are discussed in the current paper.     

It has been said that, “There are three degrees of falsehoods: Lies,damn lies,and statistics”

Applied behavior analysis (ABA) researchers have historically eschewed population-based, inferential statistics, preferring to conduct and analyze repeated observations of each participant's responding under carefully controlled and manipulated experimental conditions using single-case designs (SCDs). In addition, early attempts to adapt traditional statistical procedures for use with SCDs often involved trade-offs between experimental and statistical control that most ABA researchers have found undesirable. The statistical methods recommended for use with SCDs in the current special issue represent a welcome departure from such prior suggestions in that the current authors are proposing that SCD researchers add statistical methods to their current practices in ways that do not alter traditional SCD practices. Further refinement and use of such methods would (a) facilitate the inclusion of research using SCDs in meta-analyses and (b) aid in the development and planning of grant-funded research using SCD methods. Collaboration between SCD researchers and statisticians, particularly on research that demonstrates the benefit of these methods, may help promote their acceptance and use in ABA.     

Efficacy of guided self-help behavioural activation and physical activity for depression: a randomized controlled trial

Behavioural activation and physical activity have received empirical support that highlight their efficacy in reducing depression. Even though both behavioural activation and physical activity share the common goal of reactivating the individual, limited research has directly compared these interventions, and more research is required to evaluate their efficacy when offered in low-intensity formats. The present study involves a randomized controlled clinical trial comparing the efficacy of two guided self-help interventions for the treatment of depression: behavioural activation and physical activity. Fifty-nine participants presenting mild-to-moderate symptoms of depression were randomized either to a behavioural activation intervention (n = 20), a physical activity intervention (n = 19) or a wait-list control group (n = 20). All participants completed symptom measure pre-, mid- and post-intervention, as well as at a two-month follow-up. Mixed-model analyses of variance revealed that both interventions were significantly more efficacious in reducing depressive symptoms in comparison with the control group. Physical activity involved significantly less time-investment compared to the behavioural activation condition (less than half the amount of time). These results indicate that physical activity and behavioural activation both effectively reduce depressive symptoms and are favourably applicable in low-intensity formats. Implications of these results and avenues for future research are discussed.     

Beyond Randomized Controlled Trials in Attempted Suicide Research

There is a lack of evidence about what is the best treatment for people who present to hospital after self harm. Most treatment trials have been small and involved unrepresentative groups of patients which result in inconclusive findings. Here we note some of the characteristics of attempted suicide which make it a difficult subject to study. We describe the problems of doing randomized controlled trials in attempted suicide and outline the advantages and difficulties of randomized controlled trials, Zelen designs, patient preference designs, and cluster randomized trials in attempted suicide intervention trials. Researchers and consumers should consider other research designs when asking what is effective after self harm.     

Differential efficacy of a cognitive-behavioral intervention versus pharmacological treatment in the management of fibromyalgic syndrome

Given that studies about the differential efficacy of existing treatments in fibromyalgia syndrome are scarce, the aim of this study was to compare the differential efficacy of a cognitive-behavioral and a pharmacological therapy on fibromyalgia. Using a randomized controlled clinical trial, 28 fibromyalgic patients were assigned to one of following experimental conditions: (a) pharmacological treatment (i.e., cyclobenzaprine), (b) cognitive-behavioral intervention (i.e., stress inoculation training), (c) combined pharmacological and cognitive-behavioral treatment and (d) no treatment. The results show the superiority of cognitive-behavioral intervention to reduce the severity of fibromyalgia both at the end of the treatment and at follow-up. We conclude that cognitive-behavioral interventions must be considered a primary treatment of fibromyalgia syndrome.