Bayesian estimates of autocorrelations in single-case designs

Researchers in the single-case design tradition have debated the size and importance of the observed autocorrelations in those designs. All of the past estimates of the autocorrelation in that literature have taken the observed autocorrelation estimates as the data to be used in the debate. However, estimates of the autocorrelation are subject to great sampling error when the design has a small number of time points, as is typically the situation in single-case designs. Thus, a given observed autocorrelation may greatly over- or underestimate the corresponding population parameter. This article presents Bayesian estimates of the autocorrelation that greatly reduce the role of sampling error, as compared to past estimators. Simpler empirical Bayes estimates are presented first, in order to illustrate the fundamental notions of autocorrelation sampling error and shrinkage, followed by fully Bayesian estimates, and the difference between the two is explained. Scripts to do the analyses are available as supplemental materials. The analyses are illustrated using two examples from the single-case design literature. Bayesian estimation warrants wider use, not only in debates about the size of autocorrelations, but also in statistical methods that require an independent estimate of the autocorrelation to analyze the data.     

Decision making, movement planning and statistical decision theory

We discuss behavioral studies directed at understanding how probability information is represented in motor and economic tasks. By formulating the behavioral tasks in the language of statistical decision theory, we can compare performance in equivalent tasks in different domains. Subjects in traditional economic decision-making tasks often misrepresent the probability of rare events and typically fail to maximize expected gain. By contrast, subjects in mathematically equivalent movement tasks often choose movement strategies that come close to maximizing expected gain. We discuss the implications of these different outcomes, noting the evident differences between the source of uncertainty and how information about uncertainty is acquired in motor and economic tasks.     

Using response ratios for meta-analyzing single-case designs with behavioral outcomes

Methods for meta-analyzing single-case designs (SCDs) are needed to inform evidence-based practice in clinical and school settings and to draw broader and more defensible generalizations in areas where SCDs comprise a large part of the research base. The most widely used outcomes in single-case research are measures of behavior collected using systematic direct observation, which typically take the form of rates or proportions. For studies that use such measures, one simple and intuitive way to quantify effect sizes is in terms of proportionate change from baseline, using an effect size known as the log response ratio. This paper describes methods for estimating log response ratios and combining the estimates using meta-analysis. The methods are based on a simple model for comparing two phases, where the level of the outcome is stable within each phase and the repeated outcome measurements are independent. Although auto-correlation will lead to biased estimates of the sampling variance of the effect size, meta-analysis of response ratios can be conducted with robust variance estimation procedures that remain valid even when sampling variance estimates are biased. The methods are demonstrated using data from a recent meta-analysis on group contingency interventions for student problem behavior.     

Choosing among techniques for quantifying single-case intervention effectiveness

If single-case experimental designs are to be used to establish guidelines for evidence-based interventions in clinical and educational settings, numerical values that reflect treatment effect sizes are required. The present study compares four recently developed procedures for quantifying the magnitude of intervention effect using data with known characteristics. Monte Carlo methods were used to generate AB designs data with potential confounding variables (serial dependence, linear and curvilinear trend, and heteroscedasticity between phases) and two types of treatment effect (level and slope change). The results suggest that data features are important for choosing the appropriate procedure and, thus, inspecting the graphed data visually is a necessary initial stage. In the presence of serial dependence or a change in data variability, the nonoverlap of all pairs (NAP) and the slope and level change (SLC) were the only techniques of the four examined that performed adequately. Introducing a data correction step in NAP renders it unaffected by linear trend, as is also the case for the percentage of nonoverlapping corrected data and SLC. The performance of these techniques indicates that professionals' judgments concerning treatment effectiveness can be readily complemented by both visual and statistical analyses. A flowchart to guide selection of techniques according to the data characteristics identified by visual inspection is provided.     

Analysis and meta-analysis of single-case designs: An introduction

The last 10 years have seen great progress in the analysis and meta-analysis of single-case designs (SCDs). This special issue includes five articles that provide an overview of current work on that topic, including standardized mean difference statistics, multilevel models, Bayesian statistics, and generalized additive models. Each article analyzes a common example across articles and presents syntax or macros for how to do them. These articles are followed by commentaries from single-case design researchers and journal editors. This introduction briefly describes each article and then discusses several issues that must be addressed before we can know what analyses will eventually be best to use in SCD research. These issues include modeling trend, modeling error covariances, computing standardized effect size estimates, assessing statistical power, incorporating more accurate models of outcome distributions, exploring whether Bayesian statistics can improve estimation given the small samples common in SCDs, and the need for annotated syntax and graphical user interfaces that make complex statistics accessible to SCD researchers. The article then discusses reasons why SCD researchers are likely to incorporate statistical analyses into their research more often in the future, including changing expectations and contingencies regarding SCD research from outside SCD communities, changes and diversity within SCD communities, corrections of erroneous beliefs about the relationship between SCD research and statistics, and demonstrations of how statistics can help SCD researchers better meet their goals.     

Characteristics of single-case designs used to assess intervention effects in 2008

This article reports the results of a study that located, digitized, and coded all 809 single-case designs appearing in 113 studies in the year 2008 in 21 journals in a variety of fields in psychology and education. Coded variables included the specific kind of design, number of cases per study, number of outcomes, data points and phases per case, and autocorrelations for each case. Although studies of the effects of interventions are a minority in these journals, within that category, single-case designs are used more frequently than randomized or nonrandomized experiments. The modal study uses a multiple-baseline design with 20 data points for each of three or four cases, where the aim of the intervention is to increase the frequency of a desired behavior; but these characteristics vary widely over studies. The average autocorrelation is near to but significantly different from zero; but autocorrelations are significantly heterogeneous. The results have implications for the contributions of single-case designs to evidence-based practice and suggest a number of future research directions.     

An effect size measure and Bayesian analysis of single-case designs

This article describes a linear modeling approach for the analysis of single-case designs (SCDs). Effect size measures in SCDs have been defined and studied for the situation where there is a level change without a time trend. However, when there are level and trend changes, effect size measures are either defined in terms of changes in R² or defined separately for changes in slopes and intercept coefficients. We propose an alternate effect size measure that takes into account changes in slopes and intercepts in the presence of serial dependence and provides an integrated procedure for the analysis of SCDs through estimation and inference based directly on the effect size measure. A Bayesian procedure is described to analyze the data and draw inferences in SCDs. A multilevel model that is appropriate when several subjects are available is integrated into the Bayesian procedure to provide a standardized effect size measure comparable to effect size measures in a between-subjects design. The applicability of the Bayesian approach for the analysis of SCDs is demonstrated through an example.     

一种新决策模型——量子决策模型

近10年以来, 一些学者基于量子理论研究思想与方法探索出了量子决策模型。由于该模型独特的理论结构, 它可被用于解释传统决策理论所难以解释的问题, 尤其是个体在不确定情况下的决策行为。该模型解释了诸如分离效应、分类决策的干涉效应以及合取谬误等传统决策模型很难解释的问题, 研究者已经证实其中一种称为量子问题等式的决策模型可以精确地预测决策中的顺序效应。作为一个有助于分析心理学中决策现象的新研究领域, 量子决策模型具有深入研究的理论意义和实践意义。     

Models for deferred decision making

Deferred decision theory assumes two simple hypotheses, observations that are purchased sequentially, losses for incorrect terminal decisions, and a constant loss per observation which may depend on the true hypothesis. After every observation, the decision maker may either make a terminal decision and incur a terminal loss if he is incorrect or purchase another observation; his objective is to minimize subjectively expected loss.     

Additional comparisons of randomization-test procedures for single-case multiple-baseline designs: Alternative effect types

A number of randomization statistical procedures have been developed to analyze the results from single-case multiple-baseline intervention investigations. In a previous simulation study, comparisons of the various procedures revealed distinct differences among them in their ability to detect immediate abrupt intervention effects of moderate size, with some procedures (typically those with randomized intervention start points) exhibiting power that was both respectable and superior to other procedures (typically those with single fixed intervention start points). In Investigation 1 of the present follow-up simulation study, we found that when the same randomization-test procedures were applied to either delayed abrupt or immediate gradual intervention effects: (1) the powers of all of the procedures were severely diminished; and (2) in contrast to the previous study's results, the single fixed intervention start-point procedures generally outperformed those with randomized intervention start points. In Investigation 2 we additionally demonstrated that if researchers are able to successfully anticipate the specific alternative effect types, it is possible for them to formulate adjusted versions of the original randomization-test procedures that can recapture substantial proportions of the lost powers.     

Visual aids and structured criteria for improving visual inspection and interpretation of single-case designs

Because behavior analysis is a data-driven process, a critical skill for behavior analysts is accurate visual inspection and interpretation of single-case data. Study 1 was a basic study in which we increased the accuracy of visual inspection methods for A-B designs through two refinements of the split-middle (SM) method called the dual-criteria (DC) and conservative dual-criteria (CDC) methods. The accuracy of these visual inspection methods was compared with one another and with two statistical methods (Allison & Gorman, 1993; Gottman, 1981) using a computer-simulated Monte Carlo study. Results indicated that the DC and CDC methods controlled Type I error rates much better than the SM method and had considerably higher power (to detect real treatment effects) than the two statistical methods. In Study 2, brief verbal and written instructions with modeling were used to train 5 staff members to use the DC method, and in Study 3, these training methods were incorporated into a slide presentation and were used to rapidly (i.e., 15 min) train a large group of individuals (N = 87). Interpretation accuracy increased from a baseline mean of 55% to a treatment mean of 94% in Study 2 and from a baseline mean of 71% to a treatment mean of 95% in Study 3. Thus, Study 1 answered basic questions about the accuracy of several methods of interpreting A-B designs; Study 2 showed how that information could be used to increase the accuracy of human visual inspectors; and Study 3 showed how the training procedures from Study 2 could be modified into a format that would facilitate rapid training of large groups of individuals to interpret single-case designs.     

Median decision techniques as an alternative to the classical statistical inference test

In the research area, the criterion to test a hypothesis is frequently the distance of each subject from the mean of the distribution. This concept of distance is constantly used and re-used as a source of evidence in the search for significant contrasts with respect to models of theoretical probability. This paper proposes a number of ideas regarding the use of new distance indicators related to statistics that rely less heavily on parametric assumptions, in particular, the use of median confidence intervals. We have simulated via EXCEL different data samples for a design of two independent groups under 15 different conditions in two different situations: mean equality and mean differences. The results showed the high sensitivity of the comparison of the confidence intervals, especially if a strict decision criterion is used. However, we did not obtain good results for sensitivity. In fact, in some cases, the comparison of the confidence intervals of the mean worked better than those of median.     

Computer-augmented decision making

The principal judgmental components of multiattribute decision making are examined here with specific reference to how these components can be captured electronically. Once captured, a function, rule, or algorithm may be executed for the integration of this information and the selection of the optimal alternative(s). Two kinds of algorithms are discussed: one based on linear models, the other on fuzzy-set theory and ratio scaling. With on-line support and certain assumptions about human biases (which lead to nonoptimal decisions), the quality of decisions can be enhanced considerably. The principal concerns are with end-user acceptance of computer augmented decisions.     

Knowledge management in decision making and decision support

A knowledge-oriented view of decisions and decision making is introduced, as a complement to classical perspectives and as a contribution of under-standing computer-based possibilities for relaxing strains on decision makers. This perspective includes a model of knowledge management activities performed by a decision maker and a taxonomy of knowledge types. It leads to a characterization of decision-support-system purpose, traits, and potentials that offers a basis for new research into computer-based possibilities for knowledge management. Clyde W. Holsapple holds the Rosenthal Endowed Chair in Management Information Systems and is Professor of Decision Science and Information Systems at the College of Business and Economics, University of Kentucky.     

A brief account of statistical tests for single-case research with persons with developmental disabilities

Education and rehabilitation research with persons with developmental disabilities is often based on single-case designs (with small numbers of ordinal data points collected at irregular intervals) and relies upon graphic display and visual inspection of the data. This paper (a) provides a brief account of some statistical tests, which may serve to supplement the visual inspection process and (b) underlines some of their strengths and limitations to help education and rehabilitation personnel make a reasonable choice among them.     

From a single-level analysis to a multilevel analysis of single-case experimental designs

Multilevel modeling provides one approach to synthesizing single-case experimental design data. In this study, we present the multilevel model (the two-level and the three-level models) for summarizing single-case results over cases, over studies, or both. In addition to the basic multilevel models, we elaborate on several plausible alternative models. We apply the proposed models to real datasets and investigate to what extent the estimated treatment effect is dependent on the modeling specifications and the underlying assumptions. By considering a range of plausible models and assumptions, researchers can determine the degree to which the effect estimates and conclusions are sensitive to the specific assumptions made. If the same conclusions are reached across a range of plausible assumptions, confidence in the conclusions can be enhanced. We advise researchers not to focus on one model but conduct multiple plausible multilevel analyses and investigate whether the results depend on the modeling options.     

Decision making styles and progress in occupational decision making

This study provides a reexamination of the role of different decisional strategies in facilitating progress in occupational decision making. Although the assumptions that a rational decision making style is the preferred mode of vocational functioning has been endorsed in a variety of career theories and interventions, there has been conflicting evidence about the validity of this assumption. To examine the role of different decisional approaches in the progress of making an occupational decision, the rational, intuitive, and dependent decision making style scores of 71 undergraduate students were used to predict progress in occupational decision making. The results of the regression analyses failed to provide support for the assumption that a rational style is the most effective in accomplishing this careerrelated task, but indicated strong support for the conclusion that the use of dependent decisional strategies is damaging, particularly in early stages of the decisional process.     

Numeracy and decision making

A series of four studies explored how the ability to comprehend and transform probability numbers relates to performance on judgment and decision tasks. On the surface, the tasks in the four studies appear to be widely different; at a conceptual level, however, they all involve processing numbers and the potential to show an influence of affect. Findings were consistent with highly numerate individuals being more likely to retrieve and use appropriate numerical principles, thus making themselves less susceptible to framing effects, compared with less numerate individuals. In addition, the highly numerate tended to draw different (generally stronger or more precise) affective meaning from numbers and numerical comparisons, and their affective responses were more precise. Although generally helpful, this tendency may sometimes lead to worse decisions. The less numerate were influenced more by competing, irrelevant affective considerations. Analyses showed that the effect of numeracy was not due to general intelligence. Numerical ability appears to matter to judgments and decisions in important ways.     

Group decision process and incrementalism in organizational decision making

In two studies examining resource allocation, support is found for the notion that group decisions are affected in systematic ways depending on whether or not there was individual consideration of the problem before meeting as a group. Specifically, compared to no prior consideration groups, prior consideration groups (1) escalate their commitment more in progress (i.e., ongoing) decisions, and (2) are less willing to concentrate resources on a single project in adoption (i.e., resource utilization) decisions. The findings challenge the blanket assertion that promoting divergent views in a group decision context is always related to better decisions.