THE CHANGING CRITERION DESIGN

This article describes and illustrates with two case studies a relatively novel form of the multiple-baseline design called the changing criterion design. It also presents the design's formal requirements, and suggests target behaviors and circumstances for which the design might be useful.     

Increasing Validity with Forced‐Choice Criterion Measurement Formats

The relative validities of forced‐choice (ipsative) and Likert rating‐scale item formats as criterion measures are examined. While there has been much debate about the relative technical and psychometric merits and demerits of ipsative instruments, the present research focused on the crucial question of whether the use of this format has any practical benefit – in terms of improved validity. An analysis is reported from a meta‐analysis data set. This demonstrates that higher operational validity coefficients (prediction of line‐manager ratings of competencies) are associated with the use of forced‐choice (r=.38) rather than rating scale (r=.25) item formats for the criterion measurement instrument when performance is rated by the same line managers on both formats and where the predictor is held constant. Thus the apparent criterion‐related validity of a predictor can increase by 50% simply by changing the format of the criterion measurement instrument. The implications of this for practice are discussed.     

How to Change the Past in One‐Dimensional Time

The possibility of changing the past by means of time‐travel appears to depend on the possibility of distinguishing the past as it is ‘before’ and ‘after’ the time‐travel. So far, all the metaphysical models that have been proposed to account for the possibility of past‐changing time‐travels operate this distinction by conceiving of time as multi‐dimensional, and thus by significantly inflating our metaphysics of time. The aim of this article is to argue that there is an intuitive sense in which past‐changing time‐travels are metaphysically possible also in one‐dimensional time.     

Increasing compliance with haircuts in a child with autism

Using a changing‐criterion design, we evaluated the effects of providing a preferred edible item and escape from sitting contingent on a child's compliance with sitting during haircuts. Results indicated that the intervention eliminated the child's escape responses and increased his sitting to a duration that permitted regular haircuts by his mother. Follow‐up sessions showed that the participant's increased compliance during haircuts continued for over 2 months. Copyright © 2010 John Wiley & Sons, Ltd.     

Model Comparisons and Model Selections Based on Generalization Criterion Methodology

The purpose of this article is to formalize the generalization criterion method for model comparison. The method has the potential to provide powerful comparisons of complex and nonnested models that may also differ in terms of numbers of parameters. The generalization criterion differs from the better known cross-validation criterion in the following critical procedure. Although both employ a calibration stage to estimate parameters, cross-validation employs a replication sample from the same design for the validation stage, whereas generalization employs a new design for the critical stage. Two examples of the generalization criterion method are presented that demonstrate its usefulness for selecting a model based on sound scientific principles out of a set that also contains models lacking sound scientific principles that are either overly complex or oversimplified. The main advantage of the generalization criterion is its reliance on extrapolations to new conditions. After all, accurate a priori predictions to new conditions are the hallmark of a good scientific theory. Copyright 2000 Academic Press.     

Effects of a Fluency-Building Program on the Reading Performance of Low-Achieving Second and Third Grade Students

This study evaluated the effects of a fluency-based reading program with 15 second and third grade students and 15 matched controls. Gains in oral reading fluency on untrained CBM probes were evaluated using a matched-pairs group-comparison design, whereas immediate and two-day retention gains in oral reading fluency on trained passages were evaluated using an adapted changing criterion design. Increases in WRCM due to training and number of trainings to criterion were also evaluated as a function of pre-training fluency levels. Results showed statistically significant gains on dependent measures for the treatment group, mean increases of two to three grade levels in passages mastered, and an optimal pre-training fluency range of 41-60 WRCM. Implications for fluency-based reading programs are discussed.     

Transparency of Assessment Centers: Lower Criterion‐related Validity but Greater Opportunity to Perform?

Assessment centers (ACs) are popular selection devices in which assessees are assessed on several dimensions during different exercises. Surveys indicate that ACs vary with regard to the transparency of the targeted dimensions and that the number of transparent ACs has increased during recent years. Furthermore, research on this design feature has put conceptual arguments forward regarding the effects of transparency on criterion‐related validity, impression management, and fairness perceptions. This study is the first to examine these effects using supervisor‐rated job performance data as the criterion. We conducted simulated ACs with transparency as a between‐subjects factor. The sample consisted of part‐time employed participants who would soon be applying for a new job. In line with our hypothesis, results showed that ratings from an AC with nontransparent dimensions were more criterion valid than ratings from an AC with transparent dimensions. Concerning impression management, our results supported the hypothesis that transparency moderates the relationship between self‐promotion and job performance, such that self‐promotion in the nontransparent AC was more positively related to job performance than self‐promotion in the transparent AC. The data lent no support for the hypothesis that participants’ perceptions of their opportunity to perform are higher in the transparent AC.     

A further refinement of procedures addressing food selectivity

Individuals with autism and/or developmental disabilities are more likely than their typically developing peers to be food selective. Evidence‐based treatment packages for food refusal and selectivity include the use of reinforcement, escape extinction, and manipulation of antecedent events . In the current study, the variety of foods consumed by two adolescent males with food selectivity were systematically increased to include a variety of foods across all of the food groups. A changing criterion design was used to systematically increase the expectation of the amount and variety of foods that were consumed.     

Exposure-based treatment to control excessive blood glucose monitoring.

We investigated an exposure‐based procedure for reducing excessive checking of blood glucose by a child with diabetes. In a changing criterion design, an exposure‐based procedure was implemented by systematically exposing the child to decreasing amounts of information about blood sugar levels (checking) and thereby increasing exposure to potential hypoglycemia. Access to information was reduced in graduated increments, with the parents setting criteria to levels at which they were willing to adhere. Results demonstrated that the procedure was effective in reducing excessive blood glucose checking and in improving metabolic control.     

The Rhythm Is Gonna Get You: The Influence of Circadian Rhythm Synchrony on Self‐Control Outcomes

This article aims to bridge the circadian and self‐control literatures by suggesting that people's self‐control performance varies as a function of their circadian preferences (i.e., chronotype). We review evidence for this assertion across a wide range of self‐control domains, including cognitive processing, emotion regulation, interpersonal relationships, and social influence. Across these widely ranging behaviors, the results indicate that when people's circadian preference matches the time of day, a synchrony effect occurs, and self‐control performance is at its peak. Therefore, people are more likely to display self‐regulatory failures and give in to temptations when there is a mismatch between their circadian preferences and time of day. Overall, this article offers new insights into the relationship between circadian preferences and self‐control and suggests novel and exciting new directions for future research.     

The Interdisciplinarity of Collaborations in Cognitive Science

We introduce a new metric for interdisciplinarity, based on co‐author publication history. A published article that has co‐authors with quite different publication histories can be deemed relatively “interdisciplinary,” in that the article reflects a convergence of previous research in distinct sets of publication outlets. In recent work, we have shown that this interdisciplinarity metric can predict citations. Here, we show that the journal Cognitive Science tends to contain collaborations that are relatively high on this interdisciplinarity metric, at about the 80th percentile of all journals across both social and natural sciences. Following on Goldstone and Leydesdorff (2006), we describe how scientometric tools provide a valuable means of assessing the role of cognitive science in broader scientific work, and also as a tool to investigate teamwork and distributed cognition. We describe how data‐driven metrics of this kind may facilitate this exploration without relying upon rapidly changing discipline and topic keywords associated with publications.     

A COMPARISON OF PEER VIDEO MODELING AND SELF VIDEO MODELING TO TEACH TEXTUAL RESPONSES IN CHILDREN WITH AUTISM

Peer video modeling was compared to self video modeling to teach 3 children with autism to respond appropriately to (i.e., identify or label) novel letters. A combination multiple baseline and multielement design was used to compare the two procedures. Results showed that all 3 participants met the mastery criterion in the self‐modeling condition, whereas only 1 of the participants met the mastery criterion in the peer‐modeling condition. In addition, the participant who met the mastery criterion in both conditions reached the criterion more quickly in the self‐modeling condition. Results are discussed in terms of their implications for teaching new skills to children with autism.     

Detection of long‐range dependence and estimation of fractal exponents through ARFIMA modelling

We evaluate the performance of autoregressive, fractionally integrated, moving average (ARFIMA) modelling for detecting long‐range dependence and estimating fractal exponents. More specifically, we test the procedure proposed by Wagenmakers, Farrell, and Ratcliff , and compare the results obtained with the Akaike information criterion (AIC) and the Bayes information criterion (BIC). The present studies show that ARFIMA modelling is able to adequately detect long‐range dependence in simulated fractal series. Conversely, this method tends to produce a non‐negligible rate of false detections in pure autoregressive and moving average (ARMA) series. Generally, ARFIMA modelling has a bias favouring the detection of long‐range dependence. AIC and BIC gave dissimilar results, due to the different weights attributed by the two criteria to accuracy and parsimony. Finally, ARFIMA modelling provides good estimates of fractal exponents, and could adequately complement classical methods, such as spectral analysis, detrended fluctuation analysis or rescaled range analysis.     

Assessing long‐term maintenance of staff performance following behavior skills training in a home‐based setting

We used a multiple baseline across participant design to evaluate the effects of behavior skills training on teaching three behavior therapists to implement discrete trial teaching (DTT) and evaluate the long‐term maintenance of skills acquired through behavioral skills training. For participants whose skills did not maintain, the authors evaluated an independent self‐evaluation procedure on their performance. Following DTT implementation training, maintenance probes were assessed at 2‐, 4‐, 6‐, and 8‐week follow‐ups. The results demonstrated that one participant maintained 100% procedural integrity (PI) through all follow‐ups, one participant decreased below mastery criterion at the 2‐week follow‐up, and one participant dropped below mastery criterion at the 4‐week follow‐up. Those participants that demonstrated decreased accuracy of implementation of DTT programs and were taught to implement a self‐evaluation procedure. Following self‐evaluation, PI maintained for up to 7 weeks for one participant. Our results suggest that if PI does not maintain, self‐evaluation may be a supplementary intervention to increase and maintain PI of new employees.     

Teaching self-control to reduce overt food stealing by children with autism and developmental disorders

Food stealing is often a serious behavioral problem among children with diagnoses of autism and other developmental disorders. Very few empirical studies concerning this behavioral challenge have been reported. We applied a correspondence training procedure to teach self-control as replacement behavior to four children with autism and developmental disorders who displayed food stealing in the community. A changing criterion design embedded within a nonconcurrent multiple-probe across participants design was used. The treatment succeeded for all four participants by increasing latency to eating highly preferred food to a predetermined criterion and reducing occurrences of food stealing to zero. Three participants generalized the replacement behavior to natural settings and maintained the behavior for 2 weeks, 1 month, 2 months, 3 months, and 4 months. One participant without expressive language was taught successfully during treatment trials but failed to maintain and generalize the behavior. A functional relation between delaying food eating and Say-Do correspondence training was demonstrated.     

Defining and Assessing Problem‐Solving Style: Design and Development of a New Tool

VIEW: An Assessment of Problem Solving Style (Selby, Treffinger, & Isaksen, 2002) is a new instrument for assessing problem‐solving style, for use with individuals from ages 12 through adult. It measures three dimensions of style relating to creative problem solving and change management. In this article, we discuss the construction of the instrument, the initial evidence supporting the instrument's reliability and validity, and a very brief overview of the instrument's foundations. Our reliability data involve both stability and internal consistency. We report evidence for the criterion‐related validity, based on correlational studies with relevant measures of learning style, cognitive style, and psychological type. We also conducted principal components factor analyses that support our three‐factor structure. Researchers and practitioners studying and applying Creative Problem Solving and change management methods can use VIEW in several ways. Finally, we identify several research directions that will contribute to the refinement and development of the instrument as well as to a better understanding of the “problem‐solving style” construct.     

Reducing attention‐maintained behavior through the use of positive punishment,differential reinforcement of low rates,and response marking

A differential reinforcement of low (DRL) rates procedure was implemented as a changing criterion design with positive punishment and response marking to reduce attention‐maintained behavior of screaming, profanity, and disruptive behaviors of three adolescent males. One participant was diagnosed with static encephalopathy and attention deficit hyperactivity disorder (ADHD) and severe mental retardation, the other two were diagnosed with Down syndrome, one with moderate mental retardation, and the other with severe mental retardation. Through response marking, a verbal warning was delivered immediately following the initial occurrence of a target behavior. A special token (positive punishment) was immediately delivered with a verbal cue and placed on a behavior board following the next occurrence of that behavior. Appropriate requests (hand‐raising) for attention was acknowledged and reinforced with verbal praise. The DRL procedure, combined with positive punishment and response marking, was successful in decreasing the frequency of targeted behaviors amongst all participants. Copyright © 2009 John Wiley & Sons, Ltd.