Evaluation of self-contained training procedures for the Time-Sample Behavioral Checklist

The comparative effectiveness of two time-limited modes of training observers to code the behavior of clients in residential treatment programs on the Time-Sample Behavioral Checklist (TSBC) was evaluated. The susceptibility of training procedures to consensual observer drift and the predictability of TSBC mastery from trainee characteristics were also examined. Two equated groups of undergraduate students (N=15 each) participated in full-time training on the TSBC and another instrument for 27 days, followed by criterion testingin vivo and on videotapes. One group was trained by experienced personnel using procedures known to be effective but potentially subject to consensual observer drift. The other group was trained using a previously untested set of written and videotape procedures that do not rely on experienced personnel. Comparative effectiveness and observer drift were evaluated by multivariate and univariate ANOVAs on mastery scores reflecting both pattern agreement and level differences between each trainee and criterion codings. The new, more efficient training procedures were found to be as effective as the original training procedures in the degree of mastery achieved by trainees. Original training procedures were found to be resistant to consensual observer drift, with such a phenomenon appearing in only 1 instance of 156 opportunities. The high degree of mastery achieved by trainees during the time-limited training period was comparable to that previously achieved with the original training procedures. No meaningful predictions of coding mastery were found, with only one trainee characteristic being significantly related to one of six mastery criteria. The results document procedures that are both efficient and resistant to invalidity for training observers in the use of multidimensional observational systems, as well as providing guidelines for the development of standardized procedures.This article is based on a thesis submitted to the Graduate College of the University of Illinois at Urbana-Champaign in partial fulfillment of the requirements for the Ph.D. degree in psychology by the first author and under direction of the second author. The third and fourth authors also participated as supervisors. Appreciation is extended to other members of the thesis committee, Fred Kanfer, Ross Parke, and Julian Rappaport, and to John Gottman and W. Robert Nay for their comments and recommendations. This study was partially supported by Public Health Service Grants MH-25464 and MH-14257 from the National Institute of Mental Health and by grants from the Joyce Foundation, the MacArthur Foundation, and the Illinois Department of Mental Health and Developmental Disabilities.     

A demonstration of the value of interchangeable observers

The remarkably high agreement between observers using the SRIC, the TSBC, and the CFRS makes the observers interchangeable. This agreement is a product of the intensive and extensive training and monitoring of full-time observers, the use of categories rather than continua, the low degree of interpretation required by the procedures, the use of lay language, the immediate recording after short observational periods, and the familiar and standard setting. Other important features include use of rates, emphasis on time, concern for positive behavior, and close linkages between concepts and categories so that concepts are explicitly specified. Taken together, these characteristics make these systems powerful instruments for the guidance of treatment and the prediction of outcomes. The key to their success seems to be the recording of simple actions and action sequences (the basic phenomena of psychology) as the basic data from which a variety of useful indexes can be readily formed.Preparation of this article was supported by Grant MH-30654 from the National Institute of Mental Health to S. Duncan and D. Fiske.Presented at the 87th Annual Meetings of the American Psychological Association, New York City, September 1979, as part of a symposium on New assessment systems for residential treatment, management, research, and evaluation.     

Systems use to objectify program evaluation,clinical, and management decisions

The utility of the observational assessment systems at different levels — from local clinical to systemwide management — is outlined. An overview is provided of TSBC information applied to individualized problem identification and programming/monitoring, discharge and competency determinations, and both absolute and comparative program evaluation. The utility of SRIC information for prgoramming/monitoring and for staff training and evaluation is outlined in addition to the applied uses of the information of the assessment systems in combination. Once implemented, the continuous data from the systems allow for empirically based self-corrective improvements in the quality of mental health services while automatically providing a basis for legal documentation and accurate cost/effectiveness comparisons of mental health programs.Preparation of this article and the research and development on which the article is based were supported, in part, by Public Health Service Grants MH-15553 and MH-25464 from the National Institute of Mental Health, and by grants from The Joyce Foundation and the Illinois Department of Mental Health and Developmental Disabilities.Presented at the 87th Annual Meetings of the American Psychological Association, New York City, September 1979, as part of a symposium on New assessment systems for residential treatment, management, research, and evaluation.     

Judgmental relativism as a validity threat to standardized psychiatric rating scales

Judgemental relativism is a threat to the replicability and validity of measures of client behavior from direct rating scales whenever raters are exposed to different levels of client functioning since the internal standards, or anchor points, used to judge dimensional continua may vary on the basis of prior experience. Traditional interrater reliability indexes fail to identify such effects. The influence of judgmental relativism on summated ratings from the Nurses Observational Scale for Inpatient Evaluation (NOSIE-30) for 1040 adult mentally ill clients was examined with clinical staff raters from 24 treatment units in which the Time-Sample Behavioral Checklist (TSBC) provided full-week objective measures of actual client functioning via hourly direct observational coding (DOC). Regression analyses found that the same level of objective performance received higher or lower ratings across treatment units dependent on the raters'exposure to client groups that differed in level of functioning. Analyses of rating errors found that clients with better levels of functioning relative to others within treatment units were rated even higher than performance warranted. The operation of halo and contrast effects is explored and guidelines are provided for determining when judgmental relativism may produce or nullify significant differences. DOC assessments should be used instead of retrospective ratings to support most decisions in residential settings. Specific recommendations for the application of rating scales and improving data quality are provided.This study was the basis of a master's thesis at the University of Houston by Betty E. Rich under the direction of Gordon L. Paul and Marco J. Mariotto. Richard M. Rozelle, to whom appreciation is expressed for helpful comments, served on the examination committee. This study was partially supported by grants to Gordon L. Paul from the National Institute of Mental Health, Public Health Service (MH-15353; MH-25464); the Illinois Department of Mental Health and Developmental Disabilities; the Joyce Foundation; the MacArthur Foundation; the Owsley Foundation; the Cullen Foundation; and the Center for Public Policy, University of Houston.     

Spinners and Chartmoves:

A practical child management procedure is presented which incorporates self-recording, variable schedules of reinforcement, and a practical data collection system. The proposed system has two basic components, chartmoves and spinners. The chart is generally a picture with 30 to 100 dots and functions as a data collection system and indicates when a reinforcer has been programmed. Reinforcers are set up by randomly placing large dots (reinforcer dot), which make up the picture. When a child exhibits an appropriate target behavior, he is allowed to connect two dots on the picture. Each dot is covered with a scratch off ink; if the last dot connected is large the child receives a reinforcer. Chartmoves (connection of dots) are dated and initialed by the teacher allowing a record of progress to be kept with little effort by the teacher or parent. The second component to the system is the spinner. The spinner is an unevenly divided circle (generally five sections) with each section containing a reinforcer, plus an arrow, which the child spins to indicate which reinforcer he will receive. The combined system both the chart and spinner represents two variable schedules of reinforcement (the random reinforcement dots and the spinner), the child self-records the data on the chart, and the random aspect of the spinner reduces reinforcement satiation. This system has been used successfully with elementary ages children in the classroom, in the home as a contingency management program, and as a home and school program in which school behaviors are consequated in the home with the system.     

The Datafication of Learning: Data Technologies as Reflection Issue in the System of Education

Like other parts of the social system, education is becoming an information-driven venture: data technologies pervade all levels of the system. This datafication of education seems to take place alongside a general turn to learning that Gert Biesta has called learnification: a progressively singular focus on the manipulable features of individual learning in education. Given rapidly rising levels of datafication, it seems timely to take up Luhmann and Schorr’s contention that education entails a technology deficit and discuss datafication as reflection issue in the system of education. Against their argument that human learning is not amenable to a technology, I develop the counter-argument that data technologies are replacing human learning outright with data at the level of organisation. Data thus present a concretely digital form of what Raf Vanderstraeten has called education as an ersatz order. In a data-driven form of organising education, human dimensions of learning become secondary to a systemic dimension: making learning visible as data and so susceptible to databased manipulation. The text treats school-wide positive behaviour support interventions as an evidence-based exemplar of this trend towards datafication in the system of education.     

Uncovering mental processes with factorial experiments

The analysis of reaction time (RT) additivity and interactions via factorial manipulation is a widely used and potentially powerful tool for elucidating mental processing; however, current implementation is limited by the scope of systems for which predictions are available, as well as statistical weakness. Predictions for an expanded set of system dimensions (serial/parallel, independent/dependent, and selective/nonselective influence) are given. The theorems are not limited to particular families of distributions. Statistical considerations are examined in the context of an exemplary nonadditive parallel system which adequately fits “additive” data but will only be rejected as additive when power is increased beyond the traditional criteria.     

Convergent methods in psychomusical research based on integrated,interactive computer control

Much of what we learn from an experiment depends on what we ask of the subjects and how we ask it. Most music perception and cognition research is implemented using a single methodology. Recently, questions have been raised about the validity and generalizability of such a limited approach, particularly regarding such complicated theoretical constructs as musical tonality. We propose that a useful technique is to converge on the answer to experimental questions by applying multiple methods, in essence, simultaneously investigating the central research question as well as ancillary questions of method. Such an approach is facilitated by a computer hardware and software system that integrates experimental design, data collection, data analysis, and data dissemination, linking off-the-shelf components through standard file formats and data-exchange protocols. The list-based system, using an IBM-type PC running Windows 3.0, handles real-time natural sound sampling and playback, signal synthesis and analysis, MIDI input and output, digitized picture display, and various subject-response mechanisms. On the basis of graphical icons, the integrated system can be used to build custom experimental designs. Examples of convergent psychomusical experiments that were designed, implemented, and analyzed using the system are discussed.     

An APL system for interactive data analysis: Histograms,correlation, and regression

This report describes an APL system for file-oriented univariate analysis, correlation, and regression analysis. The correlation and regression processes permit analyses of an unlimited number of cases, if the cases are arranged in tables small enough to be processed one at a time in a workspace. Support functions are also given for a menu-based processing system designed to support a broad range of analyses. The functions offer features such as subset selection, data transformation, and exclusion of invalid items.     

An improved shocking surface for pigeons and rats

A resistive shocking surface and an appropriate power supply are described. The efficacy of the system is illustrated with some data on the activity of pigeons and rats as a function of the voltage applied.     

Research about the university teaching performance evaluation under the data envelopment method

How to take active and effective measures to evaluate the university scientifically and rationally has been an eternal topic that the educational circles are constantly exploring. Based on the principle of index construction, the current educational performance evaluation index system is improved and a more reasonable evaluation index system is formed. On this basis, taking the sample data in 2017 as an example, the principal component analysis method is used to reduce the dimension of input and output indicators and eliminate the correlation between indicators, and three principal components of input and three principal components of output are obtained. Secondly, data envelopment analysis model is established, and the data processed are analyzed with the help of MATLAB and DEAP2.1 operation software. The efficiency of these 24 colleges and universities is compared to understand the efficiency and differences of each college. Moreover, projection analysis of non-DEA effective DMU is completed and the direction of improvement and the specific adjustment value are pointed out.     

Apple II/FIRST system control of electrical brain stimulation in the rabbit

An Apple II/FIRST system has been developed to control classical conditioning experiments, collect analog data, and extract dependent variable measures of conditioning from uniphasic (Scandrett & Gormezano, 1980) and multiphasic (Johnson, 1981) response signals. The present paper details the development of an electrical brain stimulation system as a peripheral device to the Apple II/FIRST system. In addition, data are presented that document the capabilities of the EBS system to concurrently stimulate up to eight animals while values along the dimension of each of four stimulation parameters are manipulated by a program written in FIRST. In our judgment, the EBS system has a number of advantages over currently available commercial stimulators, including cost, number of sites that can be independently stimulated, and availability of complete software control.     

Of snakes and faces: An evolutionary perspective on the psychology of fear

Snakes and faces are unique stimuli because they are deeply grounded in evolutionarily shaped behavior systems. Snakes are the prototypical stimulus in a predatory defense system promoting escape from predators, which prepares primates for efficient processing, attentional priority, and rapid fear acquisition to snakes and other stimuli related to predation. The social submissiveness system plays a similar role on the social arena by promoting yielding to dominant individuals without jeopardizing the protection of the group. Supporting these theoretical propositions, empirical data from a research program spanning four decades demonstrate enhanced fear conditioning to snakes and threatening faces compared to neutral stimuli, as well as fast nonconscious processing of, and prioritized attention to, snakes and threatening faces. Human brain‐imaging data show that these effects are mediated by an extensive fear‐network centering on the amygdala.     

计算机动态测验中问题解决过程策略的分析：多水平混合IRT模型的拓展与应用

学生在完成计算机动态测验过程中, 会产生大量带有时间标记的过程性数据。本研究基于5个国家(地区) 3196名学生在PISA2012一道交通问题解决任务上的139990条数据, 将多水平混合IRT (MMixIRT)模型进行拓展, 用于探索问题解决过程策略的类别特点。结果表明, 该模型不仅可以基于行为序列对不同国家(地区)学生在解决问题时策略使用情况的典型特征进行分析, 还可以提供个体水平的能力估计值。拓展的MMixIRT模型可用于分析过程性数据的特征。     

Using a laboratory computer for identification of nonlinear behavioral and electrophysiological systems

A system for mathematically identifying unknown nonlinear systems based upon Wiener kernels and G-functionals is presented as an example of the sophisticated analyses of electrophysiological and behavioral data made possible by the introduction of powerful digital computers to the laboratory environment. The applicability of these methods to a wide variety of systems is emphasized. By measuring the response of the unknown system to appropriate white noise stimulation, data may be obtained which are sufficient for the characterization of the system under test. The only restrictions upon the unknown system is that it be approximately stationary during the period of measurement and that its output depend upon its input in the finite past. Various pretests to evaluate the applicability of the method are presented. Simple FORTRAN programs that clarify the methods of computation are given for the preprocessing of data, the analysis of lower-order kernels, the synthesis of system response from the computed kernels, and the computation of the proportion of the variance in the systems output which is accounted for by the kernels zero, one, and two. These programs may be implemented on a digital laboratory computer of moderate size. Analysis may be performed either on-line or after data collection, depending upon the temporal characteristics of the system under test. This method represents one of the few analytical techniques presently available for the characterization of both linear and nonlinear systems. For this reason, it constitutes a uniquely powerful tool for many branches of psychology. Successful applications of this method to electrophysiological and behavioral data are mentioned. Limiting factors in the use of Wiener kernel analysis are also discussed.     

Andisp: A Turbo pascal program to analyze and display sampled data

ANDISP is a Turbo Pascal program that analyzes, displays, and manipulates sampled data recorded with a PC-compatible data-acquisition system The data analysis routines include “integrate/differentiate a channel”, “rectify,” “smooth,” “filter,” and “add/sub/mult/div a channel by a constant, ” as well as a routine that gives some basic statistics. The data manipulation routines are “copy a channel” or “copy the data buffer to a file.” The versatile display routine can give either oscilloscope-type displays of a data channel or plots of pubUshable quality. Hardware requirements are an IBM-compatible microcomputer and an EGA color monitor.     

Application of chaos theory to biology and medicine.

The application of "chaos theory" to the physical and chemical sciences has resolved some long-standing problems, such as how to calculate a turbulent event in fluid dynamics or how to quantify the pathway of a molecule during Brownian motion. Biology and medicine also have unresolved problems, such as how to predict the occurrence of lethal arrhythmias or epileptic seizures. The quantification of a chaotic system, such as the nervous system, can occur by calculating the correlation dimension (D2) of a sample of the data that the system generates. For biological systems, the point correlation dimension (PD2) has an advantage in that it does not presume stationarity of the data, as the D2 algorithm must, and thus can track the transient non-stationarities that occur when the systems changes state. Such non-stationarities arise during normal functioning (e.g., during an event-related potential) or in pathology (e.g., in epilepsy or cardiac arrhythmogenesis). When stochastic analyses, such as the standard deviation or power spectrum, are performed on the same data they often have a reduced sensitivity and specificity compared to the dimensional measures. For example, a reduced standard deviation of heartbeat intervals can predict increased mortality in a group of cardiac subjects, each of which has a reduced standard deviation, but it cannot specify which individuals will or will not manifest lethal arrhythmogenesis; in contrast, the PD2 of the very same data can specify which patients will manifest sudden death. The explanation for the greater sensitivity and specificity of the dimensional measures is that they are deterministic, and thus are more accurate in quantifying the time-series. This accuracy appears to be significant in detecting pathology in biological systems, and thus the use of deterministic measures may lead to breakthroughs in the diagnosis and treatment of some medical disorders.     

Error monitoring in speech production: a computational test of the perceptual loop theory

A theory of speech monitoring, proposed by Levelt (1983), assumes that the quality of one's speech is checked by the speech comprehension system. This system inspects one's own overt speech but would also inspect an inner speech plan ("the inner loop"). We have elaborated and tested this theory by way of formalizing it as a computational model. This model includes a new proposal concerning the timing relation between planning the interruption and the repair: the proposal that these two processes are performed in parallel. We attempted to simulate empirical data about the distribution of error-to-cutoff and cutoff-to-repair intervals and the effect of speech rate on these intervals (these intervals are shorter with faster speech). The main questions were (1) Is an inner monitor that utilizes the speech perception system fast enough to simulate the timing data? (2) Can the model account for the effects of speech rate on these intervals? We conclude that including an inner loop through the speech comprehension system generates predictions that fit the empirical data. The effects of speed can be accounted for, given our proposal about the time course of planning interruption and repair. A novel prediction is that the error-to-cutoff interval decreases with increasing position in the phrase.     

A system for recording individualized behavioral data in a token program

A data matrix system used to record and summarize individual behavioral data is described. The major characteristics of the system are: (1) it provides a place to record and summarize all patient token exchanges, whether for standard contingencies or for idiosyncratic behavior; (2) it serves as a record and summary for time samples observed during a 24-hr period and as a record of reliability data on time sample measures; (3) it provides a place where the new token balance for each patient can be calculated and recorded for use the next day; (4) it provides for calculation of an individual patient's token balance at any given moment; and (5) it is a permanent record of each patient's daily participation in the program and documents progress in terms of his economic status as well as in changes in critical target behaviors.