Comparison of observational recordings in vivo, via mirror, and via television

Simultaneous observational recordings were made in vivo, via an observation mirror, and via closed circuit television. Three of nine observers had extensive experience recording behavior in vivo; three had extensive experience recording behavior via mirror; and three had extensive experience recording via television. Observers recorded nine categories of disruptive behavior for children in a special class setting. Frequencies of behavior recorded in vivo, via mirror, and via television differed significantly for only one category, vocalization. There were no significant main effects or interactions involving the observers' previous experience. Occurrence reliability coefficients computed within and between media demonstrated the similarity of observer agreement in all three media. Data collection procedures using an observational mirror or closed circuit television appear to be reasonable alternatives to in vivo observation in circumstances similar to those in the present study.     

Percentage agreement and phi: A conversion table

STUDIES IN APPLIED BEHAVIOR ANALYSIS HAVE USED TWO EXPRESSIONS OF RELIABILITY FOR HUMAN OBSERVATIONS: percentage agreement (including percentage occurrence and percentage nonoccurrence agreement) and correlational techniques (including the phi coefficient). The formal relationship between these two expressions is demonstrated, and a table for converting percentage agreement to phi, or vice-versa, is presented. It is suggested that both expressions be reported in order to communicate reliability unambiguously and to facilitate comparison of the reliabilities from different studies.     

Considerations in the choice of interobserver reliability estimates

Two types of interobserver reliability values may be needed in treatment studies in which observers constitute the primary data-acquisition system: trial reliability and the reliability of the composite unit or score which is subsequently analyzed, e.g., daily or weekly session totals. Two approaches to determining interobserver reliability are described: percentage agreement and "correlational" measures of reliability. The interpretation of these estimates, factors affecting their magnitude, and the advantages and limitations of each approach are presented.     

EMPLOYING TASK ARRANGEMENTS AND VERBAL CONTINGENCIES TO PROMOTE VERBALIZATIONS BETWEEN RETARDED CHILDREN1

This study investigated the effects of arranging task events for interdependence, to increase the probability of social responding. During task interdependence, the subjects, participating in dyads and a four-person group, obtained task materials (a puzzle piece) from their partner before completing their task (appropriately placing the puzzle piece). The verbal contingency required a verbal request to precede a subject's receiving a task material from his partner. The verbal contingency yoked with task interdependence made task completion contingent on the appropriate verbalization. The findings from two experiments suggested that task interdependence was sufficient to increase partner-directed verbalizations for three of the four subjects. When the verbal contingency was added, all subjects increased their requests and other verbalizations to partner. Applied to a four-person group, the verbal contingency yoked with varying levels of task interdependence correspondingly affected the pattern and level of group communications. The greater the task interdependence, i.e., the more members each subject depended on to complete his task, the more complex the social network of verbal contacts, and the higher the level of both requests and other verbalizations for the group.     

Correlation and prediction in ordinal test theory

Based on the test theory model for ordinal measurements proposed by Schulman and Haden, the present paper considers correlations between tests, attenuation, regressions involving true and observed scores, and prediction of test reliability.The population correlation between tests is shown to be related to the expected sample correlation for samples of sizen ₁ andn ₂. Errors of estimation, measurement and prediction are found to be similar to their counterparts in interval test theory, while attenuation is identical to its counterpart. The bias in estimating population reliability from sample data is compared for Kendall's tau and Spearman's rho.The author wishes to thank the referees for their helpful comments on an earlier draft of this paper, and in particular, for the suggested alternative methods of establishing some of the presented results.     

Reliability of variable message signs information: A field experiment study in Chongqing

A variable message sign (VMS), which is a key component of intelligent transportation systems, has been frequently used in the management of urban roads and motorways to provide drivers with real-time traffic condition information about a road section or area. Nevertheless, there is a lack of unified regulations for VMS design, which makes it difficult to accurately determine the understandability and legibility of VMS information. In practice, inappropriate designs of VMSs are common, such as overload of VMS information and excessive number of phases, particularly on the urban roads of China. Building on our earlier findings obtained by surveys, in this study, field tests were conducted to assess the reliability of VMS information in terms of the visual perception characteristics of drivers. For a full visual perception, the information displayed on a VMS must be reliable, i.e., it should be easy for drivers to detect, obtain, and understand the messages they require. Two basic theories—information theory and visual perception theory—were introduced to build a quantification model of VMS information, select evaluation indices, and design the experimental process. A total of 24 drivers participated in the field experiments and questionnaires. Five indices—information obtainment rate, periodic validity, driver subjective scoring and the corresponding UI (represents information intelligibility), and CE (represents information legibility)—were analyzed to evaluate their relationships to VMS information reliability. The results confirmed that the amount of information, and the scrolling period considering redundant distance, significantly affect the reliability of VMSs information. For static VMSs, the information obtainment rate and the subjective scoring decrease with increasing amount of information. The recollection accuracy of the drivers significantly declines when the amount of information shown increases to 90 bits, corresponding to an information obtainment rate of less than 0.8 and UI equal to 0. For dynamic VMSs, the information reliability deteriorates as the scrolling period shortens and the number of phases increases. Unreasonable reliability is found when the periodic validity is less than 0.9, i.e., the actual scrolling period is more than 10% less than the calculated one, corresponding to CE equal to 0. The reliability of information was evaluated by combining the subjective scoring of the drivers and a data-based statistical analysis and considering driving safety. Accordingly, it was recommended that 90 bit is the maximum amount of information to be shown on a VMS and that the preferred scrolling period of each phase of a dynamic VMS is 5 s. The results of this study support the objective of providing reliable information to drivers by addressing the problems related to the amount of information and presentation time for VMSs. These findings provide a basis for determining the thresholds for VMS information to promote practical and user-friendly designs of VMSs on urban roads.     

Improved Test–retest Reliability Criteria as Applied to Probability Assessments

Test–retest reliability is a common indicator of response consistency. It is argued that using regression coefficients for detecting systematic response error is less appropriate than testing for shifts in the mean (median) and variance. This procedure is exemplified using probability response data. For this data, shifts in centrality were found to be about 2.5 times more likely than shifts in variability. The shifts in centrality did not favor any particular direction; however, variability tended to decrease over time in early sessions.     

Indicators of Affect Dynamics: Structure,Reliability, and Personality Correlates

Researchers are increasingly interested in the affect dynamics of individuals for describing and explaining personality and psychopathology. Recently, the incremental validity of more complex indicators of affect dynamics (IADs; e.g. autoregression) has been called into question (Dejonckheere et al., 2019), with evidence accumulating that these might convey little unique information beyond mean level and general variability of emotions. Our study extends the evidence for the construct validity of IADs by investigating their redundancy and uniqueness, split-half reliability based on indices from odd-numbered and even-numbered days, and association with big five personality traits. We used three diverse samples that assessed daily and momentary emotions, including community participants, individuals with personality pathology, and their significant others (total N = 1192, total number of occasions = 51 278). Mean and variability of affects had high reliability and distinct nomological patterns to big five personality traits. In contrast, more complex IADs exhibited substantial redundancies with mean level and general variability of emotions. When partialing out these redundancies by using residual variables, some of the more complex IADs had acceptable reliability, but only a few of these showed incremental associations with big five personality traits, indicating that IADs have limited validity using the current assessment practices. © 2020 The Authors. European Journal of Personality published by John Wiley & Sons Ltd on behalf of European Association of Personality Psychology     

The norms and stability of new measures of the multidimensional body image construct

The measurement of the body image construct has become increasingly multidimensional. New assessments are available to measure various facets of body image functioning: The Body Image Disturbance Questionnaire, Appearance Schemas Inventory-Revised, Body Image Coping Strategies Inventory, and Body Image Quality of Life Inventory. The present article reports normative data and acceptable internal reliabilities across seven completed investigations using these measures with female and male college students. Sex differences were examined and found, as expected, for most of the measures. In addition, a study evaluated an undetermined but essential psychometric property of these facets of the construct—their test–retest reliability. In a sample of 107 college students, the 2-week stability of these measures was found to be acceptable. Several directions for future research with these assessments are identified.