Trifactor Models for Multiple-Ratings Data

In this study we extend and assess the trifactor model for multiple-ratings data in which two different raters give independent scores for the same responses (e.g., in the GRE essay or to subset of PISA constructed-responses). The trifactor model was extended to incorporate a cross-classified data structure (e.g., items and raters) instead of a strictly hierarchical structure. we present a set of simulations to reflect the incompleteness and imbalance in real-world assessments. The effects of the rate of missingness in the data and of ignoring differences among raters are investigated using two sets of simulations. The use of the trifactor model is also illustrated with empirical data analysis using a well-known international large-scale assessment.     

Quality home visits: Activities to promote meaningful interactions

Examination of and support for specific practices that promote high-quality home visiting are essential as family support programs continue to expand across the country. The current study used direct observation of 91 home visits across 41 home visitors to examine relations among interaction partners, content of the interactions, the home-visitors’ activities, and quality of home-visitors’ practices and family-members’ engagement within programs funded by the Maternal, Infant, and Early Childhood Home Visiting program. More time spent in triadic interactions focused on child-related content, as measured by the Home Visit Rating Scale-Revised, was related to higher quality of family engagement in home visits, as measured with the Home Visit Observation Rating Scales. Time spent in adult-focused interactions and administrative tasks, however, was related to lower quality of home-visiting practices and family engagement. Implications for research and practice are discussed.     

让自适应测验更知人善选——基于推荐系统的选题策略

基于推荐系统中协同过滤推荐的思想, 提出两种可以利用已有答题者数据的CAT选题策略：直接基于答题者推荐(DEBR)和间接基于答题者推荐(IEBR)。通过两个模拟研究, 在不同题库和不同长度的测验中, 比较了两种推荐选题策略与两种传统选题策略(FMI和BAS)在测量精度和对题目曝光率控制上的表现, 以及影响推荐选题策略表现的因素。结果发现：两种推荐选题策略对题目曝光率的控制优于两种传统选题策略, 测量精度不亚于BAS方法, 其中DEBR侧重选题精度, IEBR对题目曝光率控制最好。已有答题者数据的特点和质量是影响推荐选题策略表现的主要因素。     

Impact of a comprehensive safety program on bicycle helmet use among middle-school children

A bicycle helmet program was evaluated in three middle schools using a multiple baseline across schools design. Two of the three schools had histories of enforcement of helmet use. During baseline many students riding their bikes to and from school did not wear their helmets or wore them incorrectly. A program that consisted of peer data collection of correct helmet use, education on how to wear a bicycle helmet correctly, peer goal setting, public posting of the percentage of correct helmet use, and shared reinforcers, all of which were implemented by the school resource officer, increased afternoon helmet use and afternoon correct helmet use in all three schools. Probe data collected a distance from all three schools indicated that students did not remove their helmets once they were no longer in close proximity to the school, and probe data collected in the morning at two of the schools showed that the behavior change transferred to the morning.     

A variable neighborhood search method for generalized blockmodeling of two-mode binary matrices

The clustering of two-mode proximity matrices is a challenging combinatorial optimization problem that has important applications in the quantitative social sciences. We focus on one particular type of problem related to the clustering of a two-mode binary matrix, which is relevant to the establishment of generalized blockmodels for social networks. In this context, clusters for the rows of the two-mode matrix intersect with clusters of the columns to form blocks, which should ideally be either complete (all 1s) or null (all 0s). A new procedure based on variable neighborhood search is presented and compared to an existing two-mode K-means clustering algorithm. The new procedure generally provided slightly greater explained variation; however, both methods yielded exceptional recovery of cluster structure.     

A Measure for the Reliability of a Rating Scale Based on Longitudinal Clinical Trial Data

A new measure for reliability of a rating scale is introduced, based on the classical definition of reliability, as the ratio of the true score variance and the total variance. Clinical trial data can be employed to estimate the reliability of the scale in use, whenever repeated measurements are taken. The reliability is estimated from the covariance parameters obtained from a linear mixed model. The method provides a single number to express the reliability of the scale, but allows for the study of the reliability’s time evolution. The method is illustrated using a case study in schizophrenia. The authors are grateful to J&J PRD for kind permission to use their data. We gratefully acknowledge support from the Belgian IUAP/PAI network “Statistical Techniques and Modeling for Complex Substantive Questions with Complex Data.”     

Hierarchical Classes Modeling of Rating Data

Hierarchical classes (HICLAS) models constitute a distinct family of structural models for N-way N-mode data. All members of the family include N simultaneous and linked classifications of the elements of the N modes implied by the data; those classifications are organized in terms of hierarchical, if–then-type relations. Moreover, the models are accompanied by comprehensive, insightful graphical representations. Up to now, the hierarchical classes family has been limited to dichotomous or dichotomized data. In the present paper we propose a novel extension of the family to two-way two-mode rating data (HICLAS-R). The HICLAS-R model preserves the representation of simultaneous and linked classifications as well as of generalized if–then-type relations, and keeps being accompanied by a comprehensive graphical representation. It is shown to bear interesting relationships with classical real-valued two-way component analysis and with methods of optimal scaling. The research reported in this paper was supported by the Research Fund of the University of Leuven (GOA/00/02 and GOA/05/04) and by the Fund for Scientific Research-Flanders (project G.0146.06). Eva Ceulemans is a Post-doctoral Researcher supported by the Fund for Scientific Research, Flanders. The authors gratefully acknowledge the help of Gert Quintiens and Kaatje Bollaerts in collecting the data used in Section 4 and of Jan Schepers in additional analyses of these data.     

Measuring the ability of transitive reasoning,using product and strategy information

Cognitive theories disagree about the processes and the number of abilities involved in transitive reasoning. This led to controversies about the influence of task characteristics on individuals' performance and the development of transitive reasoning. In this study, a computer test was constructed containing 16 transitive reasoning tasks having different characteristics with respect to presentation form, task format, and task content. Both product and strategy information were analyzed to measure the performance of 6- to 13-year-old children. Three methods (MSP, DETECT, and Improved DIMTEST) were used to determine the number of abilities involved and to test the assumptions imposed on the data by item response models. Nonparametric IRT models were used to construct a scale for transitive reasoning. Multiple regression was used to determine the influence of task characteristics on the difficulty level of the tasks. It was concluded that: (1) the qualitatively distinct abilities predicted by Piaget's theory could not be distinguished by means of different dimensions in the data structure; (2) transitive reasoning could be described by one ability, and some task characteristics influenced the difficulty of a task; and (3) strategy information provided a stronger scale than product information.     

Enhancing frequency recording by developmental disabilities treatment staff

We evaluated a staff training and management package for increasing accuracy of recording frequency of problem behavior in a residential care facility. A multiple baseline design across the first and second work shifts showed that 2 of 8 participants increased their accuracy following in-service training, and all 8 improved during a condition with supervisor presence and feedback. Improvements were maintained when feedback was removed and generalized to activity periods when neither supervisor presence nor feedback was provided. Other staff behavior was not adversely affected by the intervention package.     

Consumer brand choice: individual and group analyses of demand elasticity

Following the behavior-analytic tradition of analyzing individual behavior, the present research investigated demand elasticity of individual consumers purchasing supermarket products, and compared individual and group analyses of elasticity. Panel data from 80 UK consumers purchasing 9 product categories (i.e., baked beans, biscuits, breakfast cereals, butter, cheese, fruit juice, instant coffee, margarine and tea) during a 16-week period were used. Elasticity coefficients were calculated for individual consumers with data from all or only 1 product category (intra-consumer elasticities), and for each product category using all data points from all consumers (overall product elasticity) or 1 average data point per consumer (interconsumer elasticity). In addition to this, split-sample elasticity coefficients were obtained for each individual with data from all product categories purchased during weeks 1 to 8 and 9 to 16. The results suggest that: 1) demand elasticity coefficients calculated for individual consumers purchasing supermarket food products are compatible with predictions from economic theory and behavioral economics; 2) overall product elasticities, typically employed in marketing and econometric research, include effects of interconsumer and intraconsumer elasticities; 3) when comparing demand elasticities of different product categories, group and individual analyses yield similar trends; and 4) individual differences in demand elasticity are relatively consistent across time, but do not seem to be consistent across products. These results demonstrate the theoretical, methodological, and managerial relevance of investigating the behavior of individual consumers.