Extending exploratory diagnostic classification models: Inferring the effect of covariates

Diagnostic models provide a statistical framework for designing formative assessments by classifying student knowledge profiles according to a collection of fine-grained attributes. The context and ecosystem in which students learn may play an important role in skill mastery, and it is therefore important to develop methods for incorporating student covariates into diagnostic models. Including covariates may provide researchers and practitioners with the ability to evaluate novel interventions or understand the role of background knowledge in attribute mastery. Existing research is designed to include covariates in confirmatory diagnostic models, which are also known as restricted latent class models. We propose new methods for including covariates in exploratory RLCMs that jointly infer the latent structure and evaluate the role of covariates on performance and skill mastery. We present a novel Bayesian formulation and report a Markov chain Monte Carlo algorithm using a Metropolis-within-Gibbs algorithm for approximating the model parameter posterior distribution. We report Monte Carlo simulation evidence regarding the accuracy of our new methods and present results from an application that examines the role of student background knowledge on the mastery of a probability data set.     

Ambiguity attitudes in qualitative contexts: The role of prior beliefs

Most studies of ambiguity aversion rely on experimental paradigms involving monetary bets. Thus, the extent to which ambiguity aversion occurs outside of such contexts is much less understood, particularly when the situation cannot easily be reduced to numerical terms. The present work seeks to understand whether people prefer to avoid ambiguous decisions in a variety of different qualitative domains (e.g., work, family, love, friendship, exercise, study, and health), and, if so, to determine the role played by prior beliefs in those domains. Across three studies, we presented participants with 24 vignettes and measured the degree to which they preferred risk to ambiguity in each. We also asked them for their prior probability estimates about the likely outcomes in the ambiguous events. Ambiguity aversion was observed in the vast majority of vignettes, but at different magnitudes. It was predicted by whether the vignette involved gain or loss as well as by people's prior beliefs; however, the heterogeneity between people meant that the role of prior beliefs was only evident in an individual-level analysis (i.e., not at the group level). Our results suggest that the desire to avoid ambiguity occurs in a wide variety of qualitative contexts but to different degrees for different people and may be partially driven by unfavorable prior estimates of the likely outcomes of the ambiguous events.     

The Fallacy Fallacy: From the Owl of Minerva to the Lark of Arete

The fallacy fallacy is either the misdiagnosis of fallacy or the supposition that the conclusion of a fallacy must be a falsehood. This paper explores the relevance of these and related errors of reasoning for the appraisal of arguments, especially within virtue theories of argumentation. In particular, the fallacy fallacy exemplifies the Owl of Minerva problem, whereby tools devised to understand a norm make possible new ways of violating the norm. Fallacies are such tools and so are vices. Hence a similar problem arises with argumentative vices. Fortunately, both instances of the problem have a common remedy.

     

Organizational change: A strategy for successful implementation

This paper examines the major problems and issues in implementing planned organizational change. It highlights the current debates in the literature on change and itemizes the most important factors which need to be considered by these responsible for implementing change. However, in addition the paper argues that while this provides an initial direction in attempting to bring about organizational change there are, nevertheless, limitations in the literature. The paper overcomes these by developing a broader eight steps model based on case study examples which should provide those responsible for change with a realistic framework of implementation.     

The concurrent validity of three self-report measures of borderline personality

The recent accumulation of self-report measures of borderline personality disorder (BPD) affords the opportunity to evaluate both the construct validity of the concept and the quality of these measures. This study examines the relationship among three recently developed self-report instruments for assessing BPD from the Personality Assessment Inventory (PAI; Morey, 1991), the MMPI Personality Disorders Scales (MPD; Morey, Waugh, & Blashfield, 1985), and the Bell Object Relations Inventory (BORI: Bell, Billington, & Becker, 1986). Data on the three measures were provided by 119 undergraduate subjects from a southeastern university. A correlational analysis addresses the convergence of these measures of BPD, their divergence from measures of different but related traits, and their independence from variance due to method. Application of the Campbell-Fiske (1959) criteria indicates adequate convergence for all the BPD measures but a lack of discriminant validity for the BORI scales. The fit of the data to a structural model of construct validity is tested using confirmatory factor analysis, and these results are consistent with the hypothesis of a latent borderline trait factor independent of measurement method factors. In sum, the construct validity of the borderline personality concept using self-report methodologies receives support, and a strong association between borderline personality and paranoid phenomena is also suggested.     

An alternative procedure for analyzing job analysis results for content-oriented test development

Content-oriented test development requires information derived from a comprehensive job analysis. The job data typically consist of task and skill statements identified by subject matter experts. These statements characterize a target job and are then used to develop the test plan and item budget. The predominant practice for combining data from the task and job skill domains to facilitate preparation of a test plan and item budget is an impressionistic approach. This study reports variations and extensions of an alternative quantitative procedure for linking task and job skill data (Hughes & Prien, 1989) using results obtained in field studies. The results indicate that the procedure reported by Hughes and Prien (1989) yields consistent results. Also, modifications to the procedure result in more detailed solutions.Dr. Charles Lawshe graciously contributed three sets of job analysis data to this research project. Dr. Lawshe also provided a very thoughtful and cogent critique of our attempts to express these thoughts to a user audience and for this we are grateful.     

Calibration of a color monitor for visual psychophysics

It has become common for stimuli used in visual psychophysical experiments to be presented on high-resolution color cathode-ray tubes (CRTs) such as the Barco CDCT 6551. These enable a flexibility of color, spatial-frequency content, temporal-frequency content, duration, size, and position that is not provided by most other media. CRTs are, however, not perfect; they suffer from the effects of temporal instability, spatial variability, lack of phosphor constancy, gun interdependence, and gun nonlinearity. This paper describes methods of assessing these aspects of monitor performance with respect to how significant each may be in psychometric terms. Although every application of CRT use in visual psychophysics is different, some general rules can be formulated to help ensure that unwanted effects are kept to a minimum. For the CRT used in this study (Barco CDCT 6551), a warm-up time of 30-45 min is necessary before chromatic and luminous stability ensues. Restriction of individual gun outputs to within 10%-90% of the possible range ensures that the effects of gun interdependence and lack of phosphor constancy are negligible. Calibration methods dealing with the linearization of gun output are also discussed.