Consistency of Cluster Analysis for Cognitive Diagnosis: The Reduced Reparameterized Unified Model and the General Diagnostic Model

The asymptotic classification theory of cognitive diagnosis (ACTCD) provided the theoretical foundation for using clustering methods that do not rely on a parametric statistical model for assigning examinees to proficiency classes. Like general diagnostic classification models, clustering methods can be useful in situations where the true diagnostic classification model (DCM) underlying the data is unknown and possibly misspecified, or the items of a test conform to a mix of multiple DCMs. Clustering methods can also be an option when fitting advanced and complex DCMs encounters computational difficulties. These can range from the use of excessive CPU times to plain computational infeasibility. However, the propositions of the ACTCD have only been proven for the Deterministic Input Noisy Output “AND” gate (DINA) model and the Deterministic Input Noisy Output “OR” gate (DINO) model. For other DCMs, there does not exist a theoretical justification to use clustering for assigning examinees to proficiency classes. But if clustering is to be used legitimately, then the ACTCD must cover a larger number of DCMs than just the DINA model and the DINO model. Thus, the purpose of this article is to prove the theoretical propositions of the ACTCD for two other important DCMs, the Reduced Reparameterized Unified Model and the General Diagnostic Model.     

Joint Maximum Likelihood Estimation for Diagnostic Classification Models

Joint maximum likelihood estimation (JMLE) is developed for diagnostic classification models (DCMs). JMLE has been barely used in Psychometrics because JMLE parameter estimators typically lack statistical consistency. The JMLE procedure presented here resolves the consistency issue by incorporating an external, statistically consistent estimator of examinees’ proficiency class membership into the joint likelihood function, which subsequently allows for the construction of item parameter estimators that also have the consistency property. Consistency of the JMLE parameter estimators is established within the framework of general DCMs: The JMLE parameter estimators are derived for the Loglinear Cognitive Diagnosis Model (LCDM). Two consistency theorems are proven for the LCDM. Using the framework of general DCMs makes the results and proofs also applicable to DCMs that can be expressed as submodels of the LCDM. Simulation studies are reported for evaluating the performance of JMLE when used with tests of varying length and different numbers of attributes. As a practical application, JMLE is also used with “real world” educational data collected with a language proficiency test.     

Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions

Traditional testing procedures typically utilize unidimensional item response theory (IRT) models to provide a single, continuous estimate of a student’s overall ability. Advances in psychometrics have focused on measuring multiple dimensions of ability to provide more detailed feedback for students, teachers, and other stakeholders. Diagnostic classification models (DCMs) provide multidimensional feedback by using categorical latent variables that represent distinct skills underlying a test that students may or may not have mastered. The Scaling Individuals and Classifying Misconceptions (SICM) model is presented as a combination of a unidimensional IRT model and a DCM where the categorical latent variables represent misconceptions instead of skills. In addition to an estimate of ability along a latent continuum, the SICM model provides multidimensional, diagnostic feedback in the form of statistical estimates of probabilities that students have certain misconceptions. Through an empirical data analysis, we show how this additional feedback can be used by stakeholders to tailor instruction for students’ needs. We also provide results from a simulation study that demonstrate that the SICM MCMC estimation algorithm yields reasonably accurate estimates under large-scale testing conditions.     

Accountability: a social magnifier of the dilution effect

This research demonstrated that accountability can not only reduce judgmental bias, but also exacerbate it--in this case, the dilution effect. Ss made predictions from either diagnostic information alone or diagnostic information plus mixtures of additional data (nondiagnostic information, additional diagnostic data pointing to either the same conclusion or the opposite conclusion). Relative to unaccountable Ss, accountable Ss (a) diluted their predictions in response to nondiagnostic information and (b) were more responsive to additional diagnostic information. The accountability manipulation motivated subjects to use a wide range of information in making judgments, but did not make them more discriminating judges of the usefulness of that information.     

Invariance Properties for General Diagnostic Classification Models

In item response theory (IRT), the invariance property states that item parameter estimates are independent of the examinee sample, and examinee ability estimates are independent of the test items. While this property has long been established and understood by the measurement community for IRT models, the same cannot be said for diagnostic classification models (DCMs). DCMs are a newer class of psychometric models that are designed to classify examinees according to levels of categorical latent traits. We examined the invariance property for general DCMs using the log-linear cognitive diagnosis model (LCDM) framework. We conducted a simulation study to examine the degree to which theoretical invariance of LCDM classifications and item parameter estimates can be observed under various sample and test characteristics. Results illustrated that LCDM classifications and item parameter estimates show clear invariance when adequate model data fit is present. To demonstrate the implications of this important property, we conducted additional analyses to show that using pre-calibrated tests to classify examinees provided consistent classifications across calibration samples with varying mastery profile distributions and across tests with varying difficulties.     

Choosing to Enter or Avoid Diagnostic Social Situations

ABSTRACT— Three studies suggest that people control the nature of their relationships, in part, by choosing to enter (or avoid) situations providing feedback about other people's social interest. In Study 1 , chronically avoidant individuals (but not others) preferred social options that would provide no information about other people's evaluations of them over social options that would, but did not prefer nondiagnostic situations more generally. In Study 2 , chronically avoidant students (but not others) in a methods class preferred to have their teacher assign them to working groups (a nondiagnostic situation) over forming their own groups (a diagnostic situation). In Study 3 , individuals experimentally primed to feel avoidant were less likely than those primed to feel secure to choose to receive feedback about how another person felt about them. Overall, the research suggests that choices of socially diagnostic versus socially nondiagnostic situations play an important role in guiding people's social relationships.     

部分呈现条件下类别分类学习中的信息表征机制

先前研究者普遍认为, 类别推理学习条件下可以同时表征诊断性信息和非诊断性信息, 而类别分类学习条件下中只能表征诊断性信息, 不能表征非诊断性信息。而最近又有研究者发现部分呈现条件下的类别分类学习可以表征非诊断性信息。本研究通过两个实验系统比较了全部呈现和部分呈现条件下类别分类学习的结果, 进一步探讨了分类学习条件下信息的表征情况, 并进一步探讨了部分呈现条件下的分类学习能够表征非诊断性信息的原因。实验1发现全部呈现6个特征、缺失1个特征(即部分呈现5个特征)、缺失2个特征(即部分呈现4个特征)3种条件下都能表征诊断性信息, 但只有部分呈现条件下能表征非诊断性信息。实验2发现全部呈现7个特征、缺失2个特征(即部分呈现5个特征)、全部呈现5个特征3种条件下都能表征诊断性信息, 但只有部分呈现条件下能表征非诊断性信息。总的实验结果表明：全部呈现条件下的分类学习只能表征诊断性信息, 而部分呈现条件下的分类学习能够同时表征诊断性信息和非诊断性信息, 并且部分呈现条件下表征非诊断性信息的原因是被试进行了推理学习, 而非注意广度的变化。     

Identifiability of Diagnostic Classification Models

Diagnostic classification models (DCMs) are important statistical tools in cognitive diagnosis. In this paper, we consider the issue of their identifiability. In particular, we focus on one basic and popular model, the DINA model. We propose sufficient and necessary conditions under which the model parameters are identifiable from the data. The consequences, in terms of the consistency of parameter estimates, of fulfilling or failing to fulfill these conditions are illustrated via simulation. The results can be easily extended to the DINO model through the duality of the DINA and DINO models. Moreover, the proposed theoretical framework could be applied to study the identifiability issue of other DCMs.     

Separating the wheat from the chaff: does discriminating between diagnostic and nondiagnostic information eliminate the dilution effect?

The dilution effect refers to the finding that judgments are often unduly influenced by nondiagnostic information, producing regressive judgment. Because the dilution effect is a problem in various domains, strategies to control the impact of nondiagnostic information were explored by drawing on a perceptual and a conversational account of the dilution effect. Three experiments (n = 259) demonstrate that explicit instructions to discriminate between diagnostic and nondiagnostic information did not reduce the dilution effect. Rather, consistent with a perceptual explanation but not consistent with a conversational explanation, the dilution effect disappeared only when participants engage in perceptual control, that is, when they actively remove nondiagnostic pieces of information before making a judgment. Copyright © 2004 John Wiley & Sons, Ltd.     

Hierarchical Diagnostic Classification Models: A Family of Models for Estimating and Testing Attribute Hierarchies

Although latent attributes that follow a hierarchical structure are anticipated in many areas of educational and psychological assessment, current psychometric models are limited in their capacity to objectively evaluate the presence of such attribute hierarchies. This paper introduces the Hierarchical Diagnostic Classification Model (HDCM), which adapts the Log-linear Cognitive Diagnosis Model to cases where attribute hierarchies are present. The utility of the HDCM is demonstrated through simulation and by an empirical example. Simulation study results show the HDCM is efficiently estimated and can accurately test for the presence of an attribute hierarchy statistically, a feature not possible when using more commonly used DCMs. Empirically, the HDCM is used to test for the presence of a suspected attribute hierarchy in a test of English grammar, confirming the data is more adequately represented by hierarchical attribute structure when compared to a crossed, or nonhierarchical structure.     

Ethics in Case Conceptualization and Diagnosis: Incorporating a Medical Model Into the Developmental Counseling Tradition

In this literature review, the author focuses on several ethical considerations in case conceptualization and diagnosis, including diagnostic training and competence. Meeting the American Counseling Association's (1995) ethical standard for diagnostic training has several ethical implications for counselors, counselor educators, and supervisors. For counselors who might struggle with how to meet their ethical responsibilities in diagnosis but who want to remain true to their developmental counseling emphases, the author discusses some of their concerns, the implications of and possible approaches to this aspect of their work. Conclusion Yalom (2002) asked a poignant question of counselors in his book, The Gift of Therapy: “If you were in personal psychotherapy or are considering it, what DSM‐IV diagnosis do you think your therapist could justifiably use to describe someone as complicated as you?” (p. 5). This question and continued dialogue about the ethics and implications of diagnosis are essential aspects of diagnostic training. Yalom's poignant and deeply personal question seems especially appropriate for increasing a counselor's empathy toward the client's sensitivity and vulnerability during the diagnostic process. Counselor educators might ask how one remains true to a developmental model of counseling while adhering to the ethical and accreditation standards of teaching the DSM's medical model of diagnosis. Counselors may also question how to use diagnosis ethically and empathically. Seligman (1999) recommended that clinicians view the DSM as one of many important sources of information about a person. Furthermore, counselors should seek to incorporate diagnostic information into a holistic context, recognizing that a diagnosis does not reflect the totality of the client. Some counselor educators have advised students to integrate the DSM model into their work with clients rather than abandoning their developmental roots (Waldo et al., 1993). Some counselors may not actually put their diagnoses in writing; Seligman believed, however, that thinking diagnostically may assist counselors in determining the best approaches to help clients and to help clients help themselves. This clinical and ethical debate about how, and in fact, whether, to integrate the medical model of the DSM and the developmental origins and distinctiveness of counseling continues. However, the CACREP (2001) standards, managed care systems, and other forces have pushed counseling professionals toward a medical model by mandating counselor knowledge and use of the DSM. Whatever a counselor's stance and behavior on client assessment and diagnosis may be, the literature presented in this review and discussion seems to suggest a need for heightened sensitivity to, preparation for, and accuracy in all facets of client assessment, especially diagnosis.     

选择作业中证伪思维的影响因素

研究了选择作业中证伪思维的影响因素。被试为星海音乐学院本科生592名。实验1、2研究了问题的诊断性对证伪思维的影响;实验3研究了提示证伪样例是否促进证伪;实验4研究了证真证伪相对难度改变是否影响证伪。结果表明：（1）证真和证伪都为诊断性问题,人们倾向于证真;证真为非诊断问题,证伪为诊断性问题,人们倾向于证伪。（2）提示证伪样例,可以促进证伪。（3）证真难度加大,可以促进证伪。结合正反信息转换系统的心理模型理论,可以很好地解释该研究结果。     

基于分部评分模型思路的多级评分认知诊断模型开发

基于分部评分模型的思路, 本文提出了一般化的分部评分认知诊断模型(General Partial Credit Diagnostic Model, GPCDM), 与国际上已有的基于分部评分模型思路的多级评分模型GDM (von Davier, 2008)和PC-DINA (de la Torre, 2012)相比, GPCDM的Q矩阵定义更加灵活, 项目参数的约束条件更少。Monte Carlo实验研究表明, GPCDM模型的参数估计精度指标RMSE介于[0.015, 0.043], 表明估计精度尚可; TIMSS (2007)实证数据应用研究表明, 与GDM和PC-DINA模型相比, GPCDM与该数据的拟合度更好, 并且使用GPCDM分析该数据的诊断效果也更优。总之, 本研究提供了一种约束条件更少、功能更为强大的多级评分认知诊断模型。     

重参数化的多分属性诊断分类模型及其判准率影响因素

多分属性比传统的二分属性提供更多更详细的诊断反馈信息, 符合对知识技能的多水平要求, 具有较好的应用前景。本文首先介绍了多分属性和多分Q矩阵的概念; 之后重参数化了3个分别满足连接、分离和补偿缩合规则的多分属性诊断分类模型并研究了其判准率影响因素, 结果发现它们的判准率(1)均随多分属性数量的增加而降低, 建议实际使用中不宜高于5个; (2)均随多分属性的最高水平数增加而降低, 建议实际使用中不宜高于4水平; (3)均随多分属性间统计相关性增加而增加, 但影响不大; (4)受多分属性层级结构的影响较大; (4)受被试量影响不大; (5)均随题目数量增加而增加且影响较大。最后, 针对“多分属性与多级评分的关系”和“多分属性与二分属性之间的关系”这两个问题进行了讨论。以期为实证研究者提供相关的理论支持和使用建议。     

Assessing Growth in a Diagnostic Classification Model Framework

A common assessment research design is the single-group pre-test/post-test design in which examinees are administered an assessment before instruction and then another assessment after instruction. In this type of study, the primary objective is to measure growth in examinees, individually and collectively. In an item response theory (IRT) framework, longitudinal IRT models can be used to assess growth in examinee ability over time. In a diagnostic classification model (DCM) framework, assessing growth translates to measuring changes in attribute mastery status over time, thereby providing a categorical, criterion-referenced interpretation of growth. This study introduces the Transition Diagnostic Classification Model (TDCM), which combines latent transition analysis with the log-linear cognitive diagnosis model to provide methodology for analyzing growth in a general DCM framework. Simulation study results indicate that the proposed model is flexible, provides accurate and reliable classifications, and is quite robust to violations to measurement invariance over time. The TDCM is used to analyze pre-test/post-test data from a diagnostic mathematics assessment.     

应用动态因果模型研究阅读神经网络背、腹侧通路的协作机制

在神经网络的最新取向下, 探讨阅读脑机制中背侧和腹侧通路的协作机制, 是解决语言认知神经科学多个理论问题共同面临的焦点。本项目拟通过两个脑功能成像实验, 建构汉字阅读的动态因果模型, 系统地考察汉字阅读的神经网络, 以及阅读网络中背、腹侧通路的协作机制。实验一利用快速适应实验范式的优点, 识别和考察汉字阅读涉及的认知成分所对应的功能脑区, 以及脑区联结形成的神经回路, 并建构汉字阅读的动态因果模型; 实验二进一步考察在刺激属性(语音和语义信息)和任务要求下阅读脑区的动态激活及相互作用。通过不同任务下的模型对比, 重点探讨阅读网络的脑区联结模式变化, 尤其是背、腹侧通路受刺激和任务影响时的协作机制。研究结果将为揭示阅读的神经生理模型、解决语言特异性脑区激活的争论等理论问题提供直接的证据, 还能为语言教学、阅读障碍矫治、以及临床应用提供理论基础与指导。     

Dilution in legal decision making: Effect of non-diagnostic information in relation to amount of diagnostic evidence

Judgments about others are often based on information that varies in terms of its diagnosticity or usefulness in predicting a certain outcome. Previous studies have demonstrated a “dilution effect” in which the addition of nondiagnostic or irrelevant information yields less extreme judgments than those based solely on diagnostic information. Two studies investigated the dilution effect in a juror decision making context in which no midpoint of a scale was provided by researchers. Study 1 examined the inclusion of positive, negative, or neutral character information in a criminal case and found that this nondiagnostic information affected attitude toward the defendant but did not “dilute” guilt judgments. The cases in Study 1 contained a larger amount of diagnostic information than studies that demonstrated the dilution effect. Thus, the amount of diagnostic evidence provided was varied in Study 2, and the results showed “diluted” judgments only when a small amount of diagnostic information was presented. Limitations to the dilution effect were discussed.     

多级评分认知诊断模型述评

认知诊断评估旨在探讨个体内部的知识掌握结构,并提供关于学生优缺点的详细诊断信息,以促进个体的全面发展。当前研究者已开发了大量0-1评分的认知诊断模型,但对于多级评分认知诊断模型的研究还比较少。本文对已有的多级评分认知诊断模型进行了归纳,介绍了模型的假设,计量特征以及适用范围,为实际应用者和研究者在多级评分认知诊断模型的比较和选用上提供借鉴和参考。最后,对未来关于多级评分诊断模型的研究方向进行了展望。     

Dilution in legal decision making: Effect of non-diagnostic information in relation to amount of diagnostic evidence

Judgments about others are often based on information that varies in terms of its diagnosticity or usefulness in predicting a certain outcome. Previous studies have demonstrated a “dilution effect” in which the addition of nondiagnostic or irrelevant information yields less extreme judgments than those based solely on diagnostic information. Two studies investigated the dilution effect in a juror decision making context in which no midpoint of a scale was provided by researchers. Study 1 examined the inclusion of positive, negative, or neutral character information in a criminal case and found that this nondiagnostic information affected attitude toward the defendant but did not “dilute” guilt judgments. The cases in Study 1 contained a larger amount of diagnostic information than studies that demonstrated the dilution effect. Thus, the amount of diagnostic evidence provided was varied in Study 2, and the results showed “diluted” judgments only when a small amount of diagnostic information was presented. Limitations to the dilution effect were discussed.     

Berufliche Leistungsfähigkeit

In different fields of law the assessment of capability of performance is important and often poses a particular challenge for medical or psychological experts. The present article introduces a model for the assessment of capability of performance developed on the background of a multicenter and interdisciplinary research project. This approach of assessment assumes that the performance capability of a person with psychiatric or psychosomatic disorders does not arise from symptomatology alone; instead, it is also a consequence of interactions between coping processes of the individual and environmental factors. The presented model of the assessment of performance capability is mainly oriented to the “International Classification of Functioning” (ICF; WHO) whose central diagnostic categories are, in addition to the disorder and the somatic and psychiatric functions, the levels of activity and participation. The different assessing levels are operationalized in the assessment model. In this article the various diagnostic levels and their interactions and the criteria which have been formulated are described. The process of decision-making is explored as well. This assessment model aims to make a more reliable, valid and transparent process of expert decision-making in order to increase the objectivity and justice of the assessment procedure.