On the empirical indistinguishability of knowledge structures

In recent years a number of articles have focused on the identifiability of the basic local independence model. The identifiability issue usually concerns two model parameter sets predicting an identical probability distribution on the response patterns. Both parameter sets are applied to the same knowledge structure. However, nothing is known about cases where different knowledge structures predict the same probability distribution. This situation is referred to as ʻempirical indistinguishabilityʼ between two structures and is the main subject of the present paper. Empirical indistinguishability is a stronger form of unidentifiability, which involves not only the parameters, but also the structural and combinatorial properties of the model. In particular, as far as knowledge structures are concerned, a consequence of empirical indistinguishability is that the existence of certain knowledge states cannot be empirically established. Most importantly, it is shown that model identifiability cannot guarantee that a certain knowledge structure is empirically distinguishable from others. The theoretical findings are exemplified in a number of different empirical scenarios.     

Testing the actual equivalence of automatically generated items

If the automatic item generation is used for generating test items, the question of how the equivalence among different instances may be tested is fundamental to assure an accurate assessment. In the present research, the question was dealt by using the knowledge space theory framework. Two different ways of considering the equivalence among instances are proposed: The former is at a deterministic level and it requires that all the instances of an item template must belong to exactly the same knowledge states; the latter adds a probabilistic level to the deterministic one. The former type of equivalence can be modeled by using the BLIM with a knowledge structure assuming equally informative instances; the latter can be modeled by a constrained BLIM. This model assumes equality constraints among the error parameters of the equivalent instances. An approach is proposed for testing the equivalence among instances, which is based on a series of model comparisons. A simulation study and an empirical application show the viability of the approach.     

Assessing Parameter Invariance in the BLIM: Bipartition Models

In knowledge space theory, the knowledge state of a student is the set of all problems he is capable of solving in a specific knowledge domain and a knowledge structure is the collection of knowledge states. The basic local independence model (BLIM) is a probabilistic model for knowledge structures. The BLIM assumes a probability distribution on the knowledge states and a lucky guess and a careless error probability for each problem. A key assumption of the BLIM is that the lucky guess and careless error probabilities do not depend on knowledge states (invariance assumption). This article proposes a method for testing the violations of this specific assumption. The proposed method was assessed in a simulation study and in an empirical application. The results show that (1) the invariance assumption might be violated by the empirical data even when the model’s fit is very good, and (2) the proposed method may prove to be a promising tool to detect invariance violations of the BLIM.     

Recovering a Probabilistic Knowledge Structure by Constraining its Parameter Space

In the Basic Local Independence Model (BLIM) of Doignon and Falmagne (Knowledge Spaces, Springer, Berlin, 1999), the probabilistic relationship between the latent knowledge states and the observable response patterns is established by the introduction of a pair of parameters for each of the problems: a lucky guess probability and a careless error probability. In estimating the parameters of the BLIM with an empirical data set, it is desirable that such probabilities remain reasonably small. A special case of the BLIM is proposed where the parameter space of such probabilities is constrained. A simulation study shows that the constrained BLIM is more effective than the unconstrained one, in recovering a probabilistic knowledge structure.     

Distributed skill functions and the meshing of knowledge structures

The scope of knowledge space theory was extended by bringing into the picture the underlying skills and capabilities that are relevant to solving the problems in a knowledge domain. A major challenge to this approach comes from the need to aggregate distributed information on (partially) overlapping domains and skill sets. The notion of a distributed skill function is introduced for formalizing the integration of several skill functions that represent the assignment of skills to problems. It is shown that their consistency is captured by the meshability of the delineated knowledge structures. This result draws upon a characterization of the meshing of finite or infinite collections of knowledge structures, which extends and generalizes previous results on the binary case. The discussion covers implications for knowledge assessment and for practical applications, such as integrating skill assignments coming from different experts or distributed resources in technology-enhanced learning.     

Knowledge space theory, formal concept analysis, and computerized psychological assessment

In the present study, the use of knowledge space theory (KST), jointly with formal concept analysis (FCA), is proposed for developing a formal representation of the relations between the items of a questionnaire and a set of psychodiagnostic criteria. This formal representation can be used to develop an efficient adaptive tool for psychological assessment. Rusch and Wille (1996) have shown some interesting connections between KST and FCA; these connections are applied in the construction of knowledge structures, starting from a formal context representing the relations between items and criteria. The proposed general methodology was applied, as an example, to the Maudsley Obsessional-Compulsive Questionnaire. We used a data set provided by a sample of patients with a diagnosis of obsessive-compulsive disorder to validate the obtained structures. The parameters of the basic local independence model (BLIM) were estimated for the obtained knowledge structures. The fit of each model was tested by parametric bootstrap because of the sparseness of the derived data matrix. The results are discussed in light of both psychological and methodological relapses. In particular, we propose a reinterpretation of the BLIM parameters that seems suitable for testing reliability and construct validity; furthermore, it is pointed out how the obtained structures could represent the starting point for the development of a computerized assessment tool.     

The assessment of knowledge and learning in competence spaces: The gain–loss model for dependent skills

The gain–loss model (GaLoM) is a formal model for assessing knowledge and learning. In its original formulation, the GaLoM assumes independence among the skills. Such an assumption is not reasonable in several domains, in which some preliminary knowledge is the foundation for other knowledge. This paper presents an extension of the GaLoM to the case in which the skills are not independent, and the dependence relation among them is described by a well‐graded competence space. The probability of mastering skill s at the pretest is conditional on the presence of all skills on which s depends. The probabilities of gaining or losing skill s when moving from pretest to posttest are conditional on the mastery of s at the pretest, and on the presence at the posttest of all skills on which s depends. Two formulations of the model are presented, in which the learning path is allowed to change from pretest to posttest or not. A simulation study shows that models based on the true competence space obtain a better fit than models based on false competence spaces, and are also characterized by a higher assessment accuracy. An empirical application shows that models based on pedagogically sound assumptions about the dependencies among the skills obtain a better fit than models assuming independence among the skills.     

Nonparametric item response theory axioms and properties under nonlinearity and their exemplification with knowledge space theory

This paper investigates the dichotomous Mokken nonparametric item response theory (IRT) axioms and properties under incomparabilities among latent trait values and items. Generalized equivalents of the unidimensional nonparametric IRT axioms and properties are formulated for nonlinear (quasi-ordered) person and indicator spaces. It is shown that monotone likelihood ratio (MLR) for the total score variable and nonlinear latent trait implies stochastic ordering (SO) of the total score variable, but may fail to imply SO of the nonlinear latent trait. The reason for this and conditions under which the implication holds are specified, based on a new, simpler proof of the fact that in the unidimensional case MLR implies SO. The approach is applied in knowledge space theory (KST), a combinatorial test theory. This leads to a (tentative) Mokken-type nonparametric axiomatization in the currently parametric theory of knowledge spaces. The nonparametric axiomatization is compared with the assumptions of the parametric basic local independence model which is fundamental in KST. It is concluded that this paper may provide a first step toward a basis for a possible fusion of the two split directions of psychological test theories IRT and KST.     

基于多层面模型的教学效能感量表

运用层面理论方法,在参照已有研究的基础上,对10名中学教师进行了开放式访谈,结合已有研究和访谈结果,设计了映射语句,根据层面元素组合,设计项目,编制了教学效能感量表。对200名中学教师进行初测收回有效问卷162份,运用SSA分析技术,对项目质量做了分析,对部分项目进行了删减,并修改了映射语句,再对530名中学教师进行正式施测,收回有效问卷505份,运用SSA和验证性因素分析技术对量表的构想效度进行了检验,发现量表的信、效度较高,量表符合心理测量学的要求,并证明教学效能感结构为二层面雷达图结构。     

A Proposal for the Demarcation of Theory and Knowledge

Research communication in interdisciplinary research projects requires a way of demarcation of theory and knowledge that is easy to communicate, is inconsequential for the framework of concepts, results, and procedures within existing scientific disciplines, and abstains from trying to resolve the dispute between (neo)positivists and constructivists. A simple way of demarcation starts from the notion of language-independent and language-dependent reality. Currently, what passes for knowledge (“news”) and myth (“fake news”) depends, besides on sheer volume and frequency of the messages, increasingly on the internal consistency of (computer) language-dependent reality and decreasingly on language-independent reality. All language is instruction, and knowledge is to know which instructions (that is, theory) are predictive of a result, state, or situation in language-independent reality. Any theory that doesn’t reduce outcome space, or contains one or more empirically/physically impossible instructions, or produces wrong predictions, or falls short of demonstration is not knowledge.     

Indistinguishability tests in the actor–partner interdependence model

When considering dyadic data, one of the questions is whether the roles of the two dyad members can be considered equal. This question may be answered empirically using indistinguishability tests in the actor–partner interdependence model. In this paper several issues related to such indistinguishability tests are discussed: the difference between maximum likelihood and restricted maximum likelihood based tests for equality in variance parameters; the choice between the structural equation modelling and multilevel modelling framework; and the use of sequential testing rather than one global test for a set of indistinguishability tests. Based on simulation studies, we provide guidelines for best practice. All different types of tests are illustrated with cross-sectional and longitudinal data, and corroborated with corresponding R code.     

A Method for Comparing Knowledge Structures Concerning Their Adequacy

A knowledge domain is represented in knowledge space theory by a finite set Q of problems. The knowledge of a subject in the domain is described by the subset of problems from Q that the subject is able to solve. The set of all possible knowledge states of subjects is called the knowledge structure on Q. A crucial problem is how a hypothetical knowledge structure, obtained for example by querying an expert, can be evaluated empirically. We describe a method for comparing two knowledge structures concerning their ability to describe a set of empirically observed response patterns. Our comparison method is tested in two simulation studies. The results of these studies show that the method is highly accurate in its evaluation of knowledge structures. Copyright 2001 Academic Press.     

A Review of the Effects of Interspersing Procedures on the Stages of Academic Skill Development

Basic and applied research related to the effects of interspersing trials of maintenance (i.e., review) tasks among trials of acquisition tasks on academic skill development is reviewed. In general, empirical research suggests that interspersing procedures are effective in facilitating acquisition, learning rate, and maintenance. However, some inconsistency exists among the data that suggests student learning, in some instances, may be impeded. Research also suggests that interspersing procedures are not conducive to facilitating generalization. The discrepancy between various research findings and the consistent failure of research on interspersing procedures to increase generalization is critically reviewed in relation to a hierarchical learning theory set forth by Haring and Eaton (1978). From this hierarchical learning theory perspective, inconsistent results may be better understood and predicted. Implications for current practice and directions for future research are also discussed.     

Earman on Underdetermination and Empirical Indistinguishability

Earman (1993) distinguishes three notions of empirical indistinguishability and offers a rigorous framework to investigate how each of these notions relates to the problem of underdetermination of theory choice. He uses some of the results obtained in this framework to argue for a version of scientific anti- realism. In the present paper we first criticize Earman's arguments for that position. Secondly, we propose and motivate a modification of Earman's framework and establish several results concerning some of the notions of indistinguishability in this modified framework. Finally, we interpret these results in the light of the realism/anti- realism debate. This revised version was published online in July 2006 with corrections to the Cover Date.     

The Role of Theory and Research in the Practice of Cognitive-Behavioral Couple Therapy: If You Build It,They Will Come

Cognitive-behavioral couple therapy (CBCT), while empirically validated and highly efficacious, does not always have positive results for couples. One factor that may limit the efficacy of this intervention is the way in which therapists are trained to provide this type of therapy. More specifically, there is a need for therapists to gain a solid foundation in CBCT theory in order to maximize treatment results. This paper presents an argument for why an understanding of theory is necessary for therapists in treating couples. Then, this paper presents our training model and how we integrate theory into our training of both graduate student therapists and more experienced clinicians. We take the stance that “if you build it, they will come.”     

The nature of importance perceptions: a test of a cognitive model

This study empirically examines an alternative conceptual approach to the nature of importance perceptions. Current interpretations of importance evaluations--demand, need, and value-based approaches--have been primarily connotative and continue to remain bereft of convincing logical or empirical support. Borrowing from social-psychological theories of cognitive structure, the study tests the usefulness of a conceptualization of importance perceptions which presents the construct as a function of cognitive centrality, dependence, criticalness, and temporary salience of factors and outcomes in the individual cognitive space. The model is examined over a wide range of factors with findings indicating that significant proportions of variance in importance rating/rankings may be thus explained. Implications for management theory and practices are discussed.     

An iterative procedure for extracting skill maps from data

The methodologies for the construction of a knowledge structure mainly refer to the query to experts, the skill maps, and the data-driven approaches. This last method is of growing interest in recent literature. In this paper, an iterative procedure for building a skill map from a set of data is introduced. This procedure is based on the minimization of the distance between the knowledge structure delineated by a given skill map and the data. The accuracy of the proposed method is tested through a number of simulation studies where the amount of noise in the data is manipulated as well as the kind of structure to be reconstructed. Results show that the procedure is accurate and that its performance tends to be sufficiently stable even with high error rates. The procedure is compared to two already-existing methodologies to derive knowledge structures from a set of data. The use of the corrected Akaike Information Criterion (AICc) as a stopping criterion of the iterative reconstruction procedure is tested against the app criterion introduced by Schrepp. Moreover, two empirical applications on clinical data are reported, and their results show the applicability of the procedure.     

An empirically feasible approach to the epistemology of arithmetic

Recent years have seen an explosion of empirical data concerning arithmetical cognition. In this paper that data is taken to be philosophically important and an outline for an empirically feasible epistemological theory of arithmetic is presented. The epistemological theory is based on the empirically well-supported hypothesis that our arithmetical ability is built on a protoarithmetical ability to categorize observations in terms of quantities that we have already as infants and share with many nonhuman animals. It is argued here that arithmetical knowledge developed in such a way cannot be totally conceptual in the sense relevant to the philosophy of arithmetic, but neither can arithmetic understood to be empirical. Rather, we need to develop a contextual a priori notion of arithmetical knowledge that preserves the special mathematical characteristics without ignoring the roots of arithmetical cognition. Such a contextual a priori theory is shown not to require any ontologically problematic assumptions, in addition to fitting well within a standard framework of general epistemology.     

Logical and psychological partitioning of mind: Depicting the same map?

The aim of this paper is to demonstrate that empirically delimited structures of mind are also differentiable by means of systematic logical analysis. For the sake of this aim, the paper first summarizes Demetriou’s theory of cognitive organization and growth. This theory assumes that the mind is a multistructural entity that develops across three fronts: the processing system that constrains processing potentials, a set of specialized structural systems (SSSs) that guide processing within different reality and knowledge domains, and a hypercognitive system that monitors and controls the functioning of all other systems. The second part of the paper focuses on the SSSs, which are the target of our logical analysis, and it summarizes a series of empirical studies demonstrating their autonomous operation. The third part develops the logical proof showing that each SSS involves a kernel element that cannot be reduced to standard logic or to any other SSS. The implications of this analysis for the general theory of knowledge and cognitive development are discussed.     

The Contingent A Priori and Implicit Knowledge

By introducing a name 'one meter' and stipulating that it refers to the length of stick S, the stipulator appears to be in a position to gain immediate (and arguably a priori ) knowledge of a mind- and language-independent fact—the fact that the length of stick S is one meter. It appears that other users of the name can gain this knowledge only through empirical enquiry. I argue that this presents a paradox. After clarifying the nature of the paradox, I offer a solution by arguing that, contrary to appearances, other users of 'one meter' implicitly knew that the length of stick S is one meter before learning the name, as did the stipulator prior to introducing the name. There is some distinct knowledge that other users of the name can only gain empirically, but the stipulator cannot gain this knowledge without empirical enquiry either.