Measurement of Ability in Adaptive Learning and Assessment Systems when Learners Use On-Demand Hints

Adaptive learning and assessment systems support learners in acquiring knowledge and skills in a particular domain. The learners’ progress is monitored through them solving items matching their level and aiming at specific learning goals. Scaffolding and providing learners with hints are powerful tools in helping the learning process. One way of introducing hints is to make hint use the choice of the student. When the learner is certain of their response, they answer without hints, but if the learner is not certain or does not know how to approach the item they can request a hint. We develop measurement models for applications where such on-demand hints are available. Such models take into account that hint use may be informative of ability, but at the same time may be influenced by other individual characteristics. Two modeling strategies are considered: (1) The measurement model is based on a scoring rule for ability which includes both response accuracy and hint use. (2) The choice to use hints and response accuracy conditional on this choice are modeled jointly using Item Response Tree models. The properties of different models and their implications are discussed. An application to data from Duolingo, an adaptive language learning system, is presented. Here, the best model is the scoring-rule-based model with full credit for correct responses without hints, partial credit for correct responses with hints, and no credit for all incorrect responses. The second dimension in the model accounts for the individual differences in the tendency to use hints.     

Behavioral Health Consultation and Primary Care: Lessons Learned

This article provides an overview of 20 years of professional experiences with developing and implementing a model for integrating behavioral health services into primary care. The Primary Care Behavioral Health (PCBH) model is designed to provide immediate access to behavioral care for a large number of primary care patients by positioning a behavioral health consultant in the exam room area to function as a core member of the primary care team. In an initial era of discovery, the authors were directly involved in developing and testing a variety of new approaches to providing behavioral health services in general medicine. In a second era focused on feasibility, the authors worked with Kaiser Permanente, the United States Air Force and Navy, the Veteran’s Administration, and the Bureau of Primary Care to system test this innovative model of integrated care. Now in an era devoted to dissemination, the authors review the various roles formal research, system level quality improvement initiatives and stakeholder analysis play in promoting integrated care. The authors also describe current efforts to (1) create a tool that helps systems develop integration targets and (2) use the PCBH model as a platform for teaching medical residents and behavioral health providers to work together in a redesigned primary care team model.     

THE EMERGENCE OF TRANSCENDENTAL NORMS IN HUMAN SYSTEMS

Terrence Deacon has described three orders of emergence; Arthur Peacocke and others have suggested four levels of human systems and sciences; and Philip Clayton has postulated an additional, transcendent, level. Orders and levels describe distinct aspects of emergence, with orders characterizing topological complexity and levels characterizing theoretical knowledge and causal power. By using Deacon's orders to analyze and relate each of the four "lower" levels one can project that analysis on the transcendent level to gain insight into the teleodynamic emergence of transcendent-level systems. I argue that cross-cultural interactions among human cultural-level systems results in the emergence of the "universal" transcendental norms historically characterized as the Greek Good, Beauty, and Truth. These norms require a dynamic existence that I characterize as the emergence of Spirit, using Josiah Royce's community of interpretation, and that I suggest provides a pragmatic clarification of Clayton's transcendent level. An understanding of those emergent norms clarifies ethical systems, highlights the importance of aesthetics in understanding scientific systems, and suggests the necessity of community in fruitful science-and-religion dialogue on human systems.     

Post‐traumatic functional recovery and reorganization in animal models: A theoretical and methodological challenge

Studies addressing cerebral functional localization face methodological and theoretical problems. Lesion experiments expect that when a functionally specialized structure is missing, its function can be deduced from the resulting impairments. Mostly, however, initial impairments are partially or fully eliminated through functional recovery. Apparently, such a recovery contradicts the notion of functional localization. In order to understand the mechanisms of recovery, improved methodology and a new theoretical framework are required. Insights into the mechanisms of recovery can be achieved by using “challenge” techniques, where functionally recovered individuals are exposed to organic and behavioral challenges, e.g. pharmacological manipulations or additional lesions, as well as modified test situations. Using such methods, a number of principles of functional recovery have emerged. We evaluate some of the available theories of post‐traumatic recovery against these principles and find that none of them can account for the principles. Finally, we present a new conceptual framework – the Reorganization of Elementary Functions (REF) model. This model reconceptualizes the term “function”, suggests mechanisms of post‐traumatic reorganizations, and resolves the contradiction between localization and functional recovery.     

Analysis of multinomial models under inequality constraints: Applications to measurement theory

Multinomial random variables are used across many disciplines to model categorical outcomes. Under this framework, investigators often use a likelihood ratio test to determine goodness-of-fit. If the permissible parameter space of such models is defined by inequality constraints, then the maximum likelihood estimator may lie on the boundary of the parameter space. Under this condition, the asymptotic distribution of the likelihood ratio test is no longer a simple χ² distribution. This article summarizes recent developments in the constrained inference literature as they pertain to the testing of multinomial random variables, and extends existing results by considering the case of jointly independent mutinomial random variables of varying categorical size. This article provides an application of this methodology to axiomatic measurement theory as a means of evaluating properly operationalized measurement axioms. This article generalizes Iverson and Falmagne’s [Iverson, G. J. & Falmagne, J. C. (1985). Statistical issues in measurement. Mathematical Social Sciences, 10, 131-153] seminal work on the empirical evaluation of measurement axioms and provides a classical counterpart to Myung, Karabatsos, and Iverson’s [Myung, J. I., Karabatsos, G. & Iverson, G. J. (2005). A Bayesian approach to testing decision making axioms. Journal of Mathematical Psychology, 49, 205-225] Bayesian methodology on the same topic.     

Self-Interest, Symbolic Attitudes, and Support for Public Policy: A Multilevel Analysis

This paper examines the role of self-interest and symbolic attitudes as predictors of support for two domestic policy issues—guaranteed jobs and incomes and national health insurance—in the American National Election Survey (ANES) between 1972 and 2004. As was the case in 1976 when Sears, Lau, Tyler, and Allen (1980 ) first explored this topic, symbolic attitudes continue to be much more important predictors of policy attitudes than various indicators of self-interest over the 30 years we analyze. We explore this finding further to determine whether any individual/internal and external/contextual variables affect the magnitude of self-interest effects on policy support. Five possible internal moderators of self-interest effects are examined: (1) political knowledge, (2) issue publics, (3) political values, (4) social identifications, and (5) emotions, but none are found to boost the magnitude of the self-interest effect. However, we do find some evidence that contextual variables representing the social/information environment moderate the impact of self-interest on public opinion.     

The evolution of frequency distributions: Relating regularization to inductive biases through iterated learning

The regularization of linguistic structures by learners has played a key role in arguments for strong innate constraints on language acquisition, and has important implications for language evolution. However, relating the inductive biases of learners to regularization behavior in laboratory tasks can be challenging without a formal model. In this paper we explore how regular linguistic structures can emerge from language evolution by iterated learning, in which one person’s linguistic output is used to generate the linguistic input provided to the next person. We use a model of iterated learning with Bayesian agents to show that this process can result in regularization when learners have the appropriate inductive biases. We then present three experiments demonstrating that simulating the process of language evolution in the laboratory can reveal biases towards regularization that might not otherwise be obvious, allowing weak biases to have strong effects. The results of these experiments suggest that people tend to regularize inconsistent word-meaning mappings, and that even a weak bias towards regularization can allow regular languages to be produced via language evolution by iterated learning.     

Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables

This paper uses log-linear models with latent variables (Hagenaars, in Loglinear Models with Latent Variables, 1993) to define a family of cognitive diagnosis models. In doing so, the relationship between many common models is explicitly defined and discussed. In addition, because the log-linear model with latent variables is a general model for cognitive diagnosis, new alternatives to modeling the functional relationship between attribute mastery and the probability of a correct response are discussed.     

Software Tools in Logic Education: Some Examples

Computers are increasingly present in education and make manyresources and activities available to teachers and pupils. Newpedagogical resources development is very interesting for both.Our digital library Summa Logicae is overtly involved in innovationand pedagogical systematization. It includes some software toolsfor teaching logic developed by computer science students, andin this article we present two of these tools. The MAFIA toolis especially attractive for first year students and helps themto understand the basic concepts of logic in an interactiveway using sematic tableaux. It also allows them to solve thecrazy cases in Mafia which their fellow students from previousyears proposed. The Modelos de Kripke tool, oriented to a moreadvanced level, serves for understanding the link between theproperties of the accessibility relation and the modal formulas,which is at the basis of the current developments of modal logic.     

Psychology's limits as a scientific discipline: A personal view

I provided a more personal view of Wachtel's (1980) article. I began by discussing the extent to which my own research program complied with his distinctive recommendations. After offering a different take on the impact of high productivity, I focused on (a) the negative effects of the quest for extramural funding and (b) the positive effects of a better balance between theoretical and empirical contributions. I then turn to some of my own theoretical and empirical studies of the place that theory has in successful science. This research suggests that theory only has a beneficial effect when it is integrative in function and when it is closely constrained by available data. I end with a speculation regarding the value of having theories that are maximally formal, even mathematical.