Inferring Learners' Knowledge From Their Actions

Watching another person take actions to complete a goal and making inferences about that person's knowledge is a relatively natural task for people. This ability can be especially important in educational settings, where the inferences can be used for assessment, diagnosing misconceptions, and providing informative feedback. In this paper, we develop a general framework for automatically making such inferences based on observed actions; this framework is particularly relevant for inferring student knowledge in educational games and other interactive virtual environments. Our approach relies on modeling action planning: We formalize the problem as a Markov decision process in which one must choose what actions to take to complete a goal, where choices will be dependent on one's beliefs about how actions affect the environment. We use a variation of inverse reinforcement learning to infer these beliefs. Through two lab experiments, we show that this model can recover people's beliefs in a simple environment, with accuracy comparable to that of human observers. We then demonstrate that the model can be used to provide real‐time feedback and to model data from an existing educational game.     

Identifying expectations about the strength of causal relationships

When we try to identify causal relationships, how strong do we expect that relationship to be? Bayesian models of causal induction rely on assumptions regarding people’s a priori beliefs about causal systems, with recent research focusing on people’s expectations about the strength of causes. These expectations are expressed in terms of prior probability distributions. While proposals about the form of such prior distributions have been made previously, many different distributions are possible, making it difficult to test such proposals exhaustively. In Experiment 1 we used iterated learning—a method in which participants make inferences about data generated based on their own responses in previous trials—to estimate participants’ prior beliefs about the strengths of causes. This method produced estimated prior distributions that were quite different from those previously proposed in the literature. Experiment 2 collected a large set of human judgments on the strength of causal relationships to be used as a benchmark for evaluating different models, using stimuli that cover a wider and more systematic set of contingencies than previous research. Using these judgments, we evaluated the predictions of various Bayesian models. The Bayesian model with priors estimated via iterated learning compared favorably against the others. Experiment 3 estimated participants’ prior beliefs concerning different causal systems, revealing key similarities in their expectations across diverse scenarios.     

Learning the form of causal relationships using hierarchical bayesian models

People learn quickly when reasoning about causal relationships, making inferences from limited data and avoiding spurious inferences. Efficient learning depends on abstract knowledge, which is often domain or context specific, and much of it must be learned. While such knowledge effects are well documented, little is known about exactly how we acquire knowledge that constrains learning. This work focuses on knowledge of the functional form of causal relationships; there are many kinds of relationships that can apply between causes and their effects, and knowledge of the form such a relationship takes is important in order to quickly identify the real causes of an observed effect. We developed a hierarchical Bayesian model of the acquisition of knowledge of the functional form of causal relationships and tested it in five experimental studies, considering disjunctive and conjunctive relationships, failure rates, and cross-domain effects. The Bayesian model accurately predicted human judgments and outperformed several alternative models.     

Action understanding as inverse planning

Humans are adept at inferring the mental states underlying other agents’ actions, such as goals, beliefs, desires, emotions and other thoughts. We propose a computational framework based on Bayesian inverse planning for modeling human action understanding. The framework represents an intuitive theory of intentional agents’ behavior based on the principle of rationality: the expectation that agents will plan approximately rationally to achieve their goals, given their beliefs about the world. The mental states that caused an agent’s behavior are inferred by inverting this model of rational planning using Bayesian inference, integrating the likelihood of the observed actions with the prior over mental states. This approach formalizes in precise probabilistic terms the essence of previous qualitative approaches to action understanding based on an “intentional stance” [Dennett, D. C. (1987). The intentional stance. Cambridge, MA: MIT Press] or a “teleological stance” [Gergely, G., Nádasdy, Z., Csibra, G., & Biró, S. (1995). Taking the intentional stance at 12 months of age. Cognition, 56, 165-193]. In three psychophysical experiments using animated stimuli of agents moving in simple mazes, we assess how well different inverse planning models based on different goal priors can predict human goal inferences. The results provide quantitative evidence for an approximately rational inference mechanism in human goal inference within our simplified stimulus paradigm, and for the flexible nature of goal representations that human observers can adopt. We discuss the implications of our experimental results for human action understanding in real-world contexts, and suggest how our framework might be extended to capture other kinds of mental state inferences, such as inferences about beliefs, or inferring whether an entity is an intentional agent.     

Changing minds: Children's inferences about third party belief revision

By the age of 5, children explicitly represent that agents can have both true and false beliefs based on epistemic access to information (e.g., Wellman, Cross, & Watson, 2001). Children also begin to understand that agents can view identical evidence and draw different inferences from it (e.g., Carpendale & Chandler, 1996). However, much less is known about when, and under what conditions, children expect other agents to change their minds. Here, inspired by formal ideal observer models of learning, we investigate children's expectations of the dynamics that underlie third parties’ belief revision. We introduce an agent who has prior beliefs about the location of a population of toys and then observes evidence that, from an ideal observer perspective, either does, or does not justify revising those beliefs. We show that children's inferences on behalf of third parties are consistent with the ideal observer perspective, but not with a number of alternative possibilities, including that children expect other agents to be influenced only by their prior beliefs, only by the sampling process, or only by the observed data. Rather, children integrate all three factors in determining how and when agents will update their beliefs from evidence.     

Living and Learning: Reproducing Beliefs in Selective Experience

People's decisions shape their experience. For example, a recruitment officer decides between job applicants and cannot evaluate the suitability of rejected applicants. The selection decisions thus affect the content of the officer's experience of suitable and unsuitable applicants, and experiential learning is achieved from a selective sample of experiences. It is suggested that people's beliefs are sensitive to the content of the experienced sample, but the mind cannot adjust for the selectivity of the sample even when it results from the individual's own decisions. Two experiments with a recruitment task showed that incorrect prior beliefs survive experiential learning when the beliefs are reproduced and thus appear to be confirmed, in actual experience. When the task was to achieve high performance, incorrect prior beliefs persisted because they were reproduced in a smaller sample of selected job applicants. In contrast, when the task was focused on learning, a greater number of applicants were selected, and a more representative experience therefore revised incorrect beliefs. The actual content of the experienced sample is thus crucial for the persistence, as well as for the revision, of incorrect beliefs. Further, as predicted by the hypothesis of constructivist coding, when feedback was absent for rejected applicants, participants constructed “internal feedback” in line with the expectation that the rejected applicant was unsuitable. Thus, when fewer applicants were hired, participants came to believe that the actual proportion of suitable applicants was low. Finally, the implications for efforts to reduce bias and improve experiential learning are discussed. Copyright © 2012 John Wiley & Sons, Ltd.     

Computational models for the combination of advice and individual learning

Decision making often takes place in social environments where other actors influence individuals' decisions. The present article examines how advice affects individual learning. Five social learning models combining advice and individual learning-four based on reinforcement learning and one on Bayesian learning-and one individual learning model are tested against each other. In two experiments, some participants received good or bad advice prior to a repeated multioption choice task. Receivers of advice adhered to the advice, so that good advice improved performance. The social learning models described the observed learning processes better than the individual learning model. Of the models tested, the best social learning model assumes that outcomes from recommended options are more positively evaluated than outcomes from nonrecommended options. This model correctly predicted that receivers first adhere to advice, then explore other options, and finally return to the recommended option. The model also predicted accurately that good advice has a stronger impact on learning than bad advice. One-time advice can have a long-lasting influence on learning by changing the subjective evaluation of outcomes of recommended options.     

High‐Stakes Decision‐Making Within Complex Social Environments: A Computational Model of Belief Systems in the Arab Spring

People experiencing similar conditions may make different decisions, and their belief systems provide insight about these differences. An example of high‐stakes decision‐making within a complex social context is the Arab Spring, in which large numbers of people decided to protest and even larger numbers decided to stay at home. This study uses qualitative analyses of interview narratives and social media addressing individual decisions to develop a computational model tracing the cognitive decision‐making process. The model builds on work by Abelson and Carroll (1965), Schank and Abelson (1977), and Axelrod (1976) to systematically trace the inferences connecting beliefs to decisions. The findings show that protest decisions were often based on positive emotions such as pride, hope, courage, and solidarity, triggered by beliefs about successful protest and self‐sacrifice. By contrast, decisions to stay at home were triggered by beliefs about safety, state approval, and living conditions. As one participant said, “When I heard about the revolution in Tunisia, my heart was filled with solidarity for the people.” In the words of a non‐participant: “When people are killed, we must be careful. There are more important things than protest: safety and stability.” This study of individual explanations about events identifies the role of emotions in high‐stakes decision‐making within complex social environments.     

Curiosity-driven recommendation strategy for adaptive learning via deep reinforcement learning

The design of recommendation strategies in the adaptive learning systems focuses on utilizing currently available information to provide learners with individual-specific learning instructions. As a critical motivate for human behaviours, curiosity is essentially the drive to explore knowledge and seek information. In a psychologically inspired view, we propose a curiosity-driven recommendation policy within the reinforcement learning framework, allowing for an efficient and enjoyable personalized learning path. Specifically, a curiosity reward from a well-designed predictive model is generated to model one's familiarity with the knowledge space. Given such curiosity rewards, we apply the actor–critic method to approximate the policy directly through neural networks. Numerical analyses with a large continuous knowledge state space and concrete learning scenarios are provided to further demonstrate the efficiency of the proposed method.     

Sticking to the Evidence? A Behavioral and Computational Case Study of Micro‐Theory Change in the Domain of Magnetism

Constructing an intuitive theory from data confronts learners with a “chicken‐and‐egg” problem: The laws can only be expressed in terms of the theory's core concepts, but these concepts are only meaningful in terms of the role they play in the theory's laws; how can a learner discover appropriate concepts and laws simultaneously, knowing neither to begin with? We explore how children can solve this chicken‐and‐egg problem in the domain of magnetism, drawing on perspectives from computational modeling and behavioral experiments. We present 4‐ and 5‐year‐olds with two different simplified magnet‐learning tasks. Children appropriately constrain their beliefs to two hypotheses following ambiguous but informative evidence. Following a critical intervention, they learn the correct theory. In the second study, children infer the correct number of categories given no information about the possible causal laws. Children's hypotheses in these tasks are explained as rational inferences within a Bayesian computational framework.     

ELEMENTARY TEACHERS' BELIEFS ABOUT READING AND KNOWLEDGE OF READING CONTENT: RELATIONSHIPS TO DECISION ABOUT READING OUTCOMES

Relationships between elementary teachers' beliefs about reading, knowledge of basic reading content, and decisions about the importance of students' reading behavior in relation to grade level were examined. Significant correlation coefficients were found for (1) knowledge of reading and reading behavior outcomes focusing on discrimination of sounds represented by consonants and consonant clusters and (2) student‐centered reading beliefs and reading behavior outcomes associated with making conclusions and drawing inferences about stories read. Significant negative correlations were noted for (1) knowledge of reading and outcomes dealing with comprehension of explicitly stated meaning and details in reading passages and (2) student‐centered reading beliefs and all reading outcomes focusing on basic decoding skills and literal comprehension of story information. The best predictor of teachers' identification of important reading behavior outcomes was knowledge of reading content. Reading behavior outcomes differed significantly between primary and intermediate level teachers for recognition of sounds represented by vowels; intermediate teachers gave these higher rankings than did primary teachers. The results indicate that beliefs about reading influence elementary teachers' decisions about the importance of reading outcomes typically taught in the elementary grades. Teachers who hold student‐centered reading beliefs are not likely to value instruction that focuses on decoding.     

A Bayesian Model of Biases in Artificial Language Learning: The Case of a Word‐Order Universal

In this article, we develop a hierarchical Bayesian model of learning in a general type of artificial language‐learning experiment in which learners are exposed to a mixture of grammars representing the variation present in real learners’ input, particularly at times of language change. The modeling goal is to formalize and quantify hypothesized learning biases. The test case is an experiment ( Culbertson, Smolensky, & Legendre, 2012 ) targeting the learning of word‐order patterns in the nominal domain. The model identifies internal biases of the experimental participants, providing evidence that learners impose (possibly arbitrary) properties on the grammars they learn, potentially resulting in the cross‐linguistic regularities known as typological universals. Learners exposed to mixtures of artificial grammars tended to shift those mixtures in certain ways rather than others; the model reveals how learners’ inferences are systematically affected by specific prior biases. These biases are in line with a typological generalization—Greenberg's Universal 18—which bans a particular word‐order pattern relating nouns, adjectives, and numerals.     

Inferences about the authentic self: when do actions say more than mental states?

Three studies involving 478 undergraduates examined the perceived importance of observable actions versus mental states in revealing the "true self"-the authentic and fundamental nature of a target person. Results suggest that when people have only limited information about a target, they believe that an action is more diagnostic of the individual's true self than the accompanying mental state. When participants have knowledge concerning chronic dispositional tendencies of the target, however, they judge that a chronic mental state is more diagnostic of the true self than a chronic action tendency. Considered together, the findings suggest that people conceptualize the true self as a relatively private entity but nevertheless believe that an action of a little-known person may be particularly informative about that individual. Perceived diagnosticity of the true self was partially mediated by inferences concerning the relative stability of actions versus states but not by inferences of volition.     

The Wisdom of Individuals: Exploring People's Knowledge About Everyday Events Using Iterated Learning

Determining the knowledge that guides human judgments is fundamental to understanding how people reason, make decisions, and form predictions. We use an experimental procedure called 'iterated learning,' in which the responses that people give on one trial are used to generate the data they see on the next, to pinpoint the knowledge that informs people's predictions about everyday events (e.g., predicting the total box office gross of a movie from its current take). In particular, we use this method to discriminate between two models of human judgments: a simple Bayesian model ( Griffiths & Tenenbaum, 2006 ) and a recently proposed alternative model that assumes people store only a few instances of each type of event in memory (Min K ; Mozer, Pashler, & Homaei, 2008 ). Although testing these models using standard experimental procedures is difficult due to differences in the number of free parameters and the need to make assumptions about the knowledge of individual learners, we show that the two models make very different predictions about the outcome of iterated learning. The results of an experiment using this methodology provide a rich picture of how much people know about the distributions of everyday quantities, and they are inconsistent with the predictions of the Min K model. The results suggest that accurate predictions about everyday events reflect relatively sophisticated knowledge on the part of individuals.     

Non‐Bayesian Noun Generalization in 3‐ to 5‐Year‐Old Children: Probing the Role of Prior Knowledge in the Suspicious Coincidence Effect

It is unclear how children learn labels for multiple overlapping categories such as “Labrador,” “dog,” and “animal.” Xu and Tenenbaum (2007a) suggested that learners infer correct meanings with the help of Bayesian inference. They instantiated these claims in a Bayesian model, which they tested with preschoolers and adults. Here, we report data testing a developmental prediction of the Bayesian model—that more knowledge should lead to narrower category inferences when presented with multiple subordinate exemplars. Two experiments did not support this prediction. Children with more category knowledge showed broader generalization when presented with multiple subordinate exemplars, compared to less knowledgeable children and adults. This implies a U‐shaped developmental trend. The Bayesian model was not able to account for these data, even with inputs that reflected the similarity judgments of children. We discuss implications for the Bayesian model, including a combined Bayesian/morphological knowledge account that could explain the demonstrated U‐shaped trend.     

Relations among competence beliefs,utility value,achievement goals,and effort in mathematics

Background. Research has shown that motivation is a key factor in the learning process as well as in school achievement. In essence, a number of researchers have highlighted the close link between motivation and achievement‐related behaviours such as effort. Aims. The present study aims to acquire more specific information concerning the relations between competence beliefs, utility value and achievement goals in mathematics among secondary school students, to further document the influence of social agents, and to better understand the relationships between these variables, as well as to effort. Sample. Participants were 759 Grade 7 to Grade 11 students (389 males, 370 females). Method. Structural equation modelling techniques were used to test a model of achievement‐related behaviours (effort) in mathematics based on support from social agents, competence beliefs, utility value and achievement goals. Several self‐reported scales were administered. Results. Results indicate that effort in mathematics is mainly explained by mastery goals and competence beliefs. As for the role of social agents, results demonstrated that the perception of parental support chiefly explained variables associated with the valuing of mathematics while teachers' support acted most on competence beliefs. Conclusions. Two main conclusions stem from our results. First, mastery goals have an important and significant impact on students' effort in the learning of mathematics. Second, the nature and the strength of the relationships between competence beliefs, utility value, achievement goals and effort are not significantly influenced by age and gender, at least in mathematics.     

Can We Be Self‐Deceived about What We Believe? Self‐Knowledge,Self‐Deception,and Rational Agency

Abstract: This paper considers the question of whether it is possible to be mistaken about the content of our first‐order intentional states. For proponents of the rational agency model of self‐knowledge, such failures might seem very difficult to explain. On this model, the authority of self‐knowledge is not based on inference from evidence, but rather originates in our capacity, as rational agents, to shape our beliefs and other intentional states. To believe that one believes that p, on this view, constitutes one's belief that p and so self‐knowledge involves a constitutive relation between first‐ and second‐order beliefs. If this is true, it is hard to see how those second‐order beliefs could ever be false. I develop two counter‐examples which show that despite the constitutive relation between first‐ and second‐order beliefs in standard cases of self‐knowledge, it is possible to be mistaken, and even self‐deceived, about the content of one's own beliefs. These counter‐examples do not show that the rational agency model is mistaken—rather, they show that the possibility of estrangement from one's own mental life means that, even within the rational agency model, it is possible to have false second‐order beliefs about the content of one's first‐order beliefs. The authority of self‐knowledge does not entail that to believe that one believes that p suffices to make it the case that one believes that p.     

人际互动中社会学习的计算神经机制

人类在社会互动中通过他人的行为对他人特质、意图及特定情境下的社会规范进行学习, 是优化决策、维护积极社会互动的重要条件。近年来, 越来越多的研究通过结合计算模型与神经影像技术对社会学习的认知计算机制及其神经基础进行了深入考察。已有研究发现, 人类的社会学习过程能够较好地被强化学习模型与贝叶斯模型刻画, 主要涉及的认知计算过程包括主观期望、预期误差和不确定性的表征以及信息整合的过程。大脑对这些计算过程的执行主要涉及奖惩加工相关脑区(如腹侧纹状体与腹内侧前额叶)、社会认知加工相关脑区(如背内侧前额叶和颞顶联合区)及认知控制相关脑区(如背外侧前额叶)。需要指出的是, 计算过程与大脑区域之间并不是一一映射的关系, 提示未来研究可借助多变量分析与脑网络分析等技术从系统神经科学的角度来考察大尺度脑网络如何执行不同计算过程。此外, 将来研究应注重生态效度, 利用超扫描技术考察真实互动下的社会学习过程, 并更多地关注内隐社会学习的计算与神经机制。     

Naïve and Robust: Class‐Conditional Independence in Human Classification Learning

Humans excel in categorization. Yet from a computational standpoint, learning a novel probabilistic classification task involves severe computational challenges. The present paper investigates one way to address these challenges: assuming class‐conditional independence of features. This feature independence assumption simplifies the inference problem, allows for informed inferences about novel feature combinations, and performs robustly across different statistical environments. We designed a new Bayesian classification learning model (the dependence‐independence structure and category learning model, DISC‐LM) that incorporates varying degrees of prior belief in class‐conditional independence, learns whether or not independence holds, and adapts its behavior accordingly. Theoretical results from two simulation studies demonstrate that classification behavior can appear to start simple, yet adapt effectively to unexpected task structures. Two experiments—designed using optimal experimental design principles—were conducted with human learners. Classification decisions of the majority of participants were best accounted for by a version of the model with very high initial prior belief in class‐conditional independence, before adapting to the true environmental structure. Class‐conditional independence may be a strong and useful default assumption in category learning tasks.     

Risk aversion and elite-group ignorance

Critical race theorists and standpoint epistemologists argue that agents who are members of dominant social groups are often in a state of ignorance about the extent of their social dominance, where this ignorance is explained by these agents' membership in a socially dominant group (e.g., Mills 2007). To illustrate this claim bluntly, it is argued: 1) that many white men do not know the extent of their social dominance, 2) that they remain ignorant as to the extent of their dominant social position even where this information is freely attainable, and 3) that this ignorance is due in part to the fact that they are white men. We argue that on Buchak's (2010, 2013) model of risk averse instrumental rationality, ignorance of one's privileges can be rational. This argument yields a new account of elite-group ignorance, why it may occur, and how it might be alleviated.