Inferring Learners' Knowledge From Their Actions

Watching another person take actions to complete a goal and making inferences about that person's knowledge is a relatively natural task for people. This ability can be especially important in educational settings, where the inferences can be used for assessment, diagnosing misconceptions, and providing informative feedback. In this paper, we develop a general framework for automatically making such inferences based on observed actions; this framework is particularly relevant for inferring student knowledge in educational games and other interactive virtual environments. Our approach relies on modeling action planning: We formalize the problem as a Markov decision process in which one must choose what actions to take to complete a goal, where choices will be dependent on one's beliefs about how actions affect the environment. We use a variation of inverse reinforcement learning to infer these beliefs. Through two lab experiments, we show that this model can recover people's beliefs in a simple environment, with accuracy comparable to that of human observers. We then demonstrate that the model can be used to provide real‐time feedback and to model data from an existing educational game.     

On Distinguishing Epistemic from Pragmatic Action

We present data and argument to show that in Tetris—a real-time, interactive video game—certain cognitive and perceptual problems are more quickly, easily, and reliably solved by performing actions in the world than by performing computational actions in the head alone. We have found that some of the translations and rotations made by players of this video game are best understood as actions that use the world to improve cognition. These actions are not used to implement a plan, or to implement a reaction; they are used to change the world in order to simplify the problem-solving task. Thus, we distinguish pragmatic actions—actions performed to bring one physically closer to a goal—from epistemic actions—actions performed to uncover informatioan that is hidden or hard to compute mentally. To illustrate the need for epistemic actions, we first develop a standard information-processing model of Tetris cognition and show that it cannot explain performance data from human players of the game—even when we relax the assumption of fully sequential processing. Standard models disregard many actions taken by players because they appear unmotivated or superfluous. However, we show that such actions are actually far from superfluous; they play a valuable role in improving human performance. We argue that traditional accounts are limited because they regard action as having a single function: to change the world. By recognizing a second function of action—an epistemic function—we can explain many of the actions that a traditional model cannot. Although our argument is supported by numerous examples specifically from Tetris, we outline how the new category of epistemic action can be incorporated into theories of action more generally.     

The integration of social influence and reward: Computational approaches and neural evidence

Decades of research have established that decision-making is dramatically impacted by both the rewards an individual receives and the behavior of others. How do these distinct influences exert their influence on an individual’s actions, and can the resulting behavior be effectively captured in a computational model? To address this question, we employed a novel spatial foraging game in which groups of three participants sought to find the most rewarding location in an unfamiliar two-dimensional space. As the game transitioned from one block to the next, the availability of information regarding other group members was varied systematically, revealing the relative impacts of feedback from the environment and information from other group members on individual decision-making. Both reward-based and socially-based sources of information exerted a significant influence on behavior, and a computational model incorporating these effects was able to recapitulate several key trends in the behavioral data. In addition, our findings suggest how these sources were processed and combined during decision-making. Analysis of reaction time, location of gaze, and functional magnetic resonance imaging (fMRI) data indicated that these distinct sources of information were integrated simultaneously for each decision, rather than exerting their influence in a separate, all-or-none fashion across separate subsets of trials. These findings add to our understanding of how the separate influences of reward from the environment and information derived from other social agents are combined to produce decisions.     

博弈决策影响因素及内在机制研究现状与展望

博弈决策是指“在包含利益相互依存的两个或多个参与者的博弈情境中所进行的行为决策”。博弈决策自进入心理学研究后已成为决策心理研究领域的热点。本文述评了博弈决策的影响因素及神经生理机制的研究现状;对未来研究方向进行了展望,包括加强内在机制研究与综合理论模型建构、改进和完善研究范式与加强对多种范式的考察、加强相关认知及情绪能力的作用机制研究、加强对语言等影响因素的跨文化与文化间比较、加强应用与训练研究。     

An evaluation of human act generation performance

Act generation is important in decision making in ill-defined problems where the subject must synthesize actions that might solve the problem. Two experiments explored human abilities to generate actions which might solve ill-defined decision problems. Subjects were given unlimited time to suggest as many solutions as possible. Their suggestions were compared to a hierarchical structural model of the actions developed by the experimenters that could be taken to solve the problem. Although subjects were capable of generating several actions that might be worth taking, their suggestions were far from complete. The second experiment replicated and extended these results by introducing instructions to generate quality actions. It employed a structural model derived from cluster analysis, an improved utility estimation technique, and offered substantial monetary incentives for quality or quantity of actions generated. This study confirmed the general conclusion that subjects fail to generate important high-utility actions. The implications of this result are discussed in respect to decision analysis in which a complete structural model of the decision problem is highly desirable.     

Cooperation in repeated interactions: A systematic review of Centipede game experiments, 1992–2016

Cooperation is a fundamental form of social interaction, and turn-taking reciprocity one of its most familiar manifestations. The Centipede game provides a formal model of such alternating reciprocal cooperation, but a backward induction (BI) argument appears to prove logically that instrumentally rational players would never cooperate in this way. A systematic review of experimental research reveals that human decision makers cooperate frequently in this game, except under certain extreme conditions. Several game, situational, and individual difference variables have been investigated for their influence on cooperation. The most influential are aspects of the payoff function (especially the social gain from cooperation and the risk associated with a cooperative move), the number of players, repetitions of the game, group vs. individual decisions, and players’ social value orientations (SVOs). Our review of experimental evidence suggests that other-regarding preferences, including prosocial behavioural dispositions and collective rationality, provide the most powerful explanation for cooperation.     

Effects of Decision Training on Decision Making and Performance in Young Tennis Players: An Applied Research

This study examined the effectiveness of a 10-week decision training program on decision making and performance in tennis. Decision training was based on the analysis combining tactical questioning and video feedback about players own actions. A quasi-experimental design with intermediate tennis players (control group, n = 6; experimental group, n = 5) was developed over 18 weeks, divided into 3 phases: preintervention, postintervention, and retention. A total of 13,383 game play actions were analyzed to test differences. Experimental group improves significantly decision making and performance and maintained improvements during the retention phase. Video feedback combined with questioning is recommended to achieve sport expertise.     

A game-theoretic analysis of bargaining with reputations

In this paper we introduce and discuss some models of bargaining. These have the form of repeated plays of a game among pairs of individuals, with the opponents in each particular game drawn randomly from a large population. The players' information about one another is limited to a single quantity, termed reputation, which summarizes the behavior of a player in previous trials of the game, and so changes endogenously. We distinguish some possible decision rules or “customs” which players might use to determine their moves in the game as a function of their own and their opponent's reputation, and investigate whether or not these actions lead to a suitably defined social equilibrium. We then compare the equilibrium customs from the point of view of the welfare of the population as a whole.     

Game relativity: how context influences strategic decision making

Existing models of strategic decision making typically assume that only the attributes of the currently played game need be considered when reaching a decision. The results presented in this article demonstrate that the so-called "co-operativeness" of the previously played prisoner's dilemma games influence choices and predictions in the current prisoner's dilemma game, which suggests that games are not considered independently. These effects involved reinforcement-based assimilation to the previous choices and also a perceptual contrast of the present game with preceding games, depending on the range and the rank of their co-operativeness. A. Parducci's (1965) range frequency theory and H. Helson's (1964) adaptation level theory are plausible theories of relative judgment of magnitude information, which could provide an account of these context effects.     

Dynamic probability of reinforcement for cooperation: Random game termination in the centipede game

Experimental games have previously been used to study principles of human interaction. Many such games are characterized by iterated or repeated designs that model dynamic relationships, including reciprocal cooperation. To enable the study of infinite game repetitions and to avoid endgame effects of lower cooperation toward the final game round, investigators have introduced random termination rules. This study extends previous research that has focused narrowly on repeated Prisoner's Dilemma games by conducting a controlled experiment of two‐player, random termination Centipede games involving probabilistic reinforcement and characterized by the longest decision sequences reported in the empirical literature to date (24 decision nodes). Specifically, we assessed mean exit points and cooperation rates, and compared the effects of four different termination rules: no random game termination, random game termination with constant termination probability, random game termination with increasing termination probability, and random game termination with decreasing termination probability. We found that although mean exit points were lower for games with shorter expected game lengths, the subjects' cooperativeness was significantly reduced only in the most extreme condition with decreasing computer termination probability and an expected game length of two decision nodes.     

Model and interactive algorithm of bi‐level multi‐objective decision‐making with multiple interconnected decision makers

Furthering a prior research on two‐person bi‐level multi‐objective decision‐making problems of the leader‐follower Stackelberg game, we present an extended model of bi‐level multi‐objective decision‐making with multiple interconnected decision makers at the lower level. In the model, the upper level decision maker acts as a leader and the lower level decision makers behave as the followers, and inter‐connections and interactions exist among these followers in decision‐making scenarios. Following the rules of leader‐follower Stackelberg game, we develop an interactive algorithm of the model for solving multi‐objective decision‐making problems and reflecting the interactive natures among the decision makers. Finally, the authors exemplify the model and algorithm, and draw a conclusion on points of contributions and the significance of this study in decision‐making and support. Copyright © 2001 John Wiley & Sons, Ltd.     

Examining the effect of task constraints on the emergence of creative action in young elite football players by using a method combining expert judgement and frequency count

In adult football, small-sided games are associated with increased action variability and suggested to promote more creative actions compared to regular 11v11 formats. This aligns with predictions from an ecological approach to perception and action that creative actions emerge in environments that grant variability in action, instead of being an expression of the individual player’s ability to generate ideas. To further evidence for this prediction, the current study aimed to expand this observation to elite youth football players. To this end, the number of different and creative actions in 4v4 small-sided game and a 11v11 regular-sided game among 10- to 12-year-old elite football players were examined. We analyzed a total of 7922 actions, which were categorized for type and creativity. Based on a subset of these actions, a panel of elite football coaches judged action types occurring below 0.5% as significantly more creative than more frequent action types. Hence, we used an occurrence of 0.5% as threshold to distinguish creative actions from non-creative actions. The results showed that the total number of actions, the number of different action types, the number creative actions and the number of different creative action types was significantly higher for the small-sided game format than the regular-sided game. In conclusion, this study confirms that in elite youth football, small-sided games induce a more variable and creative action repertoire. This shows that practitioners can design learning environments that promote the emergence of creative actions.     

时态结构中的恒常联系因果学习

人工智能研究中,行动这一概念通常在理论框架中有完全的定义。然而,现实中的行动有时难以完全刻画。智能体需要从过去的经验观察中习得行动的后果。本文提出一种基于时态结构的行动一结果学习理论。在自然数时间结构中,智能体通过观察过去的恒常联系建立因果关系。智能体依据己建立的因果关系指导将来的行动。同时,我们给出关于该理论的一个完全的逻辑演绎系统,并给出基于该逻辑的智能体行动的有效算法。     

Team reasoning: Solving the puzzle of coordination

In many everyday activities, individuals have a common interest in coordinating their actions. Orthodox game theory cannot explain such intuitively obvious forms of coordination as the selection of an outcome that is best for all in a common-interest game. Theories of team reasoning provide a convincing solution by proposing that people are sometimes motivated to maximize the collective payoff of a group and that they adopt a distinctive mode of reasoning from preferences to decisions. This also offers a compelling explanation of cooperation in social dilemmas. A review of team reasoning and related theories suggests how team reasoning could be incorporated into psychological theories of group identification and social value orientation theory to provide a deeper understanding of these phenomena.     

Suboptimal tradeoffs in information seeking

Explicit information-seeking actions are needed to evaluate alternative actions in problem-solving tasks. Information-seeking costs are often traded off against the utility of information. We present three experiments that show how subjects adapt to the cost and information structures of environments in a map-navigation task. We found that subjects often stabilize at suboptimal levels of performance. A Bayesian satisficing model (BSM) is proposed and implemented in the ACT-R architecture to predict information-seeking behavior. The BSM uses a local decision rule and a global Bayesian learning mechanism to decide when to stop seeking information. The model matched the human data well, suggesting that adaptation to cost and information structures can be achieved by a simple local decision rule. The local decision rule, however, often limits exploration of the environment and leads to suboptimal performance. We propose that suboptimal performance is an emergent property of the dynamic interactions between cognition and the environment.     

What's My Motivation? Video Games and Interpretative Performance

The interpretation of character motivations is a crucial part of the understanding of many narratives, including those found in video games. This interpretation can be complicated in video games by the player performing the role of a player‐character within the game narrative. Such performance finds the player making choices for the character and also interpreting the resulting character actions and their effect on the game's narrative. This can lead to interpretative difficulties for game narratives and their players: if a decision to act is made by the player, is it that the player's own imaginative reasons for acting warrant some narrative interpretations and exclude others? To answer this I argue that we need to investigate (a) the interactive ontology of video game narratives, (b) the notion of game playing as interpretative performance, and (c) the player‐character, an artifact through which performance is focused in narrative games. Doing so shows there to be at least two problems with the notion of the correct interpretation of narrative games. Neither of these problems entirely negates the normativity of game narratives, however, and so players are left with the problem of how they might decide which of the possible playings are warranted. I end by making some practical suggestions for the thoughtful and narratively interested game player.     

Pack behavior

We analyze a sequential decision model, where every decision maker has her own preferences over actions. Some actions have positive Externalities if a large enough population has previously chosen them, and those Externalities may lead a decision-maker to ignore her own preferred actions to benefit from those Externalities. We give a fairly general sufficient condition such that, on almost every equilibrium path, every decision maker eventually prefers to benefit from group Externalities. We show that this situation, called pack behavior, is observationally equivalent to a herd behavior, but it is triggered by positive Externalities generated by group decision, instead of standard belief-based decisions. We also show that this notion is not ex-ante Pareto-efficient in general.     

Learning to cooperate without awareness in multiplayer minimal social situations

Experimental and Monte Carlo methods were used to test theoretical predictions about adaptive learning of cooperative responses without awareness in minimal social situations—games in which the payoffs to players depend not on their own actions but exclusively on the actions of other group members. In Experiment 1, learning occurred slowly over 200 rounds in a dyadic minimal social situation but not in multiplayer groups. In Experiments 2–4, learning occurred rarely in multiplayer groups, even when players were informed that they were interacting strategically and were allowed to communicate with one another but were not aware of the game’s payoff structure. Monte Carlo simulation suggested that players approach minimal social situations using a noisy version of the win–stay, lose–shift decision rule, deviating from the deterministic rule less frequently after rewarding than unrewarding rounds.     

What you don’t know won’t hurt me: Costly (but quiet) exit in dictator games

We used simple economic games to examine pro-social behavior and the lengths that people will take to avoid engaging in it. Over two studies, we found that about one-third of participants were willing to “exit” a $10 dictator game and take $9 instead. The exit option left the receiver nothing, but also ensured that the receiver never knew that a dictator game was to be played. Because most social utility models are defined over monetary outcomes, they cannot explain choosing the ($9, $0) exit outcome over the dominating $10 dictator game, since the game includes outcomes of ($10, $0) and ($9, $1). We also studied exiting using a “private” dictator game. In the private game, the receiver never knew about the game or from where any money was received. Gifts in this game were added innocuously to a payment for a separate task. Almost no dictators exited from the private game, indicating that receivers’ beliefs are the key factor in the decision to exit. When, as in the private game, the receivers’ beliefs and expectations cannot be manipulated by exit, exit is seldom taken. We conclude that giving often reflects a desire not to violate others’ expectations rather than a concern for others’ welfare per se. We discuss the implications of our results for understanding ethical decisions and for testing and modeling social preferences. An adequate specification of social preferences should include “psychological” payoffs that directly incorporate beliefs about actions into the utility function.