Cognitive Evaluation of Machine Learning Agents

Advances in applying statistical Machine Learning (ML) led to several claims of human-level or near-human performance in tasks such as image classification & speech recognition. Such claims are unscientific primarily for two reasons, (1) They incorrectly enforce the notion that task-specific performance can be treated as manifestation of General Intelligence and (2) They are not verifiable as currently there is no set benchmark for measuring human-like cognition in a machine learning agent. Moreover, ML agent’s performance is influenced by knowledge ingested in it by its human designers. Therefore, agent’s performance may not necessarily reflect its true cognition. In this paper, we propose a framework that draws parallels from human cognition to measure machine’s cognition. Human cognitive learning is quite well studied in developmental psychology with frameworks and metrics in place to measure actual learning. To either believe or refute the claims of human-level performance of machine learning agent, we need scientific methodology to measure its cognition. Our framework formalizes incremental implementation of human-like cognitive processes in ML agents with an implicit goal to measure it. The framework offers guiding principles for measuring, (1) Task-specific machine cognition and (2) General machine cognition that spans across tasks. The framework also provides guidelines for building domain-specific task taxonomies to cognitively profile tasks. We demonstrate application of the framework with a case study where two ML agents that perform Vision and NLP tasks are cognitively evaluated.     

An expanded model for perceptual visual single object recognition system using expectation priming following neuroscientific evidence

Under numerous circumstances, humans recognize visual objects in their environment with remarkable response times and accuracy. Existing artificial visual object recognition systems have not yet surpassed human vision, especially in its universality of application. We argue that modeling the recognition process in an exclusive feedforward manner hinders those systems’ performance. To bridge that performance gap between them and human vision, we present a brief review of neuroscientific data, which suggests that considering an agent’s internal influences (from cognitive systems that peripherally interact with visual-perceptual processes) recognition can be improved. Then, we propose a model for visual object recognition which uses these systems’ information, such as affection, for generating expectation to prime the object recognition system, thus reducing its execution times. Later, an implementation of the model is described. Finally, we present and discuss an experiment and its results.     

Declarative working memory: A bio-inspired cognitive architecture proposal

Memory is considered one of the most important functions since it allows us to code, store and retrieve knowledge. These qualities make it an indispensable function for a virtual creature. In general, memory can be classified based on the durability of the stored data in working memory and long-term memory. Working memory refers to the capacity to maintain temporarily a limited amount of information in mind, which can then be used to support various abilities, including learning, reasoning, planning and decision-making. Unlike short-term memory, working memory is not only a storage site, but it is also a framework of interacting processes that involve the temporary storage and manipulation of information in the service of performing complex cognitive activities. Declarative memory is a type of long-term memory related with the storage of facts and events. This research focuses on the development of a cognitive architecture for the type of working memory that maintains and manipulates declarative information. The construction of the model was grounded in theoretical evidence taken from cognitive sciences such as neuroscience and psychology, which gave us the components and their processes. The model was evaluated through a case study that covers the encoding, storing, and retrieval stages. Our hypothesis is that a virtual creature endowed with our working memory model will provide faster access to the information needed for the ongoing task. Therefore, it improves the planning and decision-making processes.     

A study on understanding cognitive states through gait analysis

In this work, we attempted to find out the relationship between different gait patterns and their corresponding cognitive states by using different statistical and machine learning approaches. This paper strongly focusses on the simulations followed by implementation of the proposed cognitive states i.e. (i) EmotionOriented State (EOS) (ii) Thinking Oriented State (TOS) (iii) Memory Oriented State(MOS) (iv) Simple Regular Oriented State (SROS). A novel approach was implemented by creating different environmental contexts for different gaits in our lab. An experimental method was performed to isolate movement artifact using Independent Component Analysis from recorded EEG(Electroencephalogram) signals. Measurement of joint angles from joint positions captured using Kinect V2 sensors was done with the help of OpenSim software. The relationship between different gaits and mental states was established using Pearsons Correlation Coefficient, ANOVA(Analysis of variance) and SVM(Support Vector Machine) classifier respectively. A strong relationship was found between them. The SVM classifier for the EOS and the non-EOS states based on joint angles inferred an accuracy of 81.08%. The ROC Curve for SVM classification depicted an AUC (area under the curve) of 0.9724.     

多级评分认知诊断模型述评

认知诊断评估旨在探讨个体内部的知识掌握结构,并提供关于学生优缺点的详细诊断信息,以促进个体的全面发展。当前研究者已开发了大量0-1评分的认知诊断模型,但对于多级评分认知诊断模型的研究还比较少。本文对已有的多级评分认知诊断模型进行了归纳,介绍了模型的假设,计量特征以及适用范围,为实际应用者和研究者在多级评分认知诊断模型的比较和选用上提供借鉴和参考。最后,对未来关于多级评分诊断模型的研究方向进行了展望。     

正式反馈的发展与临床应用

摘要:正式反馈通过系统收集当事人的效果反馈来追踪其治疗进展,进而识别缺乏治疗进展的个案,并通过促进治疗策略的调整来阻止治疗失败。正式反馈是近20年来发展出的一种循证治疗手段,其目的为进一步提高当事人的咨询效果。正式反馈的发展大致经历了临床有效性检验和实用性发展两个阶段。正式反馈可用于促进个体、夫妻和团体咨询的效果,也可用于评估临床督导效果、确定督导个案和指导临床督导。未来研究可探讨咨询师对待正式反馈的态度,拓展正式反馈的使用价值,建立正式反馈系统的本土化常模。     

多时点结果跨期选择的决策效应及其心理机制

多时点结果跨期选择涉及两个及两个以上多个时点结果的权衡, 正日益成为跨期选择的研究重点。现有研究主要从两时点结果和序列结果两方面深入探讨了多时点结果跨期选择中的诸多决策效应, 包括：隐藏的零效应、象征效应、额外金额效应、前端金额效应以及序列效应, 并运用序列模型和权衡模型对上述决策效应进行阐释。未来还应深入探索同时涉及损益结果混合的多时点结果跨期选择, 运用决策过程追踪技术揭示决策效应的过程机制。     

寻找镜中的自我：精神分裂症患者的视觉观点采择

精神分裂症患者的视觉观点采择影响其必要的社交活动和生活质量。本文通过对精神分裂症患者视觉观点采择的研究进行梳理,总结出相关的心理特点：精神分裂症患者的视觉观点采择过程受损,表现出自我中心性和他人中心性。在神经生理上表现出与视觉观点采择相关的脑区激活减弱。可能的原因是患者认知功能中反应抑制能力和心理旋转能力的受损所致,建议采用认知干预疗法可能会改善患者的视觉观点采择能力。目前的研究论证了精神分裂症患者视觉观点采择过程的损伤,对于不同水平的视觉观点采择下精神分裂症患者的认知模式还未有明确的定论,未来的研究可以从该角度入手,更加明确精神分裂症患者不同水平视觉观点采择的特点和神经生理活动。     

Challenging anxious cognitions or accepting them? Exploring the efficacy of the cognitive elements of cognitive behaviour therapy and acceptance and commitment therapy in the reduction of children's fear of the dark

Anxiety is highly prevalent in pre-adolescent children. Distorted cognitions are characteristic for dysfunctional levels of anxiety. However, applying cognitive elements in pre-adolescent children cannot be fully ascertained, as it is not until adolescence before children can apply logical and abstract reasoning in a sophisticated manner. Cognitive behaviour therapy (CBT) and acceptance and commitment therapy (ACT) both target distorted cognitions. Whereas CBT encourages children to change the content of negative cognitions by applying cognitive restructuring, ACT stimulates youth to have a more accepting attitude towards these thoughts by applying cognitive defusion. The current study examined the efficacy of applying cognitive elements and compared the cognitive elements of CBT and ACT in pre-adolescent children. We included no behavioural elements to specifically study the developmental appropriateness of the cognitive elements in this age group. Highly anxious children, aged 8–12 years were randomised to a 30-minute cognitive restructuring (n = 21) or cognitive defusion intervention (n = 22). Subjective fear of the dark levels, behavioural darkness toleration, and comprehension and fun associated with the interventions were assessed. Both interventions had a significantly positive impact on children's fear of the dark. Cognitive restructuring led to more favourable results on subjective fear than cognitive defusion, no differences were found for darkness toleration.     

Does inconsistency always lead to negative affect? The influence of need for closure on affective reactions to cognitive inconsistency

We present an experiment showing that need for closure (NFC)—defined as the epistemic desire for certainty—can moderate individuals' affective reactions to cognitive inconsistency. Informed by Kruglanski and colleagues' new theory, that cognitive inconsistency elicits negative affect particularly under certain circumstances, we find that NFC (i.e. the desire for certain, stable and unambiguous knowledge) influences the strength of consistency effects and resulting negative affect. More specifically, we find that individuals who are high on NFC experience more negative affect upon encountering an inconsistent (vs. consistent) cognition. However, when individuals are low on NFC, inconsistency is irrelevant, and their affect depends on whether the ultimate outcome of the cognition is positive or negative. We discuss the theoretical and practical implications of this research.