Analyzing Machine-Learned Representations: A Natural Language Case Study

As modern deep networks become more complex, and get closer to human-like capabilities in certain domains, the question arises as to how the representations and decision rules they learn compare to the ones in humans. In this work, we study representations of sentences in one such artificial system for natural language processing. We first present a diagnostic test dataset to examine the degree of abstract composable structure represented. Analyzing performance on these diagnostic tests indicates a lack of systematicity in representations and decision rules, and reveals a set of heuristic strategies. We then investigate the effect of training distribution on learning these heuristic strategies, and we study changes in these representations with various augmentations to the training set. Our results reveal parallels to the analogous representations in people. We find that these systems can learn abstract rules and generalize them to new contexts under certain circumstances—similar to human zero-shot reasoning. However, we also note some shortcomings in this generalization behavior—similar to human judgment errors like belief bias. Studying these parallels suggests new ways to understand psychological phenomena in humans as well as informs best strategies for building artificial intelligence with human-like language understanding.     

Estimating the heritability of cognitive traits across dog breeds reveals highly heritable inhibitory control and communication factors

Animal Cognition - Trait heritability is necessary for evolution by both natural and artificial selection, yet we know little about the heritability of cognitive traits. Domestic dogs are a...     

Summation,Variety, and Indeterminate Value

In this paper, I consider two sorts of objections to summative theories of value. The first objection concerns “indeterminate” value. The second concerns the importance of variety. I argue that both objections pose serious problems for the summative approach. I also argue that if we accept certain plausible views about the value of variety, we should reject certain forms of argument concerning what sorts of states have intrinsic value.     

This time it’s personal: the memory benefit of hearing oneself

The production effect is the memory advantage of saying words aloud over simply reading them silently. It has been hypothesised that this advantage stems from production featuring distinctive information that stands out at study relative to reading silently. MacLeod (2011) (I said, you said: The production effect gets personal. Psychonomic Bulletin & Review, 18, 1197–1202. doi:10.3758/s13423-011-0168-8) found superior memory for reading aloud oneself vs. hearing another person read aloud, which suggests that motor information (speaking), self-referential information (i.e., “I said it”), or both contribute to the production effect. In the present experiment, we dissociated the influence on memory of these two components by including a study condition in which participants heard themselves read words aloud (recorded earlier) – a first for production effect research – along with the more typical study conditions of reading aloud, hearing someone else speak, and reading silently. There was a gradient of memory across these four conditions, with hearing oneself lying between speaking and hearing someone else speak. These results imply that oral production is beneficial because it entails two distinctive components: a motor (speech) act and a unique, self-referential auditory input.     

Editorial

The Journal of Ethics -     

Theory learning as stochastic search in the language of thought

We present an algorithmic model for the development of children's intuitive theories within a hierarchical Bayesian framework, where theories are described as sets of logical laws generated by a probabilistic context-free grammar. We contrast our approach with connectionist and other emergentist approaches to modeling cognitive development. While their subsymbolic representations provide a smooth error surface that supports efficient gradient-based learning, our symbolic representations are better suited to capturing children's intuitive theories but give rise to a harder learning problem, which can only be solved by exploratory search. Our algorithm attempts to discover the theory that best explains a set of observed data by performing stochastic search at two levels of abstraction: an outer loop in the space of theories and an inner loop in the space of explanations or models generated by each theory given a particular dataset. We show that this stochastic search is capable of learning appropriate theories in several everyday domains and discuss its dynamics in the context of empirical studies of children's learning.     

General recognition theory extended to include response times: Predictions for a class of parallel systems

General Recognition Theory (GRT; Ashby & Townsend, 1986) is a multidimensional theory of classification. Originally developed to study various types of perceptual independence, it has also been widely employed in diverse cognitive venues, such as categorization. The initial theory and applications have been static, that is, lacking a time variable and focusing on patterns of responses, such as confusion matrices. Ashby proposed a parallel, dynamic stochastic version of GRT with application to perceptual independence based on discrete linear systems theory with imposed noise (Ashby, 1989). The current study again focuses on cognitive/perceptual independence within an identification classification paradigm. We extend stochastic GRT and its implicated methodology for cognitive/perceptual independence, to an entire class of parallel systems. This goal is met in a distribution-free manner and includes all linear and non-linear systems satisfying very general conditions. A number of theorems are proven concerning stochastic forms of independence. However, the theorems all assume the stochastic version of decisional separability. A vital task remains to investigate the consequences of failures of stochastic decisional separability.     

One and Done? Optimal Decisions From Very Few Samples

In many learning or inference tasks human behavior approximates that of a Bayesian ideal observer, suggesting that, at some level, cognition can be described as Bayesian inference. However, a number of findings have highlighted an intriguing mismatch between human behavior and standard assumptions about optimality: People often appear to make decisions based on just one or a few samples from the appropriate posterior probability distribution, rather than using the full distribution. Although sampling‐based approximations are a common way to implement Bayesian inference, the very limited numbers of samples often used by humans seem insufficient to approximate the required probability distributions very accurately. Here, we consider this discrepancy in the broader framework of statistical decision theory, and ask: If people are making decisions based on samples—but as samples are costly—how many samples should people use to optimize their total expected or worst‐case reward over a large number of decisions? We find that under reasonable assumptions about the time costs of sampling, making many quick but locally suboptimal decisions based on very few samples may be the globally optimal strategy over long periods. These results help to reconcile a large body of work showing sampling‐based or probability matching behavior with the hypothesis that human cognition can be understood in Bayesian terms, and they suggest promising future directions for studies of resource‐constrained cognition.