Using automatic face analysis to score infant behaviour from video collected online

Online testing of infants by recording video with a webcam has the potential to improve the replicability of developmental studies by facilitating larger sample sizes and by allowing methods (including recruitment) to be specified in code. However, the recorded video still needs to be manually scored. This labour-intensive process puts downward pressure on sample sizes and requires subjective judgements that may not be reproducible in a different laboratory. Here we present the first fully automatic pipeline, using a face analysis software-as-a-service and a discriminant-analysis classifier to score infant videos acquired online. We compare human and machine performance for looking time and preferential looking paradigms; machine performance demonstrates a promising proof of principle for looking time and is above chance in classifying preferential looking. Additionally, we studied the characteristics of the video and the child that influenced automated scoring, so that future studies can acquire data that maximises the performance of automatic gaze coding and/or focus on improving automatic coding for particularly challenging data. We believe this technology has great promise for developmental science.     

Autonomy in Action: Linking the Act of Looking to Memory Formation in Infancy via Dynamic Neural Fields

Looking is a fundamental exploratory behavior by which infants acquire knowledge about the world. In theories of infant habituation, however, looking as an exploratory behavior has been deemphasized relative to the reliable nature with which looking indexes active cognitive processing. We present a new theory that connects looking to the dynamics of memory formation and formally implement this theory in a Dynamic Neural Field model that learns autonomously as it actively looks and looks away from a stimulus. We situate this model in a habituation task and illustrate the mechanisms by which looking, encoding, working memory formation, and long‐term memory formation give rise to habituation across multiple stimulus and task contexts. We also illustrate how the act of looking and the temporal dynamics of learning affect each other. Finally, we test a new hypothesis about the sources of developmental differences in looking.     

Examining the role of social cues in early word learning

Infants watched a video of an adult pointing towards two different objects while hearing novel labels. Analyses indicated that 14- and 18-month-olds looked longer at the target object, but only 18-month-olds showed word learning. The results suggest that different types of social cues are available at different ages.     

In any way,shape, or form? Toddlers’ understanding of shapes

Using the intermodal preferential looking paradigm, two-year-old children’s ability to discriminate valid (typical and atypical) squares, rectangles, triangles, and circles from invalid distractors was examined. The cognitive and environmental factors that might predict this ability were also investigated. Two-year-old children (N = 33) were able to discriminate squares, triangles, and circles, but not rectangles. No significant cognitive or environmental predictors of this ability were found. The results suggest that the ability to shape discriminate at age two is under considerable development and that other factors may be responsible for children’s ability to discriminate shapes.     

Overshadowing the Reminiscence Bump: Memories of a Struggle for Independence

We describe a study in which young and older groups of Bangladeshi participants recalled and dated autobiographical memories from across the lifespan. Memories were subsequently plotted in terms of the age of participants at time of encoding. As expected the reminiscence bump, preferential recall of memories from the period of 10 to 30 years of age, was observed. This was very marked in the younger group and but less so in the older group who also showed a second bump in the period 35 to 55 years of age. This second bump corresponded to the period of national conflict between Pakistan and the Bengalee people that resulted in the formation of an independent Bangladesh. It is proposed that both the reminiscence bump and later periods of unexpected rises in recall can be accounted for by the raised accessibility of sets of memories and this in turn is a product of the privileged encoding of highly self-relevant experiences.     

What am I supposed to be looking at? Controls and measures in inter-modal preferential looking

Intermodal preferential looking (IMPL) is widely used in experimental studies of infant development, especially language development. Control measures vary, and it is not clear how these affect findings. We examined effects of parental awareness of stimuli. Infants (17–19 months) looked at paired pictures, one name-known and one name-unknown, each assigned target status in 50 % of trials. Infants looked longer at a name-known than a name-unknown target, regardless of parents’ awareness. When parents were aware, looking to a name-unknown target increased over a paired name-known non-target. There is evidence that infants’ looking at pictures in this paradigm is not due to direct matching of targets to novel names, but is influenced by additional cues present, in a way that could alter the conclusions of studies of infant word learning and other aspects of infant learning. Implications of these findings are discussed, emphasising replicability and theoretical conclusions drawn from studies using this method.     

The use of social and salience cues in early word learning

This article explores young infants' ability to learn new words in situations providing tightly controlled social and salience cues to their reference. Four experiments investigated whether, given two potential referents, 15-month-olds would attach novel labels to (a) an image toward which a digital recording of a face turned and gazed, (b) a moving image versus a stationary image, (c) a moving image toward which the face gazed, and (d) a gazed-on image versus a moving image. Infants successfully used the recorded gaze cue to form new word-referent associations and also showed learning in the salience condition. However, their behavior in the salience condition and in the experiments that followed suggests that, rather than basing their judgments of the words' reference on the mere presence or absence of the referent's motion, infants were strongly biased to attend to the consistency with which potential referents moved when a word was heard.     

Rapid word learning in 13- and 17-month-olds in a naturalistic two-word procedure: looking versus reaching measures

This study investigated infants’ rapid learning of two novel words using a preferential looking measure compared with a preferential reaching measure. In Experiment 1, 21 13-month-olds and 20 17-month-olds were given 12 novel label exposures (6 per trial) for each of two novel objects. Next, in the label comprehension tests, infants were shown both objects and were asked, “Where’s the [label]?” (looking preference) and then told, “Put the [label] in the basket” (reaching preference). Only the 13-month-olds showed rapid word learning on the looking measure; neither age group showed rapid word learning on the reaching measure. In Experiment 2, the procedure was repeated 24 h later with 10 participants per age group from Experiment 1. After a further 12 labels per object, both age groups now showed robust evidence of rapid word learning, but again only on the looking measure. This is the earliest looking-based evidence of rapid word learning in infants in a well-controlled (i.e., two-word) procedure; our failure to replicate previous reports of rapid word learning in 13-month-olds with a preferential reaching measure may be due to our use of more rigorous controls for object preferences. The superior performance of the younger infants on the looking measure in Experiment 1 was not straightforwardly predicted by existing theoretical accounts of word learning.     

Task instructions and implicit theory of mind

It has been hypothesized that humans are able to track other’s mental states efficiently and without being conscious of doing so using their implicit theory of mind (iToM) system. However, while iToM appears to operate unconsciously recent work suggests it does draw on executive attentional resources ( Schneider, Lam, Bayliss, & Dux, 2012) bringing into question whether iToM is engaged efficiently. Here, we examined other aspects relating to automatic processing: The extent to which the operation of iToM is controllable and how it is influenced by behavioral intentions. This was implemented by assessing how task instructions affect eye-movement patterns in a Sally–Anne false-belief task. One group of subjects was given no task instructions (No Instructions), another overtly judged the location of a ball a protagonist interacted with (Ball Tracking) and a third indicated the location consistent with the actor’s belief about the ball’s location (Belief Tracking). Despite different task goals, all groups’ eye-movement patterns were consistent with belief analysis, and the No Instructions and Ball Tracking groups reported no explicit mentalizing when debriefed. These findings represent definitive evidence that humans implicitly track the belief states of others in an uncontrollable and unintentional manner.