Coding infant engagement in the Face-to-Face Still-Face paradigm using deep neural networks |
| |
| Affiliation: | 1. Department of Mathematics, University of British Columbia, Vancouver, British Columbia, Canada;2. Centre for Addiction and Mental Health, Toronto, Ontario, Canada;3. Mathematics & Statistics Undergraduate Program, McMaster University, Hamilton, Ontario, Canada;4. Integrated Biomedical Engineering & Health Sciences Undergraduate Program, McMaster University, Hamilton, Ontario, Canada;5. Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada;1. The University of Melbourne, Department of Medicine (Royal Melbourne Hospital), The University of Melbourne, VIC 3010, Australia;2. Department of Medicine (Royal Melbourne Hospital), The University of Melbourne.Academic Director, Australian Rehabilitation Research Centre, Royal Melbourne Hospital, Parkville, VIC 3010, Australia;1. Developmental Psychobiology Lab, IRCCS Mondino Foundation, Pavia, Italy;2. Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy;3. Pediatric Neuroscience Center & Developmental Psychobiology Lab, IRCCS Mondino Foundation, Pavia, Italy;1. Division of Behavioral Medicine and Clinical Psychology, Cincinnati Children’s Hospital Medical Center, USA;2. Center for Language and Brain, Shenzhen Institute of Neuroscience, Shenzhen, China;3. Department of Psychology, University of Maryland, USA;4. Department of Communication Sciences and Disorders, University of Utah, USA;5. Neurodevelopmental and Behavioral Phenotyping Service, National Institute of Mental Health, USA;6. Department of Speech and Hearing Sciences, Washington State University, USA;7. Office of the Clinical Director, National Human Genome Research Institute, USA;1. Department of Developmental and Social Psychology, University of Padua, Padua, Italy;2. 0-3 Center for the at-Risk Infant, Scientific Institute IRCCS Eugenio Medea, Bosisio Parini, Lecco, Italy;1. Department of Psychology and Gonda Brain Research Center, Bar-Ilan University, Ramat-Gan 52900, Israel;2. Center for Developmental Social Neuroscience, Reichman University, Herzliya 4601010, Israel |
| |
| Abstract: | BackgroundThe Face-to-Face Still-Face (FFSF) task is a validated and commonly used observational measure of mother-infant socio-emotional interactions. With the ascendence of deep learning-based facial emotion recognition, it is possible that common complex tasks, such as the coding of FFSF videos, could be coded with a high degree of accuracy by deep neural networks (DNNs). The primary objective of this study was to test the accuracy of four DNN image classification models against the coding of infant engagement conducted by two trained independent manual raters.Methods68 mother-infant dyads completed the FFSF task at three timepoints. Two trained independent raters undertook second-by-second manual coding of infant engagement into one of four classes: 1) positive affect, 2) neutral affect, 3) object/environment engagement, and 4) negative affect.ResultsTraining four different DNN models on 40,000 images, we achieved a maximum accuracy of 99.5% on image classification of infant frames taken from recordings of the FFSF task with a maximum inter-rater reliability (Cohen's κ-value) of 0.993.LimitationsThis study inherits all sampling and experimental limitations of the original study from which the data was taken, namely a relatively small and primarily White sample.ConclusionsBased on the extremely high classification accuracy, these findings suggest that DNNs could be used to code infant engagement in FFSF recordings. DNN image classification models may also have the potential to improve the efficiency of coding all observational tasks with applications across multiple fields of human behavior research. |
| |
| Keywords: | Machine learning Deep neural networks Face-to-Face Still-Face Task Developmental psychology Perinatal psychiatry |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
