The Observer Video-Pro: New software for the collection, management, and presentation of time-structured data from videotapes and digital media files

The Observer Video-Pro is a system for collecting, managing, analyzing, and presenting observational data. It integrates The Observer software with time code and multimedia hardware components. It extends the functionality of a conventional real-time event recording program in various ways. Observational data can be collected, reviewed, and edited with synchronized display of the corresponding video images. For optimal visual feedback during coding, one can display the video image in a window on the computer screen. Video playback from either a VCR or a digital media file can be controlled by the computer, allowing software-controlled jog, shuttle, and search functions. Besides a wide range of VCRs, The Observer Video-Pro supports all major digital video file formats. The software allows the user to summarize research findings in numerical, graphical, or multimedia format. One can create a time-event plot for a quick glance at the temporal structure of the observed process, or run specific analysis procedures and generate reports with statistics. An Event Summary function is available for exploratory and qualitative analysis. Video material can be summarized in a Video Play List, which allows on-screen summary presentations or the creation of highlight compilations on tape, CD, or other media. Video images can be captured and saved as disk files, for use as illustrations in documents, slides for presentations, and so forth. In this paper we describe the design and operation of the system, illustrated with a case study from research on Repetitive Strain Injury (RSI).     

Using automatic face analysis to score infant behaviour from video collected online

Online testing of infants by recording video with a webcam has the potential to improve the replicability of developmental studies by facilitating larger sample sizes and by allowing methods (including recruitment) to be specified in code. However, the recorded video still needs to be manually scored. This labour-intensive process puts downward pressure on sample sizes and requires subjective judgements that may not be reproducible in a different laboratory. Here we present the first fully automatic pipeline, using a face analysis software-as-a-service and a discriminant-analysis classifier to score infant videos acquired online. We compare human and machine performance for looking time and preferential looking paradigms; machine performance demonstrates a promising proof of principle for looking time and is above chance in classifying preferential looking. Additionally, we studied the characteristics of the video and the child that influenced automated scoring, so that future studies can acquire data that maximises the performance of automatic gaze coding and/or focus on improving automatic coding for particularly challenging data. We believe this technology has great promise for developmental science.     

Seeing Is Believing: Using Video Feedback in Cognitive Therapy for Social Anxiety Disorder

Distorted negative self-images and impressions appear to play a key role in maintaining Social Anxiety Disorder (SAD). In previous research, McManus et al. (2009) found that video feedback can help people undergoing cognitive therapy for SAD (CT-SAD) to develop a more realistic impression of how they appear to others, and this was associated with significant improvement in their social anxiety. In this paper we first present new data from 47 patients that confirms the value of video feedback. Ninety-eighty percent of the patients indicated that they came across more favorably than they had predicted after viewing a video of their social interactions. Significant reductions in social anxiety were observed during the following week and these reductions were larger than those observed after control periods. Comparison with our earlier data (McManus et al., 2009) suggests we may have improved the effectiveness of video feedback by refining and developing our procedures over time. The second part of the paper outlines our current strategies for maximizing the impact of video feedback. The strategies have evolved in order to help patients with SAD overcome a range of processing biases that could otherwise make it difficult for them to spot discrepancies between their negative self-imagery and the way they appear on video.     

A duration model analysis of consumer preferences and determinants of video game consumption

Despite the enormous revenues generated by the video game industry in recent years, relatively little research has been undertaken into consumer preferences and the determinants of video game consumption. This study addresses this deficiency through the analysis of data from a popular online video game that includes historic behavioural information for 1,408 consumers participating in 728,811 unique rounds of gameplay. We analyse these data with the goal of estimating determinants of the aggregate amount of time that a consumer spends playing the game. Through the estimation of duration models, we show that less experienced consumers are less likely to continue playing the game at any given point, possibly due to having achieved mastery and becoming bored. However, we also find that consumers tend to play the game for longer periods when using a wider range of character roles and vehicles, implying that a consumer's interest can be maintained through exposure to greater variety. Our results represent the first such evidence on in‐game consumer preferences, which has important implications for video game consumption through optimisation of gameplay experiences to satisfy these preferences.     

Dynamic brain imaging: Event-related optical signal (EROS) measures of the time course and localization of cognitive-related activity

This paper describes the concept of dynamic brain imaging and introduces a new methodology, the event-related optical signal, or EROS. Dynamic brain imaging allows one to study noninvasively the time course of activity in specific brain areas. Brain imaging data can contribute to the analysis of the subcomponents of the human information processing system and of their interactions. Several brain imaging methods provide data that possess spatial and temporal resolution at various degrees and can be used for this purpose. In this paper, we focus on the EROS method, which yields data with millisecond temporal resolution and subcentimeter spatial resolution. We describe the fundamentals of this method and report several examples of the types of data that can be obtained with it. Finally, we discuss the possibility of combining different imaging methods, as well as the advantages and limitations that can be expected in this process.     

Video tracking in the extreme: Video analysis for nocturnal underwater animal movement

Computer analysis of video footage is one option for recording locomotor behavior for a range of neurophysiological and behavioral studies. This technique is reasonably well established and accepted, but its use for some behavioral analyses remains a challenge. For example, filming through water can lead to reflection, and filming nocturnal activity can reduce resolution and clarity of filmed images. The aim of this study was to develop a noninvasive method for recording nocturnal activity in aquatic decapods and test the accuracy of analysis by video tracking software. We selected crayfish, Cherax destructor, because they are often active at night, they live underwater, and data on their locomotion is important for answering biological and physiological questions such as how they explore and navigate. We constructed recording arenas and filmed animals in infrared light. Wethen compared human observer data and software-acquired values. In this article, we outline important apparatus and software issues to obtain reliable computer tracking.     

Enumeration versus multiple object tracking: the case of action video game players

Here, we demonstrate that action video game play enhances subjects' ability in two tasks thought to indicate the number of items that can be apprehended. Using an enumeration task, in which participants have to determine the number of quickly flashed squares, accuracy measures showed a near ceiling performance for low numerosities and a sharp drop in performance once a critical number of squares was reached. Importantly, this critical number was higher by about two items in video game players (VGPs) than in non-video game players (NVGPs). A following control study indicated that this improvement was not due to an enhanced ability to instantly apprehend the numerosity of the display, a process known as subitizing, but rather due to an enhancement in the slower more serial process of counting. To confirm that video game play facilitates the processing of multiple objects at once, we compared VGPs and NVGPs on the multiple object tracking task (MOT), which requires the allocation of attention to several items over time. VGPs were able to successfully track approximately two more items than NVGPs. Furthermore, NVGPs trained on an action video game established the causal effect of game playing in the enhanced performance on the two tasks. Together, these studies confirm the view that playing action video games enhances the number of objects that can be apprehended and suggest that this enhancement is mediated by changes in visual short-term memory skills.     

Video Game Therapy: Application of the Couple and Family Technology Framework

The couple and family technology framework outlines how technology affects couple and family relationships. This framework has been applied to understanding the impact of technology on infidelity, contributed to our understanding of online gaming issues in couple relationships, and sexting. The purpose of this paper is to outline how to implement video gaming as a disciplinary tool for youth congruent with the couple and family technology framework. Specifically, we identify the elements of video games that can be applied as an intervention in family treatment with youth in a token economy. This proposed technique focuses on utilizing technology (and specifically video games) as a strategy to improve youth’s behavior. Consistent with the couple and family technology framework, this paper outlines how accessibility, affordability, accommodation, and approximation can be used to improve behavior as well as improve parent–child relationships. Implications are included.     

Exploring the role of emotional reflexivity in research with children

This paper explores how emotions mediate the research process. I draw upon a collaborative ethnography with children in a primary school to make a case for the practice of emotional reflexivity in research. More specifically I explore this in the context of a series of den-building research workshops I initiated with the young participants. Drawing upon a series of vignettes, I illustrate how reflecting upon and through my emotional responses to video data captured by children during these workshops gave presence to my movements through the data, thus recognising moments when meaning-making was entangled with prominent memories I had of the field and hidden shifts I made whilst interpreting the data. I suggest that as researchers we can use emotional reflexivity to recognise, on the one-hand, how our personal histories influence our research engagements, and on the other, how the spaces and places we inhabit as researchers shape our thoughts and feelings. I will show how emotional reflexivity offers opportunities for researchers to engage with their own relational emplacement during fieldwork and to use this awareness to develop their understandings of children's lives.     

Optical and electronic systems for spatial and temporal analysis of video images

In this paper are described two systems for spatial (two-dimensional coordinates) and temporal analysis of video images. In the optical version, the Apple IIe generates a pointer on an Apple monitor; the pointer is then superimposed on the video image of a second video monitor, by reflecting the pointer with a half-silvered mirror. In the electronic version, a special card was designed for the Apple IIe to generate a pointer directly on the other video monitor. Both systems use the same software. The procedures of spatial and temporal data acquisition and storage are described as well as the reading, writing, printing, removing, listing, chaining, and dating operations of the digital and graphics files.     

Emotion perception and recognition: An exploration of cultural differences and similarities

The electroencephalogram (EEG) is a powerful method for investigation of different cognitive processes. Recently, EEG analysis became very popular and important, with classification of these signals standing out as one of the mostly used methodologies. Emotion recognition is one of the most challenging tasks in EEG analysis since not much is known about the representation of different emotions in EEG signals. In addition, inducing of desired emotion is by itself difficult, since various individuals react differently to external stimuli (audio, video, etc.). In this article, we explore the task of emotion recognition from EEG signals using distance-based time-series classification techniques, involving different individuals exposed to audio stimuli. Furthermore, since some of the participants in the experiment do not understand the language of the stimuli, we also investigate the impact of language understanding on emotion perception. Using time-series distances as features for the construction of new data representations, applied here for the first time to emotion recognition and related tasks, lead to excellent classification performance, indicating that differences between EEG signals can be used to build successful models for recognition of emotions, individuals, and other related tasks. In the process, we observed that cultural differences between the subjects did not have a significant impact on the recognition tasks and models.     

Reading instruction and remediation with the aid of computer speech

In this paper, we describe the application of new computer and speech synthesis technologies for reading instruction. Stories are presented on the computer screen, and readers may designate words or parts of words that they cannot read for immediate speech feedback. The important contingency between speech sounds and their corresponding letter patterns is emphasized by displaying the letter patterns in reverse video as they are spoken. Speech feedback is provided by an advanced text-to-speech synthesizer (DECtalk). Intelligibility data are presented, showing that DECtalk can be understood almost as well as natural human speech by both normal adults and reading disabled children. Preliminary data from 26 disabled readers indicate that there are significant benefits of speech feedback for reading comprehension and word recognition, and that children enjoy reading with the system.     

Dynamic Bayesian Network Modeling of Game-Based Diagnostic Assessments

Abstract

Digital games offer an appealing environment for assessing student proficiencies, including skills and misconceptions in a diagnostic setting. This paper proposes a dynamic Bayesian network modeling approach for observations of student performance from an educational video game. Drawing from and advancing methods in dynamic Bayesian networks, cognitive diagnostic modeling, and analysis of process data, a Bayesian approach to model construction, calibration, and use in facilitating inferences about students on the fly is described, and implemented in the context of an educational video game.     

Appropriating Video Surveillance for Art and Environmental Awareness: Experiences from ARTiVIS

Arts, Real-Time Video and Interactivity for Sustainability (ARTiVIS) is an ongoing collaborative research project investigating how real-time video, DIY surveillance technologies and sensor data can be used as a tool for environmental awareness, activism and artistic explorations. The project consists of a series of digital contexts for aesthetic contemplation of nature and civic engagement, aiming to foster awareness and empowerment of local populations through DIY surveillance. At the core of the ARTIVIS efforts are a series of interactive installations (namely B-Wind!, Hug@tree and Play with Fire), that make use of surveillance technologies and real-time video as raw material to promote environmental awareness through the emotion generated by real-time connections with nature. Throughout the project development, the surveillance concept has been shifting from the use of surveillance technology in a centralized platform, to the idea of veillance with distributed peer-to-peer networks that can be used for science and environmental monitoring. In this paper we present the history of the ARTiVIS project, related and inspiring work, describe ongoing research work and explore the present and future challenges of appropriating surveillance technology for artistic, educational and civic engagement purposes.     

This is your brain on violent video games: Neural desensitization to violence predicts increased aggression following violent video game exposure

Previous research has shown that media violence exposure can cause desensitization to violence, which in theory can increase aggression. However, no study to date has demonstrated this association. In the present experiment, participants played a violent or nonviolent video game, viewed violent and nonviolent photos while their brain activity was measured, and then gave an ostensible opponent unpleasant noise blasts. Participants low in previous exposure to video game violence who played a violent (relative to a nonviolent) game showed a reduction in the P3 component of the event-related brain potential (ERP) to violent images (indicating physiological desensitization), and this brain response mediated the effect of video game content on subsequent aggressive behavior. These data provide the first experimental evidence linking violence desensitization with increased aggression, and show that a neural marker of this process can at least partially account for the causal link between violent game exposure and aggression.     

A Hierarchical Ornstein–Uhlenbeck Model for Continuous Repeated Measurement Data

In this paper, we present a diffusion model for the analysis of continuous-time change in multivariate longitudinal data. The central idea is to model the data from a single person with an Ornstein–Uhlenbeck diffusion process. We extend it hierarchically by allowing the parameters of the diffusion process to vary randomly over different persons. With this approach, both intra and interindividual differences are analyzed simultaneously. Furthermore, the individual difference parameters can be regressed on covariates, thereby providing an explanation of between-person differences. Unstructured and unbalanced data pose no problem for the model to be applied. We demonstrate the method on data from an experience sampling study to investigate changes in the core affect. It can be concluded that different factors from the five factor model of personality are related to features of the trajectories in the core affect space, such as the cross-correlation and variability of the changes.     

CowLog: Open-source software for coding behaviors from digital video

We have developed CowLog, which is open-source software for recording behaviors from digital video and is easy to use and modify. CowLog tracks the time code from digital video files. The program is suitable for coding any digital video, but the authors have used it in animal research. The program has two main windows: a coding window, which is a graphical user interface used for choosing video files and defining output files that also has buttons for scoring behaviors, and a video window, which displays the video used for coding. The windows can be used in separate displays. The user types the key codes for the predefined behavioral categories, and CowLog transcribes their timing from the video time code to a data file. CowLog comes with an additional feature, an R package called Animal, for elementary analyses of the data files. With the analysis package, the user can calculate the frequencies, bout durations, and total durations of the coded behaviors and produce summary plots from the data.     

A computer-automated laboratory for studying complex perception-action skills

In this paper, we describe a computer-based laboratory for collecting and analyzing real-time performance and eye-movement data by using video game-like tasks. These tasks assess skills that require the integration of varying numbers of perception and action components. Customized hardware and software are described for configuring tasks, collecting data, reconstructing performance, and organizing, editing, and reducing data. General design considerations are emphasized for solving several of the problems encountered in constructing a real-time data collection system. The paper concludes with examples of data that illustrate the system’s usefulness for studying complex perception-action skills.     

Capturing non-linear temporally embedded processes in organizations using recurrence quantification analysis

ABSTRACT

Despite the growing consensus that the majority of psychological phenomena at work are temporally embedded and highly dynamic, existing research is often based on simplified theoretical and methodological models, which take limited account of process dynamics and especially non-linear growth trajectories. In this paper, we highlight the potentials of using recurrence quantifications analysis (RQA) and an extension of RQA – cross-recurrence quantification analysis (CRQA) – for researching process dynamics in organizations. (C)RQA is a powerful technique that can be used to both visualize and quantify time-series data such as repeated measurements of psychological states or sequentially coded dyadic and team interactions. To illustrate the manifold opportunities of (C)RQA, we present three application examples focusing on individuals as systems, dyads as systems, and teams as systems. Specifically, we highlight how (C)RQA can be applied to individual diary data, to leader-follower communication dynamics observed during annual appraisal interviews, and to high-density coded team interactions observed during organizational meetings. We discuss the strengths and limitations of (C)RQA and provide recommendations for researchers interested in using the method.