Detecting eye movements in dynamic environments

To take advantage of the increasing number of in-vehicle devices, automobile drivers must divide their attention between primary (driving) and secondary (operating in-vehicle device) tasks. In dynamic environments such as driving, however, it is not easy to identify and quantify how a driver focuses on the various tasks he/she is simultaneously engaged in, including the distracting tasks. Measures derived from the driver’s scan path have been used as correlates of driver attention. This article presents a methodology for analyzing eye positions, which are discrete samples of a subject’s scan path, in order to categorize driver eye movements. Previous methods of analyzing eye positions recorded in a dynamic environment have relied completely on the manual identification of the focus of visual attention from a point of regard superimposed on a video of a recorded scene, failing to utilize information regarding movement structure in the raw recorded eye positions. Although effective, these methods are too time consuming to be easily used when the large data sets that would be required to identify subtle differences between drivers, under different road conditions, and with different levels of distraction are processed. The aim of the methods presented in this article are to extend the degree of automation in the processing of eye movement data by proposing a methodology for eye movement analysis that extends automated fixation identification to include smooth and saccadic movements. By identifying eye movements in the recorded eye positions, a method of reducing the analysis of scene video to a finite search space is presented. The implementation of a software tool for the eye movement analysis is described, including an example from an on-road test-driving sample.     

Allocation of visual attention while driving with simulated augmented reality

Augmented reality (AR) technologies aim to optimize the visual attention of the driver by increasing the salience of high value elements. In such systems, ‘value’ is typically seen as linked to the general activity of driving, but not manoeuvres. However, several studies have shown that during activity, eye movements are specific to the immediate goal. In our experiment, 48 participants watched videos of automobile driving situations, during which they had to make decisions. In these videos, some situational cues were graphically highlighted. Depending on the experimental group, highlighted cues related to either the general driving task (e.g. road signs, pedestrians) or to a specific manoeuvre (e.g. look for overtaking cars before changing the lane). The results show that AR impacts the allocation of visual attention more strongly during the decision-making phase. In all AR conditions, the ability to extract information is less efficient. In particular, highlighting (by AR) general cues does not affect the detection of cues related to a manoeuvre, but it does change the allocation of visual attention: fixations are more numerous and less task-specific.     

Operant control of eye movements

In a monitoring situation eye movements were required in order for signals to be presented. Detection of signals was the reinforcement. A multiple schedule of fixed-interval reinforcement, differential reinforcement of low rate, and fixed-ratio reinforcement was established for eye movements. Results demonstrated that an eye movement can act as an operant controlled by its consequences. Operant control of eye movements has important implications for human factor analysts concerned with "attention".     

Effects of antisocial personality, cocaine and opioid dependence, and gender on eye movement control

Substance-dependent patients have been reported to exhibit abnormal smooth pursuit and saccadic eye movements. However, contrasts of the effects of different substances and the effects of comorbid psychiatric symptoms such as antisocial personality have rarely been performed. Separate analyses examined the effects of cocaine dependence, opioid dependence, or antisocial personality disorder. In each analysis, sex was included as an additional grouping factor. The dependent measures were the gain of smooth pursuit eye movement and the delay and accuracy of saccadic eye movement. Analyses of covariance indicated that both cocaine dependence and antisocial personality, but not opiate dependence, were associated with a significant reduction in gain of smooth pursuit eye movement. Cocaine dependence and antisocial personality also slowed the onset of saccadic eye movements, but only in men. No group differences were found in the accuracy of saccadic eye movements. The results suggest that the neurophysiological effects of cocaine dependence and antisocial personality overshadow the effects of heroin. The significance of these findings for visual attention and reading skill has yet to be assessed.     

Using E-Z reader to model the effects of higher level language processing on eye movements during reading

Although computational models of eye-movement control during reading have been used to explain how saccadic programming, visual constraints, attention allocation, and lexical processing jointly affect eye movements during reading, these models have largely ignored the issue of how higher level, postlexical language processing affects eye movements. The present article shows how one of these models, E-Z Reader (Pollatsek, Reichle, & Rayner, 2006c), can be augmented to redress this limitation. Simulations show that with a few simple assumptions, the model can account for the fact that effects of higher level language processing are not observed on eye movements when such processing is occurring without difficulty, but can capture the patterns of eye movements that are observed when such processing is slowed or disrupted.     

What attracts attention during police pursuit driving?

Efficient deployment of attention is important to the safe execution of tasks with a high content of visual information, such as driving. Chasing a lead vehicle is an extremely demanding and dangerous task, though little is known of the visual skills required. A study is reported that recorded the eye movements of police drivers and two control groups (novices and age‐ and experienced‐ matched controls) while watching a series of video clips of driving. The clips included pursuits, emergency response drives, and control drives (at normal speeds) around Nottinghamshire, UK. Analysis of gaze durations within certain categories of stimuli revealed that daytime pursuit drives correspond with an increase in gaze durations on a lead car (controlled for exposure), though police drivers direct their attention to other sources of potential hazards, such as pedestrians, more so than other drivers. Copyright © 2005 John Wiley & Sons, Ltd.     

Obstacle avoidance under automated steering: Impact on driving and gaze behaviours

Within the context of more and more autonomous vehicles, an automatic lateral control device (AS: Automatic Steering) was used to steer the vehicle along the road without drivers’ intervention. The device was not able to detect and avoid obstacles. The experiment aimed to analyse unexpected obstacle avoidance manoeuvres when lateral control was delegated to automation. It was hypothesized that drivers skirting behaviours and eye movement patterns would be modified with automated steering compared with a control situation without automation. Eighteen participants took part in a driving simulator study. Steering behaviours and eye movements were analysed during obstacle avoidance episodes. Compared with driving without automation, skirting around obstacles was found to be less effective when drivers had to return from automatic steering to manual control. Eye movements were modified in the presence of automatic steering, revealing further ahead visual scanning of the driving environment. Resuming manual control is not only a problem of action performance but is also related to the reorganisation of drivers’ visual strategies linked to drivers’ disengagement from the steering task. Assistance designers should pay particular attention to potential changes in drivers’ activity when carrying out development work on highly automated vehicles.     

The effect of a simulated hearing loss on performance of an auditory memory task in driving

Hearing loss has been shown to exacerbate the effect of auditory distraction on driving performance in older drivers. This study controlled for the potentially confounding factor of age-related cognitive decrements, by applying a simulated hearing loss in young, normally hearing individuals. Participants drove a simulated road whilst completing auditory tasks under simulated hearing loss or normal hearing conditions. Measures of vehicle control, eye movements and auditory task performance were recorded. Results showed that performing the auditory tasks whilst driving resulted in more stable lateral vehicle control and a reduction in gaze dispersion around the road centre. These trends were not exacerbated by simulated hearing loss, suggesting no effect of hearing loss on vehicle control or eye movement patterns during auditory task engagement. However, a small effect of simulated hearing loss on the performance of the most complex auditory task was observed during driving, suggesting that the use of sound-based in-vehicle systems may be problematic for hearing impaired individuals. Further research incorporating a wider variety of driving scenarios and auditory tasks is required in order to confirm the findings of this study.     

Latencies of stimulus-driven eye movements are shorter in dyslexic subjects

Eye movements latencies toward peripherally presented stimuli were measured in 10-year-old dyslexic and control children. Dyslexic subjects, previously found to be oversensitive to stimulation of the magnocellular channel, showed reduced latencies as compared to normally reading controls. An attention shifting task was also used which showed no group differences in latencies of eye movements. The data are discussed in terms of the hypothesis of magnocellular system alteration and attention dysfunction in dyslexia. Additionally, sex differences in eye movement latencies were found and are discussed.     

Driven to distraction? The effect of simulated red light running camera flashes on attention and oculomotor control

Do similar factors influence the allocation of attention in visually sparse and abstract laboratory paradigms and complex real-world scenes? To explore this question we conducted a series of experiments that examined whether the flash that accompanies a Red Light Running Camera (RLRC) can capture observers' attention away from important roadway changes. Inhibition of Return (IOR) and eye movement direction served as indices of the spatial allocation of attention. In two experiments, participants were slower to respond to the brake lights of a vehicle in a driving scene when an RLRC flash occurred nearby or were slower to initiate eye movements to brake light signals (IOR effects). In a third experiment, we found evidence that less prevalent RLRC flashes disrupted eye movement control. Results suggest that attention can be misdirected as a result of RLRC flashes and provide additional evidence that findings from simple laboratory paradigms can predict the allocation of attention in complex settings that are more familiar to observers.     

Microsaccades keep the eyes' balance during fixation

During fixation of a stationary target, small involuntary eye movements exhibit an erratic trajectory-a random walk. Two types of these fixational eye movements are drift and microsaccades (small-amplitude saccades). We investigated fixational eye movements and binocular coordination using a statistical analysis that had previously been applied to human posture control. This random-walk analysis uncovered two different time scales in fixational eye movements and identified specific functions for microsaccades. On a short time scale, microsaccades enhanced perception by increasing fixation errors. On a long time scale, microsaccades reduced fixation errors and binocular disparity (relative to pure drift movements). Thus, our findings clarify the role of oculomotor processes during fixation.     

Attention and saccadic eye movements

Four threshold detection experiments addressed three issues concerning the relationship between movements of spatial attention and saccadic eye movements: (a) the time course of attention shifts wit saccades, (b) the response of the two systems to changes in stimulus parameters, and (c) the relationship of attention to saccadic suppression. These issues bear on the more general question of the degree of independence between the saccadic and attentional movement systems. The results of these experiments support the contention that the mechanisms that shift attention are separate from those that control saccadic eye movements. Relevant events in the visual field periphery, however, will trigger both a saccade and attention shift. The attentional response to such events does not appear to be under subjects' control. The implication of these results for theories of saccadic suppression is discussed.     

The relationship between eye movements and spatial attention

Most previous studies of the attentional consequences of making saccadic eye movements have used peripheral stimuli to elicit eye movements. It is argued that in the light of evidence showing automatic “capture” of attention by peripheral stimuli, these experiments do not distinguish between attentional effects due to peripheral stimuli and those due to eye movements. In the present study, spatial attention was manipulated by varying the probability that peripheral probe stimuli would appear in different positions, while saccades were directed by a central arrow, enabling the effects of attention and eye movements to be separated. The results showed that the time to react to a peripheral stimulus could be shortened both by advance knowledge of its likely position and, separately, by preparing to make a saccade to that position. When the saccade was directed away from the most likely position of the probe, the targets for attention and eye movements were on opposite sides of the display. In this condition, the effects of preparing to make a saccade proved to be stronger than the effects of attentional allocation until well after the saccade had finished, suggesting that making a saccade necessarily involves the allocation of attention to the target position. The effects of probe stimuli on saccade latencies were also examined: probe stimuli that appeared before the saccade shortened saccade latencies if they appeared at the saccade target, and lengthened saccade latencies if they appeared on the opposite side of fixation. These facilitatory and inhibitory effects were shown to occur at different stages of saccade preparation and suggest that attention plays an important role in the generation of voluntary eye movements. The results of this study indicate that while it is possible to make attention movements without making corresponding eye movements, it is not possible to make an eye movement (in the absence of peripheral stimulation) without making a corresponding shift in the focus of attention.     

Neural representation for the perception of the intentionality of actions

A novel population of cells is described, located in the anterior part of the superior temporal sulcus (STSa, sometimes called STPa) of the temporal lobe in the macaque monkey. These cells respond selectively to the sight of reaching but only when the agent performing the action is seen to be attending to the target position of the reaching. We describe how such conditional selectivity can be generated from the properties of distinct cell populations within STSa. One cell population responds selectively to faces, eye gaze, and body posture, and we argue that subsets of these cells code for the direction of attention of others. A second cell population is selectively responsive to limb movement in certain directions (e.g., responding to an arm movement to the left but not to an equivalent leg movement or vice versa). The responses of a subset of cells sensitive to limb movement are modulated by the direction of attention (indicated by head and body posture of the agent performing the action). We conclude that this combined analysis of direction of attention and body movements supports the detection of intentional actions.     

The influence of action video game playing on eye movement behaviour during visual search in abstract,in-game and natural scenes

Action game playing has been associated with several improvements in visual attention tasks. However, it is not clear how such changes might influence the way we overtly select information from our visual world (i.e. eye movements). We examined whether action-video-game training changed eye movement behaviour in a series of visual search tasks including conjunctive search (relatively abstracted from natural behaviour), game-related search, and more naturalistic scene search. Forty nongamers were trained in either an action first-person shooter game or a card game (control) for 10 hours. As a further control, we recorded eye movements of 20 experienced action gamers on the same tasks. The results did not show any change in duration of fixations or saccade amplitude either from before to after the training or between all nongamers (pretraining) and experienced action gamers. However, we observed a change in search strategy, reflected by a reduction in the vertical distribution of fixations for the game-related search task in the action-game-trained group. This might suggest learning the likely distribution of targets. In other words, game training only skilled participants to search game images for targets important to the game, with no indication of transfer to the more natural scene search. Taken together, these results suggest no modification in overt allocation of attention. Either the skills that can be trained with action gaming are not powerful enough to influence information selection through eye movements, or action-game-learned skills are not used when deciding where to move the eyes.     

The role of attention in the facilitation effect and another "inhibition of return"

The present study investigated how anticipation of a target's appearance affects human attention to gaze cues provided by a schematic face. Subjects in a 'catch' group received a high number of 'catch' trials, in which no target stimulus appeared. Subjects in the control group did not receive any catch trials. As in previous studies, both groups showed a facilitation effect to the cued location during shorter stimulus onset asynchrony (SOA). In both groups, an analysis of eye movements confirmed that subjects' eyes remained on the fixation point, ruling out the possibility that the facilitation effect was due to shifting eye movements (saccades) as opposed to a shift in covert attention. But while the control group's response time (RT) decreased as SOA increased, the catch group's RT had a U-shaped pattern and the facilitation effect to the cued location was reversed at the longest SOA (1005 ms). These results suggest that subjects in the catch group disengaged their attention during long SOAs because they expected the trial to be a catch trial. This disengagement of attention during long SOAs results in a delay before attention could be re-focused to the previous location regardless of the cue validity ["IOR (inhibition of return)"-like-phenomenon]. Unlike the conventional IOR, we suggest that this "IOR"-like phenomenon caused by an unpredictive central gaze cue is likely to be mediated by an endogenous mechanism.     

Conditional control in visual selection

Attention and eye movements provide a window into the selective processing of visual information. Evidence suggests that selection is influenced by various factors and is not always under the strategic control of the observer. The aims of this tutorial review are to give a brief introduction to eye movements and attention and to outline the conditions that help determine control. Evidence suggests that the ability to establish control depends on the complexity of the display as well as the point in time at which selection occurs. Stimulus-driven selection is more probable in simple displays than in complex natural scenes, but it critically depends on the timing of the response: Salience determines selection only when responses are triggered quickly following display presentation, and plays no role in longer-latency responses. The time course of selection is also important for the relationship between attention and eye movements. Specifically, attention and eye movements appear to act independently when oculomotor selection is quick, whereas attentional processes are able to influence oculomotor control when saccades are triggered only later in time. This relationship may also be modulated by whether the eye movement is controlled in a voluntary or an involuntary manner. To conclude, we present evidence that shows that visual control is limited in flexibility and that the mechanisms of selection are constrained by context and time. The outcome of visual selection changes with the situational context, and knowing the constraints of control is necessary to understanding when and how visual selection is truly controlled by the observer.     

The safety and conspicuity of pedestrian crossing at roundabouts: The effect of median refuge island and zebra markings

Roundabouts are one of the most used road intersections because, compared to signalized ones, they reduce conflict points between traffic flows and moderate driving speed. Great attention should also be paid to vulnerable road users at roundabouts. According to accident statistics, in fact, accessibility of pedestrians and cyclists is not always ensured.This paper has evaluated the effects on the visibility of pedestrian crossing before and after the displacement of zebra markings, moved before intersections, and the introduction of media refuge islands and “Yield here to pedestrians” vertical signs. The above effects have been assessed by before-after analysis of speed and visual behaviour of drivers approaching the crosswalk.Moreover, the analysis of the drivers’ eye movements has highlighted the most salient elements of the pedestrian crossing. The relation between the drivers’ visual behaviour and the vehicle speed have also been calculated. Results have confirmed that the intervention carried out has increased both visibility and safety of the studied pedestrian crosswalks.Zebra markings and the median refuge island have turned out to be the most glanced elements, respectively seen by 93.75% and 56.25% of the drivers, followed by the “Yield here to pedestrians” vertical sign. The mean distance of first fixation of the crosswalk increased from 21.98 m before the intervention, to 40.69 m after it. The drivers perceived the pedestrian crossings from a longer distance after the intervention, and they continued to glance at the crosswalk while approaching it, enhancing their visual attention.     

Nonverbal communication during the learner lesson with a professional driving instructor: A novel investigation

BackgroundFundamental for the development of the driving and road use skills of the young driver is learning to drive through driving instruction and, in graduated driver licensing programs such as in Australia, driving supervision. In Queensland young drivers are required to log a minimum of 100 h supervised practice, with recent research revealing that parents provide most of this supervision. Queensland also offers young drivers a 10-h 3-for-1 bonus for professional driving instruction, such that one hour of professional instruction can be logged as three hours of practice, to a maximum of 30 logbook hours. Recent research efforts have begun to provide insight into the nature of the verbal instruction of both parents and professional instructors, and into the nonverbal communication between parents and learners. However nothing is known regarding the nonverbal communication between professional instructors and learners.MethodTen learner lessons (five male learners) with four professional instructors (four males) were captured via GoPro cameras. The nonverbal communication during the first, middle, and last 10 min of each lesson was coded as being posture and body orientation, gestures, facial expressions, proximity, humour, and eye contact, within the context of the accompanying verbal communication according to the value of (a) eager, or (b) cautious; the valence of (a) neutral, (b) positive, or (c) negative; and the purpose of (a) rapport, or (b) communication.ResultsOverall, posture and body orientation was the most common mechanism of nonverbal communication, while facial expressions and proximity were the least common mechanisms of nonverbal communication. In general the beginning, the middle, and the end of the lessons were characterised by a plethora of neutral, cautious interactions, and positive, eager interactions. However it is noteworthy that the rates at which learners and instructors engaged in these behaviours were found to change across the lesson. Specifically learners actively communication nonverbally through mechanisms such as eye contact, facial expressions and humour, while instructors appeared to manage building rapport and communicating safe vehicle and road use through nonverbal communication such as gestures, facial expressions and posture and body orientation, summarised in a model comprising a continuum of instruction.DiscussionWhile nonverbal communication is fundamental for effective verbal communication, and on occasion can replace verbal communication, and as such the professional – and the parental – driving lesson should optimise the use of nonverbal communication, at this time the optimal nature of nonverbal communication remains unknown. In addition, optimal verbal and nonverbal communication specifically suited to the driving context which involves a dynamic environment outside the vehicle, and at times a dynamic environment inside the vehicle, remains yet to be identified. The research findings provide unique insight into the nature of the nonverbal communication used by both learner drivers and professional driving instructors, in addition to the continuum of instruction model. As such, the findings provide a solid foundation for future research into, and guidance regarding, optimising the learner driving lesson.     

Where does attention go when you blink?

Many studies have shown that covert visual attention precedes saccadic eye movements to locations in space. The present research investigated whether the allocation of attention is similarly affected by eye blinks. Subjects completed a partial-report task under blink and no-blink conditions. Experiment 1 showed that blinking facilitated report of the bottom row of the stimulus array: Accuracy for the bottom row increased and mislocation errors decreased under blink, as compared with no-blink, conditions, indicating that blinking influenced the allocation of visual attention. Experiment 2 showed that this was true even when subjects were biased to attend elsewhere. These results indicate that attention moves downward before a blink in an involuntary fashion. The eyes also move downward during blinks, so attention may precede blink-induced eye movements just as it precedes saccades and other types of eye movements.