Detecting drivers’ cognitive load from saccadic intrusion

In this paper, we have explored use of Saccadic Intrusion to detect drivers’ cognitive load and instantaneous perception of developing road hazards. Saccadic Intrusion is a type of eye gaze movement which was earlier found to be related to change in cognitive load. We have developed an algorithm to detect saccadic intrusion from a commercially available low-cost eye gaze tracker and conducted four user studies involving a driving simulator and cognitive and hazard perception tests. Our results show that average velocities of saccadic intrusion increases with increase in cognitive load and recording saccadic intrusion and eye blinks for 6 s duration can predict drivers’ instantaneous perception of developing road hazards.     

Effects of directional auditory and visual warnings at intersections on reaction times and speed reduction times

Intersection collision warning systems (ICWSs) have an important impact on driving safety because making the potential collision at intersection predictable, allow reducing the probability and severity of accidents. Among the several types of alarms to alert the driver of an imminent collision, those most used concerning the auditory and the visual stimulus. However, it is unclear whether is more effective an audio or a visual warning. In addition, no study compared the effects on drivers’ behavior induced by an acoustic and a visual directional warning. The main objective of the present study was to assess, in response to a potential conflict event at the intersections, the effects of directional auditory and visual warnings on driving performance.A driving simulator experiment was carried out to collect drivers’ behavior in response to a vehicle that failed to stop at the intersection. The parameters reaction time and speed reduction time were used for the evaluation of the effects on driving performance. These duration variables were modeled following the survival analysis, by the use of the accelerated failure time duration model with a Weibull distribution.Results showed that when the directional warning system (auditory or visual) was present, the drivers were able to detect earlier the violator vehicle. This effect led to a more comfortable braking maneuver and, thus, less possibilities of an unexpected maneuver for the following vehicle, avoiding the car – following collisions. The effectiveness of ICWSs was more evident for the directional auditory speech message; for this condition, in fact, the lower reaction time and the longer speed reduction time were obtained.The outcomes of the present study provide useful suggestions about the most effective collision warning systems that the automotive industry should develop and equip on vehicles.     

Obstacle avoidance under automated steering: Impact on driving and gaze behaviours

Within the context of more and more autonomous vehicles, an automatic lateral control device (AS: Automatic Steering) was used to steer the vehicle along the road without drivers’ intervention. The device was not able to detect and avoid obstacles. The experiment aimed to analyse unexpected obstacle avoidance manoeuvres when lateral control was delegated to automation. It was hypothesized that drivers skirting behaviours and eye movement patterns would be modified with automated steering compared with a control situation without automation. Eighteen participants took part in a driving simulator study. Steering behaviours and eye movements were analysed during obstacle avoidance episodes. Compared with driving without automation, skirting around obstacles was found to be less effective when drivers had to return from automatic steering to manual control. Eye movements were modified in the presence of automatic steering, revealing further ahead visual scanning of the driving environment. Resuming manual control is not only a problem of action performance but is also related to the reorganisation of drivers’ visual strategies linked to drivers’ disengagement from the steering task. Assistance designers should pay particular attention to potential changes in drivers’ activity when carrying out development work on highly automated vehicles.     

On-to-off-path gaze shift cancellations lead to gaze concentration in cognitively loaded car drivers: A simulator study exploring gaze patterns in relation to a cognitive task and the traffic environment

Appropriate visual behaviour is necessary for safe driving. Many previous studies have found that when performing non-visual cognitive tasks, drivers typically display an increased amount of on-path glances, along with a deteriorated visual scanning pattern towards potential hazards at locations outside their future travel path (off-path locations). This is often referred to as a gaze concentration effect. However, what has not been explored is more precisely how and when gaze concentration arises in relation to the cognitive task, and to what extent the timing of glances towards traffic-situation relevant off-path locations is affected. To investigate these specific topics, a driving simulator study was carried out. Car drivers’ visual behaviour during execution of a cognitive task (n-back) was studied during two traffic scenarios; one when driving through an intersection and one when passing a hidden exit. Aside from the expected gaze concentration effect, several novel findings that may explain this effect were observed. It was found that gaze shifts from an on-path to an off-path location were inhibited during increased cognitive load. However, gaze shifts in the other direction, that is, from an off-path to an on-path location, remained unaffected. This resulted in on-path glances increasing in duration, while off-path glances decreased in number. Furthermore, the inhibited off-path glances were typically not compensated for later. That is, off-path glances were cancelled, not delayed. This was the case both in relation to the cognitive task (near-term) and the traffic environment (far-term). There was thus a general reduction in the number of glances towards situationally relevant off-path locations, but the timing of the remaining glances was unaffected. These findings provide a deeper understanding of the mechanism behind gaze concentration and can contribute to both understanding and prediction of safety relevant effects of cognitive load in car drivers.     

What attracts attention during police pursuit driving?

Efficient deployment of attention is important to the safe execution of tasks with a high content of visual information, such as driving. Chasing a lead vehicle is an extremely demanding and dangerous task, though little is known of the visual skills required. A study is reported that recorded the eye movements of police drivers and two control groups (novices and age‐ and experienced‐ matched controls) while watching a series of video clips of driving. The clips included pursuits, emergency response drives, and control drives (at normal speeds) around Nottinghamshire, UK. Analysis of gaze durations within certain categories of stimuli revealed that daytime pursuit drives correspond with an increase in gaze durations on a lead car (controlled for exposure), though police drivers direct their attention to other sources of potential hazards, such as pedestrians, more so than other drivers. Copyright © 2005 John Wiley & Sons, Ltd.     

Gaze behavior when approaching an intersection: Dwell time distribution and comparison with a quantitative prediction

The allocation of overt visual attention is investigated in a multi-task and dynamical situation: driving. The Expectancy–Value model of attention allocation stipulates that visual exploration depends on the expectancy and the value of the task-related information available in each Area Of Interest (AOI). We consider the approach to an intersection as a multi-task situation where two subtasks are involved: vehicle control and interactions with other drivers. Each of these subtasks is associated with some specific visual information present in the associated AOIs: the driver’s lane and the intersecting road at the intersection. An experiment was conducted in a driving simulator, coupled with a head-mounted eye-tracker. The intersecting road’s AOI’s Expectancy was manipulated with the traffic density, and its Value was manipulated with the priority rule before the intersection (stop, yield, and priority). The distribution of visual attention and the dynamics of visual exploration were analyzed on 20 participants, taking into account the dwell time in the AOIs associated to the driving subtasks, and the gaze transitions between the AOIs. The results suggest that visual attention to intersecting roads varied with the priority rule, and impacted the visual attention associated with the vehicle control subtask. In addition, a quantitative model was used to improve the understanding of the Expectancy and Value factors. The comparison of the data with the model’s predictions enables quantifying the observed differences between the experimental factors. Finally, the results associated with the traffic density are discussed in relation to the nature of the relevant information while approaching the intersection.     

Approaching intersections: Gaze behavior of drivers depending on traffic,intersection type,driving maneuver,and secondary task involvement

Urban intersections are hotspots for crashes because they provide a location for several traffic streams and types of road users to cross. A main cause of crashes is the misinformation of drivers as they fail to sense relevant visual information. We aimed to analyze the gaze behavior of car drivers in a variety of intersection scenarios, bringing together the partial findings of previous research, and examine the interdependencies of the contributing factors to provide a database for driver modeling. In a driving simulator study with 59 participants, we varied intersection scenarios regarding drivers’ right of way (yield sign, green traffic light), intersection type (T junction, X intersection), surrounding traffic (none, irrelevant, relevant), and intended driving maneuver (left turn, right turn, going straight). A total of 25 intersection scenarios were presented in a within-subjects design to a control group and a group with a cognitive load task (counting back in numbers of two). Fixations were coded regarding defined areas of interest in the field of view and separated according to three segments of the intersection approach: 75–50 m, 50–25 m, and 25–0 m before entering the intersection. The results show that the effect of surrounding traffic, secondary task engagement, and the intended driving maneuver changed dramatically depending on the right of way of the driver. Surrounding traffic primarily affected gaze behavior in scenarios of ceding the right of way close to the intersection entry. The cognitive load task increased fixations on the road center especially in situations where the driver had the right of way, but less in situations of ceding the right of way. Interactions with the type of intersection were only apparent for different driving maneuvers. This study provides a detailed and comprehensive picture of drivers’ attentional processes when approaching intersections which is relevant for understanding and modeling of driver behavior in urban traffic.     

Aging and the use of an in-vehicle intersection crossing assist system: An on-road study

One of the principal facets of age-related decline–diminished perceptual ability, can also be viewed as a prominent factor when crossing intersections, particularly rural intersections that have disproportionately high fatality rate and where vehicles travel at higher velocities. Providing information through in-vehicle technology may aid drivers in improving crossing of such intersections. The current study examines the efficacy of an in-vehicle intersection crossing assist system in a real-world rural setting across age groups. Thirty-two, older and younger drivers completed several crossings of a busy rural intersection. Drivers completed two blocks of trials in which the presence/absence of the in-vehicle system was counterbalanced. The results showed a limited impact of the system on driving performance, exhibited in longer wait time before crossing and rising trend towards reduced probability of accepting small crossing gaps. Older drivers performed similarly to younger, although they showed a greater tendency towards conservative driving behaviour. The current study represents an initial effort to examine an in-vehicle intersection crossing assist system in a real-world rural environment, generating results that reveal a potential for these types of systems to be assistive to drivers across age groups and increase the safety at rural intersections.     

Eye movements and hazard perception in police pursuit and emergency response driving

How do police cope with the visual demands placed on them during pursuit driving? This study compared the hazard ratings, eye movements, and physiological responses of police drivers with novice and with age-matched control drivers while viewing video clips of driving taken from police vehicles. The clips included pursuits, emergency responses, and control drives. Although police drivers did not report more hazards than the other participants reported, they had an increased frequency of electrodermal responses while viewing dangerous clips and a greater visual sampling rate and spread of search. However, despite an overall police advantage in oculomotor and physiological measures, all drivers had a reduced spread of search in nighttime pursuits because of the focusing of overt attention.     

“Please watch right” – Evaluation of a speech-based on-demand assistance system for urban intersections

In a driving simulator study we evaluated a speech-based driver assistance system for urban intersections (called Assistance on Demand AoD system) which supports the driver in monitoring and decision making. The system provides recommendations for suitable time gaps to enter the intersection based on the observation of crossing traffic. Following an “on-demand”-concept, the driver activates the assistance only if support is desired.In one drive, drivers used the AoD system in every situation they experienced to guarantee that every driver had the same exposure to the system when evaluating it. During another drive, drivers were free to decide if they want to use the system or not. The experimental study compared the AoD system with driving manually and with driving supported by a more conventional visual-based system which was always active at intersections (system showing colored arrows in a simulated head-up display (HUD) to visualize the crossing traffic). This resulted in four drives the drivers had to perform. Every drive consisted of several intersections with varying traffic conditions. The drivers had to turn left at every intersection.A total of 24 drivers took part in the study; one group with 14 middle-aged drivers and another group with ten high-aged drivers. Several questionnaires and online ratings were used to assess drivers’ acceptance, perceived usefulness, benefits and specific characteristics of both system variants. In addition, driving behaviour with regard to gap choice and drivers’ monitoring behaviour (using head tracking data) were analyzed.The results show that the AoD system reaches high acceptance ratings and is preferred compared to the visual, always active system. Using the speech modality for communication and the on-demand concept were both highly appreciated by the drivers. With regard to driving behaviour, the AoD system is comparably safe as manual driving while at the same time making driving easier by facilitating the monitoring of vehicles while waiting at an intersection.     

Influence of traffic conditions on driver behavior before making a right turn at an intersection: Analysis of driver behavior based on measured data on an actual road

This paper describes an investigation of the influence of the position of a forward vehicle and following vehicle on the onset of driver preparatory behavior before making a right turn at an intersection. Four experimental vehicles with various sensors and a driving recorder system were developed to measure driver behavior before making a right turn at a specific intersection on a public road. The experimental term was eight weeks to collect data on natural driving maneuvers. The relationships between the remaining distances to the center of the intersection when releasing the accelerator pedal, moving the right foot to cover the brake pedal, and activating the turn signal and the relative distances from the forward and following vehicles were analyzed based on the measured data. The time it took to reach the center of the turn and the driving speed when each behavioral event occurred were also evaluated from the viewpoint of the relative position between the driver’s vehicle and the leading or following vehicles. The results suggest that the drivers approached the target intersection in a car-following condition, and that the positions of the front and rear vehicles and the vehicle velocity influence the onset location and timing of releasing the accelerator pedal and covering the brake pedal. Drivers began to decelerate closer to the center of the intersection when they approached the intersection close to a leading or following vehicle at a reduced driving speed. However, these influences were not reflected in the turn signal operation, indicating that drivers intend to make a right turn at a constant location while approaching a target intersection and that intention appears in the turn signal activation. The findings of this observational study imply that the method of providing route guidance instruction, in which the traffic conditions surrounding the driver’s vehicle are taken into consideration, is effective in reducing driver errors in receiving instruction and following the correct route. The results also indicate that measuring and accumulating different behavioral indices based on traffic conditions contribute to determining the criteria for the presentation timing just before reaching the intersection, which can assist drivers in preparing to make a right turn at a usual location. Driver decelerating maneuvers are used while driving without leading or following vehicles and while driving with a lead and/or following vehicle at long range, and driver turn signal operations are used when approaching an intersection under close car-following conditions.     

Attention during visual secondary tasks in driving: Adaptation to the demands of the driving task

Visual attention in driving with visual secondary task is compared for two visual secondary tasks. N = 40 subjects completed a 1 h test drive in a motion-base driving simulator. During the drive, participants either solved an externally paced, highly demanding visual task or a self paced menu system task. The secondary tasks were offered in defined critical and non-critical driving situations. Eye movement behavior was analyzed and compared for both tasks. Before starting the secondary tasks, eye movement behavior shows a smaller standard deviation of gaze as well as longer fixation durations for both tasks. The comparison between the two tasks indicates that drivers use the possibilities the self paced task offers: during the secondary task, they monitor the driving scene with longer fixations and show a greater standard deviation of gaze position. Furthermore, independently of the type of secondary task, drivers adapt their eye movement behavior to the demands of the situation. In critical driving situations they direct a larger proportion of glance time to the driving task. Last, the relation between glance behavior and collisions is analyzed. Results indicate that collisions go together with an inadequate distribution of attention during distraction. The results are interpreted regarding the attentional processes involved in driving with visual secondary tasks. Based on the similarities and differences between the two secondary tasks, a cognitive approach is developed which assumes that the control of attention during distraction is based on a mental situational model of the driving situation.     

Visual awareness and the on-line modification of action.

An influential theory of visually guided action proposes that (a) conscious perception of target displacement disrupts on-line action and (b) small target perturbations are inconsequential, provided the participant is unaware of them. This study examined these claims in a study of rapid aiming movements to targets. Novel features included on-line verbal reports of target displacement, and the factorial combination of small versus large displacements occurring near peak saccadic velocity or 100 ms later. Although awareness of target displacement had no effect on movement kinematics, even small target displacements near peak saccadic velocity affected kinematic measures. These results support both a strong view of visual stream separation in the on-line control of action and richer spatial coding by unconscious processes than has previously been acknowledged.     

A predictive coding model of gaze shifts and the underlying neurophysiology

A comprehensive model of gaze control must account for a number of empirical observations at both the behavioural and neurophysiological levels. The computational model presented in this article can simulate the coordinated movements of the eye, head, and body required to perform horizontal gaze shifts. In doing so it reproduces the predictable relationships between the movements performed by these different degrees of freedom (DOFs) in the primate. The model also accounts for the saccadic undershoot that accompanies large gaze shifts in the biological visual system. It can also account for our perception of a stable external world despite frequent gaze shifts and the ability to perform accurate memory-guided and double-step saccades. The proposed model also simulates peri-saccadic compression: the mis-localization of a briefly presented visual stimulus towards the location that is the target for a saccade. At the neurophysiological level, the proposed model is consistent with the existence of cortical neurons tuned to the retinal, head-centred, body-centred, and world-centred locations of visual stimuli and cortical neurons that have gain-modulated responses to visual stimuli. Finally, the model also successfully accounts for peri-saccadic receptive field (RF) remapping which results in reduced responses to stimuli in the current RF location and an increased sensitivity to stimuli appearing at the location that will be occupied by the RF after the saccade. The proposed model thus offers a unified explanation for this seemingly diverse range of phenomena. Furthermore, as the proposed model is an implementation of the predictive coding theory, it offers a single computational explanation for these phenomena and relates gaze shifts to a wider framework for understanding cortical function.     

Coordinated flexibility: how initial gaze position modulates eye-hand coordination and reaching

Reaching to targets in space requires the coordination of eye and hand movements. In two experiments, we recorded eye and hand kinematics to examine the role of gaze position at target onset on eye-hand coordination and reaching performance. Experiment 1 showed that with eyes and hand aligned on the same peripheral start location, time lags between eye and hand onsets were small and initiation times were substantially correlated, suggesting simultaneous control and tight eye-hand coupling. With eyes and hand departing from different start locations (gaze aligned with the center of the range of possible target positions), time lags between eye and hand onsets were large and initiation times were largely uncorrelated, suggesting independent control and decoupling of eye and hand movements. Furthermore, initial gaze position strongly mediated manual reaching performance indexed by increments in movement time as a function of target distance. Experiment 2 confirmed the impact of target foveation in modulating the effect of target distance on movement time. Our findings reveal the operation of an overarching, flexible neural control system that tunes the operation and cooperation of saccadic and manual control systems depending on where the eyes look at target onset.     

Older drivers’ visual search behaviour at intersections

Previous research has indicated that older drivers are more likely to be involved in collisions in complex traffic scenarios like intersections even if they are not more involved in accidents in general. Moreover, being more vulnerable, the older driver is generally at higher risk of sustaining an injury when involved in a traffic accident. Even though there may be many factors leading to the over-involvement of older drivers in intersection collisions it is clear that the visual capacity and the ability to observe may be one of the possible causes that is of high interest to understand further. The objective of the study is to identify to what degree the visual behaviour could explain older drivers’ involvement in intersection accidents. A 20 km long route composed by intersections in rural and urban environment was selected to collect both driving and eye movement data. Two groups of drivers were compared, one group aged 35–55 years and one aged 75 and above. Apart from the driving data, neck flexibility measurement was performed. The results from the neck flexibility measurement showed a clear age effect, with the older drivers showing less neck flexibility. When it comes to visual behaviour data, a difference was also found concerning the area of interest the drivers looked at; while the older drivers looked more at lines and markings on the road to position themselves in the traffic, the younger drivers looked more at dynamic objects such as other cars representing a possible threat. The difference in the visual behaviour should be used to design safety systems for all drivers to support them when they drive through an intersection.     

Regarding Scenes

ABSTRACT— When we view the visual world, our eyes flit from one location to another about three times each second. These frequent changes in gaze direction result from very fast saccadic eye movements. Useful visual information is acquired only during fixations, periods of relative gaze stability. Gaze control is defined as the process of directing fixation through a scene in real time in the service of ongoing perceptual, cognitive, and behavioral activity. This article discusses current approaches and new empirical findings that are allowing investigators to unravel how human gaze control operates during active real-world scene perception.     

Detecting eye movements in dynamic environments

To take advantage of the increasing number of in-vehicle devices, automobile drivers must divide their attention between primary (driving) and secondary (operating in-vehicle device) tasks. In dynamic environments such as driving, however, it is not easy to identify and quantify how a driver focuses on the various tasks he/she is simultaneously engaged in, including the distracting tasks. Measures derived from the driver’s scan path have been used as correlates of driver attention. This article presents a methodology for analyzing eye positions, which are discrete samples of a subject’s scan path, in order to categorize driver eye movements. Previous methods of analyzing eye positions recorded in a dynamic environment have relied completely on the manual identification of the focus of visual attention from a point of regard superimposed on a video of a recorded scene, failing to utilize information regarding movement structure in the raw recorded eye positions. Although effective, these methods are too time consuming to be easily used when the large data sets that would be required to identify subtle differences between drivers, under different road conditions, and with different levels of distraction are processed. The aim of the methods presented in this article are to extend the degree of automation in the processing of eye movement data by proposing a methodology for eye movement analysis that extends automated fixation identification to include smooth and saccadic movements. By identifying eye movements in the recorded eye positions, a method of reducing the analysis of scene video to a finite search space is presented. The implementation of a software tool for the eye movement analysis is described, including an example from an on-road test-driving sample.     

Analyzing driver's response to the yellow onset at signalized intersections

Intersections are critical points within the highway system at which the risk of crashes increases. This study seeks to better understand drivers’ behavior at an intersection by examining the relationship between their observed driving behavior, psychological attributes and decision to proceed through an intersection. A driving simulator and self-report questionnaire were used to understand driver decision-making at the onset of the yellow phase across several signalized intersections. The simulator measured driving outcomes such as speed, braking, and throttle as drivers cross through four increasingly difficult intersections. The questionnaires measured demographics, psychological traits including mindfulness and impulsiveness along with self-reported driving behaviors. A total of 102 participants completed the questionnaire as well as the driving simulator experiment. Hierarchical clustering served to classify drivers into four groups on the basis of their observed driving in the simulator: the safest drivers, safe drivers, speed demons, and aggressive drivers. These driving styles moderated the relationship between the drivers’ psychological traits and their decision to stop or proceed at each intersection. Results showed that mindfulness was highly related to the safest drivers’ decision to stop at the first intersection, while impulsiveness and anxiety were related to the speed demons’ decision to stop at the third intersection. These findings lay a strong foundation for developing progressive educational campaigns incorporating driver psychology in their methodology. Findings also provide support for research linking driving performance and psychological traits with implications for intersection design.     

Control of a virtual vehicle influences postural activity and motion sickness

Everyday experience suggests that drivers are less susceptible to motion sickness than passengers. In the context of inertial motion (i.e., physical displacement), this effect has been confirmed in laboratory research using whole body motion devices. We asked whether a similar effect would occur in the context of simulated vehicles in a visual virtual environment. We used a yoked control design in which one member of each pair of participants played a driving video game (i.e., drove a virtual automobile). A recording of that performance was viewed (in a separate session) by the other member of the pair. Thus, the two members of each pair were exposed to identical visual motion stimuli, but the risk of behavioral contagion was minimized. Participants who drove the virtual vehicle (drivers) were less likely to report motion sickness than participants who viewed game recordings (passengers). Data on head and torso movement revealed that drivers tended to move more than passengers, and that the movements of drivers were more predictable than the movements of passengers. Before the onset of subjective symptoms of motion sickness movement differed between participants who (later) reported motion sickness and those who did not, consistent with a prediction of the postural instability theory of motion sickness. The results confirm that control is an important factor in the etiology of motion sickness and extend this finding to the control of noninertial virtual vehicles.