Effects of secondary tasks and display position on glance behavior during partially automated driving

The driving task is becoming increasingly automated, thus changing the driver’s role. Moreover, in-vehicle information systems using different display positions and information processing channels might encourage secondary task engagement. During manual driving scenarios, varying secondary tasks and display positions could influence driver’s glance behavior. However, their impact on the driver’s capability to monitor the partially automated driving system has not yet been determined. The current study assessed both the effects of different secondary tasks (Surrogate Reference Task (SuRT) vs. text reading) and display positions (head-up display (HUD) vs. center console) on driver’s glance behavior during partially automated driving in a simulated car following task. Different automation system failures regarding the lateral and longitudinal control occurred while driving. Furthermore, participants’ reported advantages, disadvantages and preferences regarding the investigated display positions as well as regarding the secondary task engagement during partially automated driving in general. Mixed design ANOVAs revealed that the HUD yielded considerably longer eyes-on display time (total and mean glance durations) than the center console. Moreover, the text reading task resulted in longer total and mean glance durations than the SuRT. Similar to manual driving scenarios, the results showed a consistent effect of display position and secondary task on the driver’s glance behavior. Despite the longer eyes-on display time for the HUD, its proximity to the driving environment might enable a faster identification of and reaction to critical situations (e.g., due to system failures). Participants would prefer the HUD as display position compared to the center console. Regarding secondary task engagement during partially automated driving participants seemed to be aware of the benefits but also of the risks.     

Evaluation of a dynamic blocking concept to mitigate driver distraction: Three simulator studies

In recent years, the number and complexity of in-vehicle infotainment systems has been steadily increasing. While these systems certainly improve the driving experience, they also increase the risk for driver distraction. International standards and guidelines provide methods of measuring this distraction along with test criteria that help automakers decide whether an interface task is too distracting to be used while driving. Any specific function failing this test should therefore be locked out for use by the driver. This study implemented and tested a dynamic approach to this blocking by algorithmically reacting to driver inputs and the pace of the interaction in order to prevent drivers from having prolonged or too intense sequences of in-vehicle interactions not directly related to driving. Three simulated driving experiments in Germany and the United States were conducted to evaluate this dynamic function blocking concept and also cater for differences in the status quo of either no blocking or static blocking. The experiments consisted of a car following scenario with various secondary interface tasks and always included a baseline condition where no blocking occurred as well as an implementation of the dynamic function blocking. While Experiments 1 and 3 were aimed at collecting and analyzing gaze and driving data from more than 20 participants, Experiment 2 focused on the user experience evaluation of different visual feedback implementations from 13 participants. The user experience as rated by these participants increased throughout the course of all three studies and helped further improve both the concept and feedback design. In the experiments the total glance time towards the road was significantly higher in the dynamic function blocking condition compared to the baseline, already accounting for the increase in total task time inherent to the dynamic condition. Participants developed two strategies of interacting with the dynamic function blocking. They either operated at their normal baseline speed and incurred task blockings or operated slower to avoid the blockings. In the latter strategy, participants chunked their interactions into smaller steps with the present data suggesting that they used the pauses in between chunks to look back onto the road ahead. Theoretical and practical implications of this first evaluation of a dynamic function blocking concept are discussed.     

Refining distraction potential testing guidelines by considering differences in glancing behavior

Driver distraction is a recognized cause of traffic accidents. Although the well-known guidelines for measuring distraction of secondary in-car tasks were published by the United States National Highway Traffic Safety Administration (NHTSA) in 2013, studies have raised concerns on the accuracy of the method defined in the guidelines, namely criticizing them for basing the diversity of the driver sample on driver age, and for inconsistent between-group results. In fact, it was recently discovered that the NHTSA driving simulator test is susceptible to rather fortuitous results when the participant sample is randomized. This suggests that the results of said test are highly dependent on the selected participants, rather than on the phenomenon being studied, for example, the effects of touch screen size on driver distraction. As an attempt to refine the current guidelines, we set out to study whether a previously proposed new testing method is as susceptible to the effects of participant randomization as the NHTSA method. This new testing method differs from the NHTSA method by two major accounts. First, the new method considers occlusion distance (i.e., how far a driver can drive with their vision covered) rather than age, and second, the new method considers driving in a more complex, and arguably, a more realistic environment than proposed in the NHTSA guidelines. Our results imply that the new method is less susceptible to sample randomization, and that occlusion distance appears a more robust criterion for driver sampling than merely driver age. Our results are applicable in further developing driver distraction guidelines and provide empirical evidence on the effect of individual differences in drivers’ glancing behavior.     

The impact of an in-vehicle display on glance distribution in partially automated driving in an on-road experiment

One of the major challenges of designing an HMI for partially automated vehicles is the trade-off between a sufficient level of system information and avoidance of distracting the driver. This study aimed to investigate drivers’ glance behavior as an indicator of distraction when vehicle guidance is partially automated. Therefore, an on-road experiment was conducted comparing two versions of an in-vehicle display (during partially automated driving) and no display (during manual driving) on a heavy congested highway segment. The distribution of drivers’ total glance durations on the HMI showed that visual attention was shifted away from monitoring the central road scene towards looking at the in-vehicle display to a considerable extent. However, an analysis of the distribution of single glance durations supports the view that using partial automation and a respective HMI does not lead to a critical increase in distraction. Driving with a simplified version of the HMI had the potential to reduce glance duration on and thus potential distraction of the in-vehicle display.     

Driver glance behavior towards displayed images on in-vehicle information systems under real driving conditions

Intelligent vehicle technologies like driver assistance systems and in-vehicle information systems, enhance convenience of the driving experience for drivers and passengers. At the same time, these systems may increase driver distraction and workload. Guidelines developed for this purpose include principles, methods, and assessments which are widely agreed upon, with some being singled out for a particular recommendation or requirement. Especially the display of graphical or photographic images are generally assumed to distract the driver from safely operating the vehicle and should be blocked during driving under all circumstances (so called per se lock outs). This study investigates the effect of displaying graphical and photographical images during driving on driveŕs glance behavior during real-world driving. Findings presented in this paper provide empirical evidence for the unobtrusiveness of these stimuli: Participants didn’t exhibit longer glance durations towards in-vehicle information systems, nor a deterioration of driver distraction parameters such as total eyes off road time and long glance proportion when being compared to driving without displaying any photographic images.     

The power and sensitivity of four core driver workload measures for benchmarking the distraction potential of new driver vehicle interfaces

This study evaluated the power and sensitivity of several core driver workload measures in order to better understand their use as a component of future driver distraction potential evaluation procedures of the in-vehicle human machine interface (HMI). Driving is a task that requires visual, manual and cognitive resources to perform. Secondary tasks, such as mobile phone use and interaction with in-built navigation, which load onto any of these three processing resources increase driver workload and can lead to impaired driving. Because workload and distraction potential are interrelated, a comprehensive method to assess driver workload that produces valid and predictive results is needed to advance the science of distraction potential evaluation. It is also needed to incorporate into New Car Assessment Program (NCAP) testing regimes. Workload measures of cognitive (DRT [Detection Response Task] Reaction Time), visual (DRT Miss Rate), subjective (NASA-TLX [driver workload questionnaire]), and temporal demand (Task Interaction Time) were collected as participants drove one of 40 vehicles while completing a variety of secondary tasks with varying interaction requirements. Of the evaluated measures, variance and power analyses demonstrated that Task Interaction Time is the most sensitive in detecting differences in driver workload between different in-vehicle HMIs, followed by DRT Miss Rate, NASA-TLX and finally DRT Reaction Time. There were relatively weak correlations between each of the four measures. These results suggest that Task Interaction Time, coupled with a reliable visual demand metric such as DRT Miss Rate, eye glance coding, or visual occlusion, more efficiently detect differences in driver workload between different HMIs compared to DRT Reaction Time and the NASA-TLX questionnaire. These results can be used to improve the understanding of the utility of each of these core driver workload measures in assessing driver distraction potential.     

Human factors evaluation of an in-vehicle active traffic and demand management (ATDM) system

This research study focused on the evaluation of an emulated in-vehicle Active Traffic and Demand Management (ATDM) system on Interstate 66 in Northern Virginia. Traditional ATDM systems rely on infrastructure-mounted variable message signs (VMS) to provide information (speed limits, lane availability, etc.) to the traveling public. By providing information about dynamic roadway conditions on an in-vehicle device, the ATDM may improve driving safety and performance by allowing drivers to remain consistently aware of forthcoming traffic conditions and roadway requirements; even when external signage is inaccessible. This study represents an initial investigation of an emulated in-vehicle ATDM to provide developers with design guidance and ensure that unintended consequences, such as distraction, do not undermine the potential benefits.Twenty younger and 20 older participants, accompanied by a member of the research team, experienced the following ATDM features on an in-vehicle device (IVD) mounted to the dashboard: (1) dynamic speed limits, (2) dynamic lane use/shoulder control, (3) High Occupancy Vehicle (HOV) restrictions, and (4) roadway information from variable message signs. The IVD was equipped with auditory and visual alerts notifying the driver when relevant visual information was updated. Research questions addressed distraction and driver behavior associated with use of the system. Qualitative and quantitative participant data was acquired from the instrumented vehicle, various questionnaires, and researcher observation.Several key findings were uncovered: (1) The IVD, as tested, did not warrant classification as a source of distraction according to the NHTSA guidelines; v2) There was a significant difference in eye-glance durations to the IVD when comparing the VMS alerts to both the speed limit and lane management alerts; and (3) The speed limit alert motivated participants to alter their speed (per survey results and participant speed data).     

Measurement of auditory cues in drivers' distraction

A driving simulator was used to examine the effects on driving performance of auditory cues in an in-vehicle information search task. Drivers' distraction by the search tasks was measured on a peripheral detection task. The difficulty of the search task was systematically varied to test the distraction caused by a quantified visual load. 58 participants completed the task. Performance on both search tasks and peripheral detection tasks was measured by mean response time and percent error. Analyses indicated that in-vehicle information search performance can be severely degraded when a target is located within a group of diverse distractors. Inclusion of an auditory cue in the visual search increased the mean response time as a result of a change in modality from auditory to visual. Inclusion of such an auditory cue seemed to influence distraction as measured by performance on the peripheral detection task; accuracy was lower when auditory cues were provided, and responses were slower when no auditory cues were provided. Distraction by the auditory cue varied according to the difficulty of the search task.     

How does glance behavior influence crash and injury risk? A ‘what-if’ counterfactual simulation using crashes and near-crashes from SHRP2

As naturalistic driving data become increasingly available, new analyses are revealing the significance of drivers’ glance behavior in traffic crashes. Due to the rarity of crashes, even in the largest naturalistic datasets, near-crashes are often included in the analyses and used as surrogates for crashes. However, to date we lack a method to assess the extent to which driver glance behavior influences crash and injury risk across both crashes and near-crashes. This paper presents a novel method for estimating crash and injury risk from off-road glance behavior for crashes and near-crashes alike; this method can also be used to evaluate the safety impact of secondary tasks (such as tuning the radio). We apply a ‘what-if’ (counterfactual) simulation to 37 lead-vehicle crashes and 186 lead-vehicle near-crashes from lead-vehicle scenarios identified in the SHRP2 naturalistic driving data. The simulation combines the kinematics of the two conflicting vehicles with a model of driver glance behavior to estimate two probabilities: (1) that each event becomes a crash, and (2) that each event causes a specific level of injury. The usefulness of the method is demonstrated by comparing the crash and injury risk of normal driving with the risks of driving while performing one of three secondary tasks: the Rockwell radio-tuning task and two hypothetical tasks. Alternative applications of the method and its metrics are also discussed. The method presented in this paper can guide the design of safer driver–vehicle interfaces by showing the best tradeoff between the percent of glances that are on-road, the distribution of off-road glances, and the total task time for different tasks.     

A multitasking general executive for compound continuous tasks

As cognitive architectures move to account for increasingly complex real-world tasks, one of the most pressing challenges involves understanding and modeling human multitasking. Although a number of existing models now perform multitasking in real-world scenarios, these models typically employ customized executives that schedule tasks for the particular domain but do not generalize easily to other domains. This article outlines a general executive for the Adaptive Control of Thought–Rational (ACT–R) cognitive architecture that, given independent models of individual tasks, schedules and interleaves the models' behavior into integrated multitasking behavior. To demonstrate the power of the proposed approach, the article describes an application to the domain of driving, showing how the general executive can interleave component subtasks of the driving task (namely, control and monitoring) and interleave driving with in-vehicle secondary tasks (radio tuning and phone dialing).     

Effects of verbal and spatial-imagery tasks on eye fixations while driving

The consequences of performing verbal and spatial-imagery tasks on visual search when driving were studied. Twelve participants drove 84 km on 2 highways and 2 roads. On each route, they performed 2 verbal tasks and 2 spatial-imagery tasks while their eye movements were recorded. The same results were repeated on all routes. Pupillary dilation indicated similar effort for each task. Visual functional-field size decreased horizontally and vertically, particularly for spatial-imagery tasks. Compared with ordinary driving, fixations were longer during the spatial-imagery task. With regard to driving performance, glance frequency at mirrors and speedometer decreased during the spatial-imagery task. Results are interpreted in terms of multiple attention-resource theories; implications of internal distractions on road safety are discussed in terms of possible impairment in relevant information processing.     

Attention during visual secondary tasks in driving: Adaptation to the demands of the driving task

Visual attention in driving with visual secondary task is compared for two visual secondary tasks. N = 40 subjects completed a 1 h test drive in a motion-base driving simulator. During the drive, participants either solved an externally paced, highly demanding visual task or a self paced menu system task. The secondary tasks were offered in defined critical and non-critical driving situations. Eye movement behavior was analyzed and compared for both tasks. Before starting the secondary tasks, eye movement behavior shows a smaller standard deviation of gaze as well as longer fixation durations for both tasks. The comparison between the two tasks indicates that drivers use the possibilities the self paced task offers: during the secondary task, they monitor the driving scene with longer fixations and show a greater standard deviation of gaze position. Furthermore, independently of the type of secondary task, drivers adapt their eye movement behavior to the demands of the situation. In critical driving situations they direct a larger proportion of glance time to the driving task. Last, the relation between glance behavior and collisions is analyzed. Results indicate that collisions go together with an inadequate distribution of attention during distraction. The results are interpreted regarding the attentional processes involved in driving with visual secondary tasks. Based on the similarities and differences between the two secondary tasks, a cognitive approach is developed which assumes that the control of attention during distraction is based on a mental situational model of the driving situation.     

The effect of the dominance of an in-vehicle agent’s voice on driver situation awareness,emotion regulation,and trust: A simulated lab study of manual and automated driving

As in-vehicle voice agents increase in popularity, related research is extending to how voice messages can affect the driver’s cognitive and emotional states. Accordingly, we investigated how in-vehicle agent (IVA) voice dominance and driving automation can affect the driver’s situation awareness (SA), emotion regulation (ER), and trust. To this end, a lab-based experiment was conducted with a medium-fidelity driving simulator using actor-recorded voice agents. Twenty-two female and nineteen male driver-licensed participants were recruited to drive simulated vehicles with voice agents and evaluated. The results demonstrated that compared with the dominant voice, the agent with a submissive voice significantly increased ER in both manual and automated driving. Furthermore, the submissive voice significantly increased trust in automated driving compared with the dominant agent. Cross and synergistic interaction effects exist between voice dominance and driving automation in SA and ER, respectively. This study revealed that both the content of the messages of the IVAs and their voice characteristics are essential for modulating the driver’s SA, ER, and trust in driving. It is expected that larger-scale future studies with simulation or on a real road would increase the validity of this study.     

Characterising mobile phone calls while driving on limited-access roads based on SHRP 2 naturalistic driving data

Research has found that mobile phone call engagement while driving negatively affects driving performance. However, no studies exist characterising hand-held mobile phone calls while driving under naturalistic conditions that include aspects such as the duration of mobile phone subtasks and glance behaviour. Identifying the particularly distracting phases of hand-held telephoning and the nature of influencing factors are the basis for developing design recommendations (e.g. for an in-vehicle information system). Potential influencing factors on driving performance such as call type, mobile phone storage location, or any self-regulatory behaviour need to be taken into account. The present study aimed to draw a comprehensive picture of making hand-held mobile phone calls while driving on limited-access roads using SHRP 2 naturalistic driving data. Mobile phone phase duration, glance behaviour, call type, and mobile phone storage location were coded manually across 98 events. The results show that the handling phase of a mobile phone call (e.g. searching for contacts, dialling) was the most dangerous due to longer mean off- than on-road glances. Outgoing calls caused longer mean off-road glances than incoming; nevertheless, the 2 s critical threshold was not exceeded. A significant influence of mobile phone storage location on glance behaviour did not exist. Moreover, at least in free-flow driving conditions, drivers had enough spare capacity to conduct a mobile phone call without reducing vehicle speed. The results suggest that in low complexity traffic situations drivers can compensate for the increased driving task demand due to telephoning by making minor changes in glance behaviour.     

Cell-Phone–Induced Driver Distraction

ABSTRACT— Our research examined the effects of hands-free cell-phone conversations on simulated driving. We found that even when participants looked directly at objects in the driving environment, they were less likely to create a durable memory of those objects if they were conversing on a cell phone. This pattern was obtained for objects of both high and low relevance, suggesting that very little semantic analysis of the objects occurs outside the restricted focus of attention. Moreover, in-vehicle conversations do not interfere with driving as much as cell-phone conversations do, because drivers are better able to synchronize the processing demands of driving with in-vehicle conversations than with cell-phone conversations. Together, these data support an inattention-blindness interpretation wherein the disruptive effects of cell-phone conversations on driving are due in large part to the diversion of attention from driving to the phone conversation.     

Effects of different non-driving-related-task display modes on drivers’ eye-movement patterns during take-over in an automated vehicle

Highly automated vehicles relieve drivers from driving tasks, allowing them to engage in non-driving-related-tasks (NDRTs). However, drivers are required to take over control in certain circumstances due to the limitations of highly automated vehicles. This study focused on drivers’ eye-movement patterns during take-overs when an NDRT (watching videos) was presented via a head-up-display (HUD) and a mobile device display (MDD), compared to no NDRT as the baseline. The experiment was conducted in a high-fidelity driving simulator with real-world driving videos scenarios. Forty-six participants took part in the experiment by completing three drives in three counterbalanced conditions (HUD, MDD and baseline). A take-over-request was issued towards the end of automated driving requesting drivers to stop the NDRT and take over control. Eye-movement data including pupil diameter, blinks, glance duration and number of AOI (Area of Interest) were collected and analysed. The results show that during automated driving, drivers were more engaged in the MDD NDRT with smaller pupil diameter and shorter glance duration on the front scenario compared to the HUD and baseline modes. The number of AOI was reduced during automated driving in both MDD and HUD modes. The take-over-request redirected drivers’ visual attention back to the driving task from NDRT by increasing drivers’ pupil diameter, glance duration and number of AOI. However, the effect of MDD NDRT on pupil diameter and glance duration continued even after the take-over-request when the NDRT was terminated. The study demonstrated HUD is a better display to help maintain drivers’ attention on the road.     

Detecting eye movements in dynamic environments

To take advantage of the increasing number of in-vehicle devices, automobile drivers must divide their attention between primary (driving) and secondary (operating in-vehicle device) tasks. In dynamic environments such as driving, however, it is not easy to identify and quantify how a driver focuses on the various tasks he/she is simultaneously engaged in, including the distracting tasks. Measures derived from the driver’s scan path have been used as correlates of driver attention. This article presents a methodology for analyzing eye positions, which are discrete samples of a subject’s scan path, in order to categorize driver eye movements. Previous methods of analyzing eye positions recorded in a dynamic environment have relied completely on the manual identification of the focus of visual attention from a point of regard superimposed on a video of a recorded scene, failing to utilize information regarding movement structure in the raw recorded eye positions. Although effective, these methods are too time consuming to be easily used when the large data sets that would be required to identify subtle differences between drivers, under different road conditions, and with different levels of distraction are processed. The aim of the methods presented in this article are to extend the degree of automation in the processing of eye movement data by proposing a methodology for eye movement analysis that extends automated fixation identification to include smooth and saccadic movements. By identifying eye movements in the recorded eye positions, a method of reducing the analysis of scene video to a finite search space is presented. The implementation of a software tool for the eye movement analysis is described, including an example from an on-road test-driving sample.     

Movement control in older adults: does old age mean middle of the road?

Old age is associated with poorer movement skill, as indexed by reduced speed and accuracy. Nevertheless, reductions in speed and accuracy can also reflect compensation as well as deficit. We used a manual tracing and a driving task to identify generalized spatial and temporal compensations and deficits associated with old age. In Experiment 1, participants used a hand-held stylus to trace a path. In Experiment 2, participants steered along paths in a virtual reality driving simulator. In both experiments, participants were required to stay within the boundaries while we manipulated task difficulty by changing path width or movement speed. The older group showed worse performance in the highly constrained conditions. Corner cutting effectively reduces the curvature of bends but yields a greater risk of error (i.e., clipping the path or road edge). Corner cutting is thus less risky on wider paths, and we found that corner cutting increased for both age groups in both tasks when paths were wider. Crucially, we observed a greater degree of corner cutting in the young group compared with the old, suggesting the old group compensated for decreased motor skill with "middle-of-the-road" behavior. Enforcing increased speed caused all participants to increase corner cutting. Thus, older participants showed spatial compensation for decreased skill by biasing their position toward the middle of the path in both a manual and steering task. External constraints (narrow paths and fast speeds) prevented this strategy and revealed age-related declines in skills central to manual control and driving.     

Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence

Adaptive cruise control (ACC), a driver assistance system that controls longitudinal motion, has been introduced in consumer cars in 1995. A next milestone is highly automated driving (HAD), a system that automates both longitudinal and lateral motion. We investigated the effects of ACC and HAD on drivers’ workload and situation awareness through a meta-analysis and narrative review of simulator and on-road studies. Based on a total of 32 studies, the unweighted mean self-reported workload was 43.5% for manual driving, 38.6% for ACC driving, and 22.7% for HAD (0% = minimum, 100 = maximum on the NASA Task Load Index or Rating Scale Mental Effort). Based on 12 studies, the number of tasks completed on an in-vehicle display relative to manual driving (100%) was 112% for ACC and 261% for HAD. Drivers of a highly automated car, and to a lesser extent ACC drivers, are likely to pick up tasks that are unrelated to driving. Both ACC and HAD can result in improved situation awareness compared to manual driving if drivers are motivated or instructed to detect objects in the environment. However, if drivers are engaged in non-driving tasks, situation awareness deteriorates for ACC and HAD compared to manual driving. The results of this review are consistent with the hypothesis that, from a Human Factors perspective, HAD is markedly different from ACC driving, because the driver of a highly automated car has the possibility, for better or worse, to divert attention to secondary tasks, whereas an ACC driver still has to attend to the roadway.