Driver glance behavior towards displayed images on in-vehicle information systems under real driving conditions

Intelligent vehicle technologies like driver assistance systems and in-vehicle information systems, enhance convenience of the driving experience for drivers and passengers. At the same time, these systems may increase driver distraction and workload. Guidelines developed for this purpose include principles, methods, and assessments which are widely agreed upon, with some being singled out for a particular recommendation or requirement. Especially the display of graphical or photographic images are generally assumed to distract the driver from safely operating the vehicle and should be blocked during driving under all circumstances (so called per se lock outs). This study investigates the effect of displaying graphical and photographical images during driving on driveŕs glance behavior during real-world driving. Findings presented in this paper provide empirical evidence for the unobtrusiveness of these stimuli: Participants didn’t exhibit longer glance durations towards in-vehicle information systems, nor a deterioration of driver distraction parameters such as total eyes off road time and long glance proportion when being compared to driving without displaying any photographic images.     

How drivers adapt their behaviour to changes in task complexity: The role of secondary task demands and road environment factors

The present study investigates the impact of different sources of task complexity such as driving demands and secondary task demands on driver behaviour. Although much research has been dedicated to understanding the impact of secondary task demands or specific road traffic environments on driving performance, there is little information on how drivers adapt their behaviour to their combined presence. This paper aims to describe driver behaviour while negotiating different sources of task complexity, including mobile phone use while driving (i.e., calling and texting) and different road environments (i.e., straight segments, curves, hills, tunnels, and curves on hills). A driving simulator experiment was conducted to explore the effects of different road scenarios and different types of distraction while driving. The collected data was used to estimate driving behaviour through a Generalized Linear Mixed Model (GLMM) with repeated measures. The analysis was divided into two phases. Phase one aimed to evaluate driver performance under the presence and absence of pedestrians and oncoming traffic, different lanes width and different types of distraction. The second phase analysed driver behaviour when driving through different road geometries and lane widths and under different types of distraction. The results of the experiment indicated that drivers are likely to overcorrect position in the vehicle lane in the presence of pedestrians and oncoming traffic. The effect of road geometry on driver behaviour was found to be greater than the effect of mobile phone distraction. Curved roads and hills were found to influence preferred speeds and lateral position the most. The results of this investigation also show that drivers under visual-manual distraction had a higher standard deviation of speed and lateral position compared to the cognitive distraction and the non-distraction condition.     

Evaluating the impacts of situational awareness and mental stress on takeover performance under conditional automation

Several safety concerns emerge for the transition of control from the automated driving system to a human driver after the vehicle issues a takeover warning under conditional vehicle automation (SAE Level 3). In this context, recent advances in in-vehicle driver monitoring systems enable tracking drivers’ physiological indicators (e.g., eye-tracking and heart rate (HR) measures) to assess their real-time situational awareness (SA) and mental stress. This study seeks to analyze differences in driver’s SA and mental stress over time (i.e., successive experiment runs) using these physiological indicators to assess their impacts on takeover performance. We use eye-tracking measures (i.e., on-road glance rate and road attention ratio) as indicators of driver’s SA during automated driving. Further, we use the pre-warning normalized HR (NHR) and HR variability (HRV) as well as the change in NHR and HRV after the takeover warning as indicators of mental stress immediately before and the change in mental stress after the takeover warning, respectively. To analyze the effects of driver state (in terms of SA and mental stress) on the overall takeover performance, this study uses a comprehensive metric, Takeover Performance Index (TOPI), proposed in our previous work (Agrawal & Peeta, 2021). The TOPI combines multiple driving performance indicators while partly accounting for their interdependencies. Results from statistical analyses of data from 134 participants using driving simulator experiments illustrate significant differences in driver state over successive experiment runs, except for the change in mental stress after the takeover warning. Some significant correlations were found between the physiological indicators of SA and mental stress used in this study. Takeover performance model results illustrate a significant negative effect of change in NHR after the takeover warning on the TOPI. However, none of the other physiological indicators show significant impacts on takeover performance. The study findings provide valuable insights to auto manufacturers for designing integrated in-vehicle driver monitoring and warning systems that enhance road safety and user experience.     

Drivers’ attitudes toward imperfect distraction mitigation strategies

Studies were conducted to assess driver acceptance of and trust in distraction mitigation strategies. Previous studies have shown that in-vehicle tasks undermine driver safety, and that there is a need for strategies to reduce the effects of in-vehicle distractions. Trust and acceptance of such strategies strongly influence their effectiveness. Different strategies intended to reduce distraction were categorized in a taxonomy. Focus groups were conducted to help refine this taxonomy and explore driver acceptance issues related to these strategies. A driving simulator experiment was then conducted using two of the strategies: an advising strategy that warns drivers of potential dangers and a locking strategy that prevents the driver from continuing a distracting task. These strategies were presented to 16 middle-aged and 12 older drivers in two modes (auditory, visual) with two levels of adaptation (true, false). Older drivers accepted and trusted the strategies more than middle-aged drivers. Regardless of age, all drivers preferred strategies that provided alerts in a visual mode rather than an auditory mode. When the system falsely adapted to the road situation, trust in the strategies declined. The findings show that display modality has a strong effect on driver acceptance and trust, and that older drivers are more trusting and accepting of distraction mitigation technology even when it operates imperfectly.     

The impact of an in-vehicle display on glance distribution in partially automated driving in an on-road experiment

One of the major challenges of designing an HMI for partially automated vehicles is the trade-off between a sufficient level of system information and avoidance of distracting the driver. This study aimed to investigate drivers’ glance behavior as an indicator of distraction when vehicle guidance is partially automated. Therefore, an on-road experiment was conducted comparing two versions of an in-vehicle display (during partially automated driving) and no display (during manual driving) on a heavy congested highway segment. The distribution of drivers’ total glance durations on the HMI showed that visual attention was shifted away from monitoring the central road scene towards looking at the in-vehicle display to a considerable extent. However, an analysis of the distribution of single glance durations supports the view that using partial automation and a respective HMI does not lead to a critical increase in distraction. Driving with a simplified version of the HMI had the potential to reduce glance duration on and thus potential distraction of the in-vehicle display.     

Effects of cognitive load presence and duration on driver eye movements and event detection performance

This study examined the effects of cognitive load on driving performance for interactions with an in-vehicle information system (IVIS) that varied in duration from 1 to 4 min. Twelve participants drove in a simulator while intermittently performing the IVIS task. There were three IVIS conditions: interacting with the IVIS, non-IVIS periods between IVIS interactions, and baseline driving without the IVIS task. Contrary to our hypothesis, driver response to lead vehicle braking was surprisingly uniform across IVIS conditions. IVIS interaction did undermine driver ability to detect the bicyclist along the side of the road, and some of these performance decrements persisted after the IVIS interaction had ended. Reaction time for bicyclist detection increased from the first to the subsequent minutes of the interaction. Eye movements were influenced by the IVIS conditions but not by task duration. Both ANOVA and factor analyses revealed that some of the changes in eye movements were concurrent with IVIS interaction while others persisted after the driver completed the IVIS interaction. Overall, the findings suggest that two mechanisms might account for the distraction-related performance decrements in this study: competition for processing resources and interference due to activation of competing goals.     

Effects of route familiarity on drivers’ psychological conditions: Based on driving behaviour and driving environment

Route familiarity affects a driver’s mental state and indirectly affects traffic safety; however, this important factor is easily overlooked. Previous research on route familiarity has only analysed psychological states in terms of unfamiliarity and familiarity, the influence of driving behaviour and driving environment on psychological states has been ignored. As a result, the mechanisms through which the route familiarity influence driver psychological states, and vice versa, are unclear. This study proposes a quantitative framework for studying driver psychological condition and route familiarity using experimental data from a real driving task and driving environment data. The experimental data included 1022 observations obtained by 23 participants over 7 consecutive trials on 6 unfamiliar experimental routes with large differences in scenarios; environmental data were automatically extracted after segmenting a driving video through the Dilated Residual NetWorks model. The results reveal that (1) the relationship between the driver’s psychological condition and route familiarity is not monotonic and is different for straight and turning sections; (2) the driver’s psychological condition is influenced by the visual scene elements and the type of road section, and the results of the multivariate regression analysis quantified the variability of the influence; and (3) unlike a majority of findings on distracted driving, our study suggest that the driver’s attention to the external environment in the urban distracted driving state will gradually approach a ‘distraction threshold’, and the time and size of the ‘distraction threshold’ are influenced by the driver. This study can further the development of urban traffic safety research and help urban designers plan and improve urban landscapes to ensure drivers maintain stable mental states when they drive.     

The prevalence of in-vehicle driving distractions in road traffic collisions as a function of road type

While some previous research suggests that conversing with passengers is the most prevalent in-vehicle distraction while driving, others have concluded instead that it is mobile phone use. One possible explanation for these differences is that distraction prevalence varies with road type. To test this proposal the current study investigated the prevalence of in-vehicle driving distraction in road traffic collisions (RTCs) as recorded in national records from the US and New Zealand. Analysis using odds ratios suggested conversing with passengers to be a more prevalent distraction in RTCs on minor roads than on major roads, and mobile phone use to be a more prevalent distraction on major roads than on minor roads. These results show the importance of considering the type of road when investigating the prevalence of driving distractions in RTCs in future research.     

Human factors evaluation of an in-vehicle active traffic and demand management (ATDM) system

This research study focused on the evaluation of an emulated in-vehicle Active Traffic and Demand Management (ATDM) system on Interstate 66 in Northern Virginia. Traditional ATDM systems rely on infrastructure-mounted variable message signs (VMS) to provide information (speed limits, lane availability, etc.) to the traveling public. By providing information about dynamic roadway conditions on an in-vehicle device, the ATDM may improve driving safety and performance by allowing drivers to remain consistently aware of forthcoming traffic conditions and roadway requirements; even when external signage is inaccessible. This study represents an initial investigation of an emulated in-vehicle ATDM to provide developers with design guidance and ensure that unintended consequences, such as distraction, do not undermine the potential benefits.Twenty younger and 20 older participants, accompanied by a member of the research team, experienced the following ATDM features on an in-vehicle device (IVD) mounted to the dashboard: (1) dynamic speed limits, (2) dynamic lane use/shoulder control, (3) High Occupancy Vehicle (HOV) restrictions, and (4) roadway information from variable message signs. The IVD was equipped with auditory and visual alerts notifying the driver when relevant visual information was updated. Research questions addressed distraction and driver behavior associated with use of the system. Qualitative and quantitative participant data was acquired from the instrumented vehicle, various questionnaires, and researcher observation.Several key findings were uncovered: (1) The IVD, as tested, did not warrant classification as a source of distraction according to the NHTSA guidelines; v2) There was a significant difference in eye-glance durations to the IVD when comparing the VMS alerts to both the speed limit and lane management alerts; and (3) The speed limit alert motivated participants to alter their speed (per survey results and participant speed data).     

Distraction while driving: The case of older drivers

As the impairment of older drivers is especially found in perception and attention, one could assume that they are especially prone to distraction effects of secondary tasks performed while driving. The aim of the study was to examine the effect of age on driving performance as well as the compensation strategies of older drivers under distraction. 10 middle-aged and 10 older drivers drove in a simulator with and without a secondary task. To assess driving performance the Lane Change Task (Mattes, 2003) was used. This method aims at estimating driver demand while a secondary task is being performed, by measuring performance degradation on a primary driving-like task in a standardized manner. The secondary task – a self-developed computer-based version of “d2 Test of Attention” was presented both with and without time pressure. The results show that older participants’ overall driving performance (mean deviation from an ideal path) was worse in all conditions as compared to the younger ones. With regard to lane change reaction time both age groups were influenced by distraction in a comparable manner. However, when the lane keeping performance (standard deviation of the lateral position) was examined, the older participants were more affected than the younger ones. This pattern could be explained by compensation strategies of the older drivers. They focused on the most relevant part of the driving task, the lane change manoeuvres and were able to maintain their performance level in a similar way as did younger drivers. The driving performance of the older participants was not additionally impaired when the secondary task imposed time pressure. Overall, subjective rating of driving performance, perceived workload and perceived distraction was found to be similar for both age groups. The observed trends and patterns associated with distraction while driving should contribute to the further research or practical work regarding in-vehicle technologies and older drivers.     

Imperfect in-vehicle collision avoidance warning systems can aid distracted drivers

The purpose of this study was to evaluate the efficacy of a type of in-vehicle collision avoidance warning system (IVCAWS) under conditions of driver distraction. Forty-three participants responded to an imperfect warning system while simultaneously driving a simulator and performing a visual/cognitive task. The major concerns were whether drivers would be more inclined to rely on such a system when they are distracted by subsidiary tasks, and if this reliance would be counterproductive. We found that distracted drivers responded, by increasing their temporal headway, to the less reliable system’s alarms, but the warning system at the higher reliability levels led to over reliance and ultimately to maintaining shorter headways. This study has practical implications for the use of warning systems as driving aids for drivers. Although aids may be helpful and, in many cases, the more reliable aid is preferable, in the case of distraction, drivers may misuse the aid.     

The power and sensitivity of four core driver workload measures for benchmarking the distraction potential of new driver vehicle interfaces

This study evaluated the power and sensitivity of several core driver workload measures in order to better understand their use as a component of future driver distraction potential evaluation procedures of the in-vehicle human machine interface (HMI). Driving is a task that requires visual, manual and cognitive resources to perform. Secondary tasks, such as mobile phone use and interaction with in-built navigation, which load onto any of these three processing resources increase driver workload and can lead to impaired driving. Because workload and distraction potential are interrelated, a comprehensive method to assess driver workload that produces valid and predictive results is needed to advance the science of distraction potential evaluation. It is also needed to incorporate into New Car Assessment Program (NCAP) testing regimes. Workload measures of cognitive (DRT [Detection Response Task] Reaction Time), visual (DRT Miss Rate), subjective (NASA-TLX [driver workload questionnaire]), and temporal demand (Task Interaction Time) were collected as participants drove one of 40 vehicles while completing a variety of secondary tasks with varying interaction requirements. Of the evaluated measures, variance and power analyses demonstrated that Task Interaction Time is the most sensitive in detecting differences in driver workload between different in-vehicle HMIs, followed by DRT Miss Rate, NASA-TLX and finally DRT Reaction Time. There were relatively weak correlations between each of the four measures. These results suggest that Task Interaction Time, coupled with a reliable visual demand metric such as DRT Miss Rate, eye glance coding, or visual occlusion, more efficiently detect differences in driver workload between different HMIs compared to DRT Reaction Time and the NASA-TLX questionnaire. These results can be used to improve the understanding of the utility of each of these core driver workload measures in assessing driver distraction potential.     

Measurement of auditory cues in drivers' distraction

A driving simulator was used to examine the effects on driving performance of auditory cues in an in-vehicle information search task. Drivers' distraction by the search tasks was measured on a peripheral detection task. The difficulty of the search task was systematically varied to test the distraction caused by a quantified visual load. 58 participants completed the task. Performance on both search tasks and peripheral detection tasks was measured by mean response time and percent error. Analyses indicated that in-vehicle information search performance can be severely degraded when a target is located within a group of diverse distractors. Inclusion of an auditory cue in the visual search increased the mean response time as a result of a change in modality from auditory to visual. Inclusion of such an auditory cue seemed to influence distraction as measured by performance on the peripheral detection task; accuracy was lower when auditory cues were provided, and responses were slower when no auditory cues were provided. Distraction by the auditory cue varied according to the difficulty of the search task.     

The impact of navigation system display size and environmental illumination on young driver mental workload

Driver distraction is a major cause of road crashes and has a great influence on road safety. In vehicles, one of the common distracting sources is navigation systems (NSs). The navigation system (NS) can distract the driver due to following directions and reading the provided information through its display. These tasks take the driver’s attention from the primary task of driving and may cause poor driving performance, increasing the risk of crashes. In this paper, the effect of the environment (i.e., urban areas and rural areas), the navigation system display (NSD) size, environmental illumination, and gender on young drivers between the ages of 18 and 29 years mental workload was investigated using a simulated driving experiment. To evaluate each driving condition, the NASA-TLX (NASA Task Load Index) workload assessment tool, and a distraction evaluation element, were introduced and used to assess the overall workload, the workload subscales and the distraction by the NSD. The assessment showed a higher perceived overall workload for urban areas and night driving as compared to a rural areas and daytime driving. Moreover, the results showed a greater perceived distraction by the NSD in urban areas compared to driving in rural areas. The subjects also felt distracted when using the small NS compared to using the large NS. The study concluded that urban areas driving, and night driving creates higher perceived workload than rural areas and daytime driving. Furthermore, small NSD leads to more perceived distraction than large NSD while driving. The NSD designers may utilize this research findings to optimize NSD designs to improve driving safety, performance and comfort. Moreover, this study contributes to our understanding of the effect of the NSD size on driving workload and distraction.     

Detecting eye movements in dynamic environments

To take advantage of the increasing number of in-vehicle devices, automobile drivers must divide their attention between primary (driving) and secondary (operating in-vehicle device) tasks. In dynamic environments such as driving, however, it is not easy to identify and quantify how a driver focuses on the various tasks he/she is simultaneously engaged in, including the distracting tasks. Measures derived from the driver’s scan path have been used as correlates of driver attention. This article presents a methodology for analyzing eye positions, which are discrete samples of a subject’s scan path, in order to categorize driver eye movements. Previous methods of analyzing eye positions recorded in a dynamic environment have relied completely on the manual identification of the focus of visual attention from a point of regard superimposed on a video of a recorded scene, failing to utilize information regarding movement structure in the raw recorded eye positions. Although effective, these methods are too time consuming to be easily used when the large data sets that would be required to identify subtle differences between drivers, under different road conditions, and with different levels of distraction are processed. The aim of the methods presented in this article are to extend the degree of automation in the processing of eye movement data by proposing a methodology for eye movement analysis that extends automated fixation identification to include smooth and saccadic movements. By identifying eye movements in the recorded eye positions, a method of reducing the analysis of scene video to a finite search space is presented. The implementation of a software tool for the eye movement analysis is described, including an example from an on-road test-driving sample.     

Measuring being lost in thought: An exploratory driving simulator study

In driver behaviour research there is considerable focus on distraction caused by specific external systems, such as navigation systems or mobile telephones. However, it is not clear whether self-paced actions such as daydreaming have the same negative effects on driving behaviour. In a driving simulator study, the effects of an internal cognitive process (internal distraction) on driving behaviour and physiological data were compared to the effects of a sound and speech task (external distraction). Three groups of participants made two drives on a motorway, with one control group, one internal distraction group and one external distraction group. Dependent measures included driving behavioural measures, physiological measures and a subjective indication of participants’ experienced involvement in the driving task.The effects of both the internal and external distraction task were reflected in speed, number of lane changes, deceleration, glances and subjective ratings. When an effect was found for both the internal and the external distraction task, the results indicated similar (negative) effects. Participants also indicated that they had the feeling they were less involved in the driving task with both secondary tasks.     

Real-time detection of drivers’ texting and eating behavior based on vehicle dynamics

With rapid advancement in cellphones and intelligent in-vehicle technologies along with driver’s inclination to multitasking, crashes due to distracted driving had become a growing safety concern in our road network. Some previous studies attempted to detect distracted driving behaviors in real-time to mitigate their adverse consequences. However, these studies mainly focused on detecting either visual or cognitive distractions only, while most of the real-life distracting tasks involve driver’s visual, cognitive, and physical workload, simultaneously. Additionally, previous studies frequently used eye, head, or face tracking data, although current vehicles are not commonly equipped with technologies to acquire such data. Also those data are comparatively difficult to acquire in real-time during traffic monitoring operations. To address the above issues, this study focused on developing algorithms for detecting distraction tasks that involve simultaneous visual, cognitive, and physical workload using only vehicle dynamics data. Specifically, algorithms were developed to detect driving behaviors under two distraction tasks – texting and eating. Experiment was designed to include the two distracted driving scenarios and a control with multiple runs for each. A medium fidelity driving simulator was used for acquiring vehicle dynamics data for each scenario and each run. Several data mining techniques were explored in this study to investigate their performance in detecting distraction. Among them, the performance of two linear (linear discriminant analysis and logistic regression) and two nonlinear models (support vector machines and random forests) is reported in this article. Random forests algorithms had the best performance, which detected texting and eating distraction with an accuracy of 85.38% and 81.26%, respectively. This study may provide useful guidance to successful development and implementation of distracted driver detection algorithms in connected vehicle environment, as well as to auto manufacturers interested in integrating distraction detection systems in their vehicles.     

The effect of the dominance of an in-vehicle agent’s voice on driver situation awareness,emotion regulation,and trust: A simulated lab study of manual and automated driving

As in-vehicle voice agents increase in popularity, related research is extending to how voice messages can affect the driver’s cognitive and emotional states. Accordingly, we investigated how in-vehicle agent (IVA) voice dominance and driving automation can affect the driver’s situation awareness (SA), emotion regulation (ER), and trust. To this end, a lab-based experiment was conducted with a medium-fidelity driving simulator using actor-recorded voice agents. Twenty-two female and nineteen male driver-licensed participants were recruited to drive simulated vehicles with voice agents and evaluated. The results demonstrated that compared with the dominant voice, the agent with a submissive voice significantly increased ER in both manual and automated driving. Furthermore, the submissive voice significantly increased trust in automated driving compared with the dominant agent. Cross and synergistic interaction effects exist between voice dominance and driving automation in SA and ER, respectively. This study revealed that both the content of the messages of the IVAs and their voice characteristics are essential for modulating the driver’s SA, ER, and trust in driving. It is expected that larger-scale future studies with simulation or on a real road would increase the validity of this study.     

The effect of a simulated hearing loss on performance of an auditory memory task in driving

Hearing loss has been shown to exacerbate the effect of auditory distraction on driving performance in older drivers. This study controlled for the potentially confounding factor of age-related cognitive decrements, by applying a simulated hearing loss in young, normally hearing individuals. Participants drove a simulated road whilst completing auditory tasks under simulated hearing loss or normal hearing conditions. Measures of vehicle control, eye movements and auditory task performance were recorded. Results showed that performing the auditory tasks whilst driving resulted in more stable lateral vehicle control and a reduction in gaze dispersion around the road centre. These trends were not exacerbated by simulated hearing loss, suggesting no effect of hearing loss on vehicle control or eye movement patterns during auditory task engagement. However, a small effect of simulated hearing loss on the performance of the most complex auditory task was observed during driving, suggesting that the use of sound-based in-vehicle systems may be problematic for hearing impaired individuals. Further research incorporating a wider variety of driving scenarios and auditory tasks is required in order to confirm the findings of this study.     

Analysis of vehicle-based lateral performance measures during distracted driving due to phone use

Distracted driving due to mobile phone use has been identified as a major contributor to accidents; therefore, it is required to develop ways for detecting driver distraction due to phone use. Though prior literature has documented various visual behavioural and physiological techniques to identify driver distraction, comparatively little is known about vehicle based performance features which can identify driver’s distracted state during phone conversation and texting while driving. Therefore, this study examined the effects of simple conversation, complex conversation, simple texting and complex texting tasks on vehicle based performance parameters such as standard deviation of lane positioning, number of lane excursions, mean and standard deviation of lateral acceleration, mean and standard deviation of steering wheel angle and steering reversal rates (for 1°, 5° and 10° angle differences). All these performance measures were collected for 100 licensed drivers, belonging to three age groups (young, mid-age and old age), with the help of a driving simulator. Effects of all the phone use conditions and driver demographics (age, gender and phone use habits) on the measures were analysed by repeated measures ANOVA tests. Results showed that 1°, 5° SRRs are able to identify all the distracted conditions except for simple conversation; while, 10° SSR can detect all the distracted conditions (including simple conversation). The results suggest that 10° SRR can be included in intelligent in-vehicle devices in order to detect distraction and alert drivers of their distracted state. This can prevent mobile phone use during driving and therefore can help in reducing the road accidents due to mobile phone distractions.