Secondary task engagement and disengagement in the context of highly automated driving

During highly automated driving (level 3 automation according to SAE International, 2014) people are likely to increase the frequency of secondary task interactions. However, the driver must still be able to take over control within a reasonable amount of time. Previous studies mainly investigated take-over behavior by forcing participants to engage in secondary tasks prior to take over, and barely addressed how drivers voluntarily schedule secondary task processing according to the availability and predictability of automated driving modes. In the current simulator study 20 participants completed a test drive with alternating sections of manual and highly automated driving. One group had a preview on the availability of the automated driving system in upcoming sections of the track (predictive HMI), while the other drivers served as a control group. A texting task was offered during both driving modes and also prior to take-over situations. Participants were free to accept or reject a given task, taking the situational demands into account. Drivers accepted more tasks during highly automated driving. Furthermore, tasks were rejected more often prior to take-over situations in the predictive HMI group. This was accompanied by safer take-over performance. However, once engaged in a task, drivers tended to continue texting even in take-over situations. The results indicate the need to discriminate different aspects of task handling regarding self-regulation: task engagement and disengagement.     

Impact of interface design on drivers’ behavior in partially automated cars: An on-road study

In partially automated vehicles, the driver and the automated system share control of the vehicle. Consequently, the driver may have to switch between driving and monitoring activities. This can critically impact the driver’s situational awareness. The human–machine interface (HMI) is responsible for efficient collaboration between driver and system. It must keep the driver informed about the status and capabilities of the automated system, so that he or she knows who or what is in charge of the driving. The present study was designed to compare the ability of two HMIs with different information displays to inform the driver about the system’s status and capabilities: a driving-centered HMI that displayed information in a multimodal way, with an exocentric representation of the road scene, and a vehicle-centered HMI that displayed information in a more traditional visual way. The impact of these HMIs on drivers was compared in an on-road study. Drivers’ eye movements and response times for questions asked while driving were measured. Their verbalizations during the test were also transcribed and coded. Results revealed shorter response times for questions on speed with the exocentric and multimodal HMI. The duration and number of fixations on the speedometer were also greater with the driving-centered HMI. The exocentric and multimodal HMI helped drivers understand the functioning of the system, but was more visually distracting than the traditional HMI. Both HMIs caused mode confusions. The use of a multimodal HMI can be beneficial and should be prioritized by designers. The use of auditory feedback to provide information about the level of automation needs to be explored in longitudinal studies.     

Secondary task engagement and vehicle automation – Comparing the effects of different automation levels in an on-road experiment

Recent and upcoming advances in vehicle automation are likely to change the role of the driver from one of actively controlling a vehicle to one of monitoring the behaviour of an assistant system and the traffic environment. A growing body of literature suggests that one possible side effect of an increase in the degree of vehicle automation is the tendency of drivers to become more heavily involved in secondary tasks while the vehicle is in motion. However, these studies have mainly been conducted in strictly controlled research environments, such as driving simulators and test tracks, and have mainly involved either low levels of automation (i.e., automation of longitudinal control by Adaptive Cruise Control (ACC)) or Highly automated driving (i.e., automation of both longitudinal and lateral control without the need for continuous monitoring). This study aims to replicate these effects during an on-road experiment in everyday traffic and to extend previous findings to an intermediate level of automation, in which both longitudinal and lateral control are automated but the driver must still monitor the traffic environment continuously (so-called Partial automation). N = 32 participants of different age groups and different levels of familiarity with ACC drove in rush-hour traffic on a highway segment. They were assisted by ACC, ACC with steering assistance (ACC+SA), or not at all. The results show that while subjective and objective driving safety were not influenced by the degree of automation, drivers who were already familiar with ACC increased the frequency of interactions with an in-vehicle secondary task in both assisted drives. However, participants generally rated performing the secondary task as less effortful when being assisted, regardless of the automation level (ACC vs. ACC+SA). The results of this on-road experiment thus validate previous findings from more-controlled research environments and extend them to Partially automated driving.     

Effects of secondary tasks and display position on glance behavior during partially automated driving

The driving task is becoming increasingly automated, thus changing the driver’s role. Moreover, in-vehicle information systems using different display positions and information processing channels might encourage secondary task engagement. During manual driving scenarios, varying secondary tasks and display positions could influence driver’s glance behavior. However, their impact on the driver’s capability to monitor the partially automated driving system has not yet been determined. The current study assessed both the effects of different secondary tasks (Surrogate Reference Task (SuRT) vs. text reading) and display positions (head-up display (HUD) vs. center console) on driver’s glance behavior during partially automated driving in a simulated car following task. Different automation system failures regarding the lateral and longitudinal control occurred while driving. Furthermore, participants’ reported advantages, disadvantages and preferences regarding the investigated display positions as well as regarding the secondary task engagement during partially automated driving in general. Mixed design ANOVAs revealed that the HUD yielded considerably longer eyes-on display time (total and mean glance durations) than the center console. Moreover, the text reading task resulted in longer total and mean glance durations than the SuRT. Similar to manual driving scenarios, the results showed a consistent effect of display position and secondary task on the driver’s glance behavior. Despite the longer eyes-on display time for the HUD, its proximity to the driving environment might enable a faster identification of and reaction to critical situations (e.g., due to system failures). Participants would prefer the HUD as display position compared to the center console. Regarding secondary task engagement during partially automated driving participants seemed to be aware of the benefits but also of the risks.     

The effect of visual HMIs of a system assisting manual drivers in manoeuvre coordination in system limit and system failure situations

Ambiguous situations in traffic often require communication and cooperation between road users. In order to resolve these situations and increase cooperative driving behavior in situations of merging or turning left, manual drivers could be assisted by an advanced driver assistance system (ADAS) for cooperative driving. This simulator study investigated the behavior of drivers confronted with system limits and failures of such a system. The ADAS used in this study informed the driver about an upcoming cooperation situation and gave advice on how to behave (e.g. reduce speed, change lane). Two test situations were implemented: a system freeze and an unexpected event, which could not be detected by the system. In order to find the most fitting HMI solution, the place of presentation (head-up display (HUD) vs. instrument cluster) as well as the form of presentation (dynamic vs. symbolic) were varied. The results indicated that the most fitting HMI solution to support the driver in a complex coordinated driving situation is a dynamic HUD, mainly due to the positive effect on glance behavior. However, advantages of both forms of presentation were revealed, as each form of presentation increased the probability of recognition for one of the test situations. The fewest collisions took place with the dynamic form of presentation.     

Individual differences in response to automation: the five factor model of personality

This study examined the relationship of operator personality (Five Factor Model) and characteristics of the task and of adaptive automation (reliability and adaptiveness-whether the automation was well-matched to changes in task demand) to operator performance, workload, stress, and coping. This represents the first investigation of how the Five Factors relate to human response to automation. One-hundred-sixty-one college students experienced either 75% or 95% reliable automation provided with task loads of either two or four displays to be monitored. The task required threat detection in a simulated uninhabited ground vehicle (UGV) task. Task demand exerted the strongest influence on outcome variables. Automation characteristics did not directly impact workload or stress, but effects did emerge in the context of trait-task interactions that varied as a function of the dimension of workload and stress. The pattern of relationships of traits to dependent variables was generally moderated by at least one task factor. Neuroticism was related to poorer performance in some conditions, and all five traits were associated with at least one measure of workload and stress. Neuroticism generally predicted increased workload and stress and the other traits predicted decreased levels of these states. However, in the case of the relation of Extraversion and Agreeableness to Worry, Frustration, and avoidant coping, the direction of effects varied across task conditions. The results support incorporation of individual differences into automation design by identifying the relevant person characteristics and using the information to determine what functions to automate and the form and level of automation.     

The impact of an in-vehicle display on glance distribution in partially automated driving in an on-road experiment

One of the major challenges of designing an HMI for partially automated vehicles is the trade-off between a sufficient level of system information and avoidance of distracting the driver. This study aimed to investigate drivers’ glance behavior as an indicator of distraction when vehicle guidance is partially automated. Therefore, an on-road experiment was conducted comparing two versions of an in-vehicle display (during partially automated driving) and no display (during manual driving) on a heavy congested highway segment. The distribution of drivers’ total glance durations on the HMI showed that visual attention was shifted away from monitoring the central road scene towards looking at the in-vehicle display to a considerable extent. However, an analysis of the distribution of single glance durations supports the view that using partial automation and a respective HMI does not lead to a critical increase in distraction. Driving with a simplified version of the HMI had the potential to reduce glance duration on and thus potential distraction of the in-vehicle display.     

Effects of explanation-based knowledge regarding system functions and driver’s roles on driver takeover during conditionally automated driving: A test track study

The present study was designed to examine the influence of explanation-based knowledge regarding system functions and the driver’s role in conditionally automated driving (Level 3, as defined in SAE J3016). In particular, we studied how safely and successfully drivers assume control of the vehicle when encountering situations that exceed the automation parameters. This examination was conducted through a test-track experiment. Thirty-two younger drivers (mean age = 37.3 years) and 24 older drivers (mean age = 71.1 years) participated in Experiments 1 and 2, respectively. Adopting a between-participants design, in each experiment the participants were divided into two age- and sex-matched groups that were given differing levels of explanation-based knowledge concerning the system limitations of automated driving. The only information given to the less-informed groups was that, during automated driving, drivers may be required to occasionally assume control of the vehicle. The well-informed groups were given the same information, as well as details regarding the auditory-visual alerts produced by the human–machine interface (HMI) during requests to intervene (RtIs), and examples of situations where RtIs would be issued. Ten and nine RtI events were staged for each participant in Experiment 1 and 2, respectively; the participants performed a non-driving-related task while the automated driving system was functioning. For both experiments it was found that, for all RtI events, more participants in the well-informed groups than the less-informed groups successfully assumed control of the vehicle. These results suggest that, in addition to providing information regarding the possible occurrence of RtIs, explanations of HMI and RtI-related situations are effective for helping both younger and older drivers safely and successfully negotiate such events.     

Measuring being lost in thought: An exploratory driving simulator study

In driver behaviour research there is considerable focus on distraction caused by specific external systems, such as navigation systems or mobile telephones. However, it is not clear whether self-paced actions such as daydreaming have the same negative effects on driving behaviour. In a driving simulator study, the effects of an internal cognitive process (internal distraction) on driving behaviour and physiological data were compared to the effects of a sound and speech task (external distraction). Three groups of participants made two drives on a motorway, with one control group, one internal distraction group and one external distraction group. Dependent measures included driving behavioural measures, physiological measures and a subjective indication of participants’ experienced involvement in the driving task.The effects of both the internal and external distraction task were reflected in speed, number of lane changes, deceleration, glances and subjective ratings. When an effect was found for both the internal and the external distraction task, the results indicated similar (negative) effects. Participants also indicated that they had the feeling they were less involved in the driving task with both secondary tasks.     

What and how to tell beforehand: The effect of user education on understanding,interaction and satisfaction with driving automation

The success of introducing automated driving systems to consumers will depend on an appropriate understanding and human-automation interaction with this technology. Educating users on driving automation technology bears the potential to attain these two requirements. In a driving simulator study, we investigated the effects of user education on mental models, human-automation interaction performance (i.e., time on task, error rate, experimenter rating) and satisfaction with a Human-Machine Interface (HMI) for automated driving. N = 80 participants were randomly assigned to one of three different user education conditions or to a baseline. Subsequently, they completed several driver-initiated control transitions between manual, Level 2 (L2), and Level 3 (L3) automated driving. The results revealed that user education promoted an accurate evolution of mental models for driving automation. These, in turn, facilitated interaction performance in transitions from manual to both L2 and L3 automated driving. There was no comparable influence of prior education on performance in transitions between the automation levels. Due to the performance enhancing effects of user education, no further improvements of interaction performance were observed for educated users in comparison to uneducated users. There was no effect of user education on satisfaction. The current findings emphasize the necessity to provide information about automated vehicle HMIs to first-time users to support accurate understanding and behavior. Based on the current findings, we propose conceptual approaches to teach users and derive implications for user studies on automated vehicle HMIs.     

Physiological indicators of driver workload during car-following scenarios and takeovers in highly automated driving

This driving simulator study, conducted as a part of Horizon2020-funded L3Pilot project, investigated how different car-following situations affected driver workload, within the context of vehicle automation. Electrocardiogram (ECG) and electrodermal activity (EDA)-based physiological metrics were used as objective indicators of workload, along with self-reported workload ratings. A total of 32 drivers were divided into two equal groups, based on whether they engaged in a non-driving related task (NDRT) during automation (SAE Level 3) or monitored the drive (SAE Level 2). Drivers in both groups were exposed to two counterbalanced experimental drives, lasting ∼ 18 min each, of Short (0.5 s) and Long (1.5 s) Time Headway conditions during automated car-following (ACF), which was followed by a takeover that happened with or without a lead vehicle. Results showed that driver workload due to the NDRT was significantly higher than both monitoring the drive during ACF and manual car-following (MCF). Furthermore, the results indicated that a lead vehicle maintain a shorter THW can significantly increase driver workload during takeover scenarios, potentially affecting driver safety. This warrants further research into understanding safe time headway thresholds to be maintained by automated vehicles, without placing additional cognitive or attentional demands on the driver. Our results indicated that ECG and EDA signals are sensitive to variations in workload, which warrants further investigation on the value of combining these two signals to assess driver workload in real-time, to help future driver monitoring systems respond appropriately to the limitations of the driver, and predict their performance in the driving task, if and when they have to resume manual control of the vehicle after a period of automated driving.     

How the initial level of trust in automated driving impacts drivers’ behaviour and early trust construction

Trust in Automation is known to influence human-automation interaction and user behaviour. In the Automated Driving (AD) context, studies showed the impact of drivers’ Trust in Automated Driving (TiAD), and linked it with, e.g., difference in environment monitoring or driver’s behaviour. This study investigated the influence of driver’s initial level of TiAD on driver’s behaviour and early trust construction during Highly Automated Driving (HAD). Forty drivers participated in a driving simulator study. Based on a trust questionnaire, participants were divided in two groups according to their initial level of TiAD: high (Trustful) vs. low (Distrustful). Declared level of trust, gaze behaviour and Non-Driving-Related Activities (NDRA) engagement were compared between the two groups over time. Results showed that Trustful drivers engaged more in NDRA and spent less time monitoring the road compared to Distrustful drivers. However, an increase in trust was observed in both groups. These results suggest that initial level of TiAD impact drivers’ behaviour and further trust evolution.     

Driver brake vs. steer response to sudden forward collision scenario in manual and automated driving modes

In autonomous vehicle operation, situations may arise when the driver is required to re-engage in manual control of the vehicle. Whether the control handoff from vehicle to human is done in a structured or unstructured manner, the process may be affected by the driver’s state, i.e. distracted or not. The study reported here was designed to measure a non-distracted driver’s response to a sudden forward collision (FC) event, in which the driver would assume manual control of the autonomous vehicle. Three driving scenarios were investigated: autonomous vehicle driven with full collision avoidance support, autonomous vehicle driven without collision avoidance support, and vehicle driven in manual mode.Forty-eight volunteers participated in a simulator study conducted in VIRTTEX. It was found that, at handoff, (1) drivers in manual mode tended to use evasive steering, rather than braking, compared to drivers in both the autonomous modes, (2) between subjects variations in speed were higher for the automation with collision support condition than for the other two scenarios, (3) for both autonomous driving scenarios, drivers reaction times were longer than for manual driving. In some cases the driver response was so late and the distance remaining so reduced that crash avoidance might be unfeasible. At a minimum, results of this study suggest that drivers may benefit from appropriate driver assistance technologies when a crash imminent situation is suddenly encountered.     

The effect of a simulated hearing loss on performance of an auditory memory task in driving

Hearing loss has been shown to exacerbate the effect of auditory distraction on driving performance in older drivers. This study controlled for the potentially confounding factor of age-related cognitive decrements, by applying a simulated hearing loss in young, normally hearing individuals. Participants drove a simulated road whilst completing auditory tasks under simulated hearing loss or normal hearing conditions. Measures of vehicle control, eye movements and auditory task performance were recorded. Results showed that performing the auditory tasks whilst driving resulted in more stable lateral vehicle control and a reduction in gaze dispersion around the road centre. These trends were not exacerbated by simulated hearing loss, suggesting no effect of hearing loss on vehicle control or eye movement patterns during auditory task engagement. However, a small effect of simulated hearing loss on the performance of the most complex auditory task was observed during driving, suggesting that the use of sound-based in-vehicle systems may be problematic for hearing impaired individuals. Further research incorporating a wider variety of driving scenarios and auditory tasks is required in order to confirm the findings of this study.     

Eye movements and hazard perception in active and passive driving

ABSTRACT

Differences in eye movement patterns are often found when comparing passive viewing paradigms to actively engaging in everyday tasks. Arguably, investigations into visuomotor control should therefore be most useful when conducted in settings that incorporate the intrinsic link between vision and action. We present a study that compares oculomotor behaviour and hazard reaction times across a simulated driving task and a comparable, but passive, video-based hazard perception task. We found that participants scanned the road less during the active driving task and fixated closer to the front of the vehicle. Participants were also slower to detect the hazards in the driving task. Our results suggest that the interactivity of simulated driving places increased demand upon the visual and attention systems than simply viewing driving movies. We offer insights into why these differences occur and explore the possible implications of such findings within the wider context of driver training and assessment.     

High perceptual load causes inattentional blindness and deafness in drivers

ABSTRACT

Perceptual load theory of attention predicts that the level of perceptual load in a primary task affects the processing of additional stimuli. Given the lack of ecologically valid evidence for the model, the current study assessed the effect of perceptual load on driver awareness during simulated driving tasks. The results showed that perceptual load dramatically affected driver awareness for visual and auditory stimuli, even those that were driving relevant and safety critical (e.g. pedestrians or the sound of a car horn). The results support load theory and suggest that perceptual load may be an important factor in driver safety.     

On-road trust and perceived risk in Level 2 automation

To encourage appropriate use of driving automation, we need to understand and monitor driver’s trust and risk perception. We examined (1) how trust and perceived risk are affected by automation, driving conditions and experience and (2) how well perceived risk can be inferred from behaviour and physiology at three levels: over traffic conditions, aggregated risk events, and individual risk events.30 users with and without automation experience drove a Toyota Corolla with driving support. Safety attitude, subjective ratings, behaviour and physiology were examined.Driving support encouraged a positive safety attitude and active driver involvement. It reduced latent hazards while maintaining saliently perceived risks. Drivers frequently overruled lane centring (3.1 times/minute) and kept their feet on or above the pedals using ACC (65.8% of time). They comfortably used support on curvy motorways and monotonic and congested highways but less in unstable traffic and on roundabouts. They trusted the automation 65.4%, perceived 36.0% risk, acknowledged the need to monitor and would not engage in more secondary tasks than during manual driving.Trust-in situation reduced 2.0% when using automation. It was 8.2% higher than trust-in-automation, presumably due to driver self-confidence. Driving conditions or conflicts between driver and automation did not affect trust-in-automation.At the traffic condition level, physiology showed weak and partially counter-intuitive effects. For aggregated risk events, skin conductance had the clearest response but was discernible from baseline in  < 50%. Pupil dilation and heart rate only increased with strong braking and active lane departure assist. For individual risk events, a CNN classifier could not identify risk events from physiology. We conclude that GSR, heart rate and pupil dilation respond to perceived risk, but lack specificity to monitor it on individual events.     

Human factors of transitions in automated driving: A general framework and literature survey

The topic of transitions in automated driving is becoming important now that cars are automated to ever greater extents. This paper proposes a theoretical framework to support and align human factors research on transitions in automated driving. Driving states are defined based on the allocation of primary driving tasks (i.e., lateral control, longitudinal control, and monitoring) between the driver and the automation. A transition in automated driving is defined as the process during which the human-automation system changes from one driving state to another, with transitions of monitoring activity and transitions of control being among the possibilities. Based on ‘Is the transition required?’, ‘Who initiates the transition?’, and ‘Who is in control after the transition?’, we define six types of control transitions between the driver and automation: (1) Optional Driver-Initiated Driver-in-Control, (2) Mandatory Driver-Initiated Driver-in-Control, (3) Optional Driver-Initiated Automation-in-Control, (4) Mandatory Driver-Initiated Automation-in-Control, (5) Automation-Initiated Driver-in-Control, and (6) Automation-Initiated Automation-in-Control. Use cases per transition type are introduced. Finally, we interpret previous experimental studies on transitions using our framework and identify areas for future research. We conclude that our framework of driving states and transitions is an important complement to the levels of automation proposed by transportation agencies, because it describes what the driver and automation are doing, rather than should be doing, at a moment of time.     

Evaluation of a dynamic blocking concept to mitigate driver distraction: Three simulator studies

In recent years, the number and complexity of in-vehicle infotainment systems has been steadily increasing. While these systems certainly improve the driving experience, they also increase the risk for driver distraction. International standards and guidelines provide methods of measuring this distraction along with test criteria that help automakers decide whether an interface task is too distracting to be used while driving. Any specific function failing this test should therefore be locked out for use by the driver. This study implemented and tested a dynamic approach to this blocking by algorithmically reacting to driver inputs and the pace of the interaction in order to prevent drivers from having prolonged or too intense sequences of in-vehicle interactions not directly related to driving. Three simulated driving experiments in Germany and the United States were conducted to evaluate this dynamic function blocking concept and also cater for differences in the status quo of either no blocking or static blocking. The experiments consisted of a car following scenario with various secondary interface tasks and always included a baseline condition where no blocking occurred as well as an implementation of the dynamic function blocking. While Experiments 1 and 3 were aimed at collecting and analyzing gaze and driving data from more than 20 participants, Experiment 2 focused on the user experience evaluation of different visual feedback implementations from 13 participants. The user experience as rated by these participants increased throughout the course of all three studies and helped further improve both the concept and feedback design. In the experiments the total glance time towards the road was significantly higher in the dynamic function blocking condition compared to the baseline, already accounting for the increase in total task time inherent to the dynamic condition. Participants developed two strategies of interacting with the dynamic function blocking. They either operated at their normal baseline speed and incurred task blockings or operated slower to avoid the blockings. In the latter strategy, participants chunked their interactions into smaller steps with the present data suggesting that they used the pauses in between chunks to look back onto the road ahead. Theoretical and practical implications of this first evaluation of a dynamic function blocking concept are discussed.     

The impact of Primacy/Recency Effects and Hazard Monitoring on attributions of other drivers

The present study examined the impact of Primacy/Recency Effects and Hazard Monitoring on driver attributions. Participants viewed a simulated near collision from the perspective of a trailing motorist. The amount of error free driving prior to the near collision varied between two groups, where the near incident occurred either early or later in their viewing experience. They were then given the opportunity to provide judgments of the offending driver based on how safe, dangerous, risky, and skilled the driver was in general, and to evaluate their overall performance. Results showed a Primacy Effect dominance in that judgments of the driver were most negative in the early group, but this was moderated by high Hazard Monitoring for ratings of “dangerous” and “safe”. This suggests that judgments of other drivers are likely to be quick and based on early information, but are impacted by personal factors such as a tendency to monitor for hazards.