Does it deliver what it promises? Evaluation of cognitive distraction caused by speech-based interfaces with detection response and box task

Speech is considered a promising modality for human-machine interaction while driving, especially in reducing visual and manual distraction. However, speech-based user interfaces themselves have shown to increase cognitive distraction. There remains a lack of standardized and unambiguous methods for measuring the impact of speech-based assistants on cognitive distraction while driving. This work aims to investigate whether the combination of the box task and the detection response task (DRT) is a suitable method for assessing the cognitive distraction caused by speech-based assistants. For this purpose, participants (N = 39) engaged in artificial (n-back tasks) and natural speech-based secondary tasks (interaction with Android’s Google Assistant and Apple's Siri) differing in predefined levels of cognitive workload while performing the box task and the DRT. The results showed that DRT performance differed between the 0-back and 1-back task but not between the different cognitive workload levels of the speech-based assistants. No clear effects emerged for the box task parameters. Thus, the combination of the box task and DRT is well-suited for measuring cognitive distraction caused by artificial secondary tasks but not by natural interactions with speech-based assistants.     

The cognitive load of narrative lies

Lying is assumed to increase cognitive load, and it has been shown to slow response times to simple questions. We employed a dual‐task methodology, the detection response task (DRT), to assess cognitive load in telling narrative lies in a live, open‐question interview. The DRT requires participants to press a button in response to a tactile stimulus every 3–5 s while performing a primary task, in this case, recounting either truthful or deceitful narratives. We found weak support for slowing in the time to initiate a narrative response when lying. In contrast, we found strong support for an increase in cognitive load when producing a narrative lie, as measured by both slowed DRT responses and increased response omissions, although this effect decreased with time on task. We advocate dual‐task methodologies such as the DRT for increasing understanding of the assumptions made by theories of deception and for refining lie detection techniques.     

Dual-task decrements in driving performance: The impact of task type,working memory,and the frequency of task performance

Driving while carrying out another (secondary) task interferes with performance, though the degree of interference may vary between tasks and individual drivers. In this study, we focused on two potentially interrelated individual difference variables that may play a role in determining dual-task interference: working memory capacity and the driver’s experience with the relevant secondary task. We used a driving simulator to measure interference, comparing single-task performance (driving alone) with driving performance during three secondary tasks: conversing on a handsfree cellphone, texting, and selecting a song on a touchscreen Mp3 player. Drivers also rated the difficulty of driving while carrying out each secondary task. For the individual difference variables, working memory was measured using the Operation Span test (OSPAN), and experience was assessed in terms of self-reported daily driving exposure and exposure to the relevant secondary tasks (frequency, duration). Overall, we found evidence of dual-task interference, though interference varied between tasks; the texting and Mp3 tasks produced significantly more interference than handsfree cellphone conversation. For the texting and Mp3 song selection tasks, interference was apparent in terms of increased steering variability, but for the Mp3 task there was also compensatory slowing, with drivers slowing down while carrying out the task. OSPAN performance and daily driving exposure were both covariates in predicting the amount of dual-task interference. However, our results suggest that in all but two cases, both involving the texting task, the effects of the OSPAN and the driving and secondary task exposure variables were independent rather than interrelated.     

Measuring drivers’ takeover performance in varying levels of automation: Considering the influence of cognitive secondary task

Mixed control by driver and automated system will remain in use for decades until fully automated driving is perfected. Thus, drivers must be able to accurately regain control of vehicles in a timely manner when the automated system sends a takeover request (TOR) at its limitation. Therefore, determining the factors that affect drivers’ takeover quality at varying levels of automated driving is important. Previous studies have shown that visually distracting secondary tasks impair drivers’ takeover performance and increase the subjective workload. However, the influence of purely cognitive distracting secondary tasks on drivers’ takeover performance and how this influence varies at different levels of automation are still unknown. Hence, a 5 (driving modes) × 3 (cognitive secondary tasks) factorial design with the within-subject factors was adopted for this driving simulator experiment. The sample consisted of 21 participants. The participants’ subjective workloads were recorded by the NASA-Task Load Index (NASA-TLX). Results showed that compared to manual driving conditions, the drivers’ subjective workloads were significantly reduced in both partially and highly automated driving conditions, even with a TOR, confirming the benefit of the automated driving system in terms of reducing the driving workload. Moreover, the drivers exhibited a lower takeover behavior quality at high levels of automation than manual driving in terms of increased reaction time, abnormal performance, standard deviation of lane position, lane departure probability, and reduced minimum of time to collision. However, at the highly automated driving condition, the drivers’ longitudinal driving safety and ability to follow instructions improved when performing a highly cognitive secondary task. This phenomenon possibly occurred because automated driving conditions lead to an underload phenomenon, and the execution of highly cognitive tasks transfers drivers into moderate load, which helps with the drivers’ takeover performance.     

Cognitive function and young drivers: The relationship between driving,attitudes, personality and cognition

Young drivers (aged 17–25 years) are the highest risk age group for driving crashes and are over-represented in car crash statistics in Australia. A relationship between cognitive functioning and driving in older drivers (60 years and older) has been consistently supported in previous literature, however, this relationship has been neglected in research regarding younger drivers. The role of cognitive functioning in young people’s driving was investigated both independently and within a current model of younger peoples driving performance. With young drivers as participants, driving behaviour, attitudes, personality and cognitive functioning were tested and driving performance was operationalised through two measures on a driving simulator, speeding and lane deviations. Cognitive functioning was found to contribute to driving behaviour, along with driving attitudes and personality traits, in accounting for young people’s driving performance. The young drivers who performed better on cognitive functioning tasks engaged in less speeding behaviour and less lane deviation on the driving simulator than those who performed worse on these tasks. This result was found independent of the role of driving behaviour, driving attitudes and personality traits, accounting for unique variance in driving ability.     

Emotion and object processing in Parkinson’s disease

The neuropsychological literature on the processing of emotions in Parkinson’s disease (PD) reveals conflicting evidence about the role of the basal ganglia in the recognition of facial emotions. Hence, the present study had two objectives. One was to determine the extent to which the visual processing of emotions and objects differs in PD. The other was to assess the impact of cognitive load on the processing of these types of information. Thirty-one patients with idiopathic PD (IPD) under dopamine replacement therapy (DRT) were compared to 30 control subjects on emotion and object recognition tasks. Recognition of objects was more accurate and faster than recognition of facial expressions of emotion, for both groups of subjects. In a second experiment using an N-back procedure with the same stimuli—a more demanding task with a higher cognitive load—patients with IPD were as accurate as control subjects in detecting the correct sequential presentation of stimuli, but were much slower in their decision responses. This indicates that IPD patients under DRT are not impaired in encoding emotion or object information, but that they have difficulty with the processing demands of the N-back task. Thus, patients with IPD appear to be more sensitive to cognitive load than to type of information, whether facial emotions or objects. In this perspective, one must consider that a deafferented dopaminergic system has problems processing more complex information before one can posit the existence of deficits affecting a specific type of information.     

The effects of driving environment complexity and dual tasking on drivers’ mental workload and eye blink behavior

The goal of the present study was to assess the effectiveness of eye blink behavior in measuring drivers’ mental workload. Previous research has shown that when mental workload increases with the primary task difficulty, blink frequency drops. On the opposite, the number of blinks increases when a cognitive secondary task has to be performed concurrently. However, the combined effects of the primary task difficulty and dual-tasking on blink rate have not been investigated. The present study was thus designed to vary systematically both the primary driving task and the cognitive secondary task demand to examine their combined effects on blink rate. The driving task was manipulated by varying the complexity of a simulated driving environment. The cognitive load was manipulated using a concurrent simple reaction time task or a complex calculation task. The results confirmed that eye blink frequency was a sensitive measure to elicit increased mental workload level coming from the driving environment. They also confirmed that blink rate increased with the introduction of a cognitive secondary task while blink duration was not affected. However, eye blink behavior did not provide a clear mental workload signature when driving task demands and dual-task conditions were varied simultaneously. The overall picture goes against the suitability of eye blink behavior to monitor drivers’ states at least when external and internal demands interact.     

Attention during visual secondary tasks in driving: Adaptation to the demands of the driving task

Visual attention in driving with visual secondary task is compared for two visual secondary tasks. N = 40 subjects completed a 1 h test drive in a motion-base driving simulator. During the drive, participants either solved an externally paced, highly demanding visual task or a self paced menu system task. The secondary tasks were offered in defined critical and non-critical driving situations. Eye movement behavior was analyzed and compared for both tasks. Before starting the secondary tasks, eye movement behavior shows a smaller standard deviation of gaze as well as longer fixation durations for both tasks. The comparison between the two tasks indicates that drivers use the possibilities the self paced task offers: during the secondary task, they monitor the driving scene with longer fixations and show a greater standard deviation of gaze position. Furthermore, independently of the type of secondary task, drivers adapt their eye movement behavior to the demands of the situation. In critical driving situations they direct a larger proportion of glance time to the driving task. Last, the relation between glance behavior and collisions is analyzed. Results indicate that collisions go together with an inadequate distribution of attention during distraction. The results are interpreted regarding the attentional processes involved in driving with visual secondary tasks. Based on the similarities and differences between the two secondary tasks, a cognitive approach is developed which assumes that the control of attention during distraction is based on a mental situational model of the driving situation.     

Perceptual Load Induces Inattentional Blindness in Drivers

Perceptual load theory states that the level of perceptual load in a task predicts the processing of task‐irrelevant information. High perceptual load has been shown to result in increased inattentional blindness; however, there is little evidence that this extends beyond artificial computer‐based tasks to real‐world behavior. In this study, we adapted a typical load‐blindness paradigm for use in a driving simulator. Forty‐two drivers performed a series of gap perception tasks where they judged if their vehicle could fit between two parked vehicles, with the task imposing either low or high perceptual load. Awareness for an unexpected pedestrian or animal at the side of the road was found to be significantly lower in the high perceptual load condition. This study is the first to demonstrate perceptual load effects on awareness in an applied setting and has important implications for road safety and future applied research on the perceptual load model.Copyright © 2016 John Wiley & Sons, Ltd.     

The power and sensitivity of four core driver workload measures for benchmarking the distraction potential of new driver vehicle interfaces

This study evaluated the power and sensitivity of several core driver workload measures in order to better understand their use as a component of future driver distraction potential evaluation procedures of the in-vehicle human machine interface (HMI). Driving is a task that requires visual, manual and cognitive resources to perform. Secondary tasks, such as mobile phone use and interaction with in-built navigation, which load onto any of these three processing resources increase driver workload and can lead to impaired driving. Because workload and distraction potential are interrelated, a comprehensive method to assess driver workload that produces valid and predictive results is needed to advance the science of distraction potential evaluation. It is also needed to incorporate into New Car Assessment Program (NCAP) testing regimes. Workload measures of cognitive (DRT [Detection Response Task] Reaction Time), visual (DRT Miss Rate), subjective (NASA-TLX [driver workload questionnaire]), and temporal demand (Task Interaction Time) were collected as participants drove one of 40 vehicles while completing a variety of secondary tasks with varying interaction requirements. Of the evaluated measures, variance and power analyses demonstrated that Task Interaction Time is the most sensitive in detecting differences in driver workload between different in-vehicle HMIs, followed by DRT Miss Rate, NASA-TLX and finally DRT Reaction Time. There were relatively weak correlations between each of the four measures. These results suggest that Task Interaction Time, coupled with a reliable visual demand metric such as DRT Miss Rate, eye glance coding, or visual occlusion, more efficiently detect differences in driver workload between different HMIs compared to DRT Reaction Time and the NASA-TLX questionnaire. These results can be used to improve the understanding of the utility of each of these core driver workload measures in assessing driver distraction potential.     

Anticipatory eye movements when approaching a curve on a rural road depend on working memory load

Where do drivers look when approaching curves on a winding road? Existing models on visual processes in curve driving have focused on path-controlling behavior. Another aspect in curve driving is the visual anticipation of potential oncoming vehicles, obstacles and road alignment. We define the occlusion point of a curve as the nearest point where the view of the road is blocked by some obstacle (e.g. vegetation). Monitoring the occlusion point is relevant for safe driving because potential oncoming vehicles or obstacles on the road will come into view on the occlusion point.In the current on-road study, 10 participants drove an instrumented car at their own pace on a low standard rural road while their eye-movements were recorded. We investigated anticipatory glances towards the occlusion point while approaching open sight curves and how anticipatory glances are affected by a cognitive secondary task without explicit visuo-spatial or motor components.The results demonstrate that drivers indeed look at the occlusion point while approaching open curves on rural roads, and that working memory load leads to a significant decrease in visual anticipation. Previously, it has been shown that cognitive secondary tasks lead to reduction of looking at the speedometer and mirrors and of safety critical visual scanning at street crossings. We show that the effect is also present in the anticipation of road curvature and hazards on rural roads.     

Effects of secondary tasks and display position on glance behavior during partially automated driving

The driving task is becoming increasingly automated, thus changing the driver’s role. Moreover, in-vehicle information systems using different display positions and information processing channels might encourage secondary task engagement. During manual driving scenarios, varying secondary tasks and display positions could influence driver’s glance behavior. However, their impact on the driver’s capability to monitor the partially automated driving system has not yet been determined. The current study assessed both the effects of different secondary tasks (Surrogate Reference Task (SuRT) vs. text reading) and display positions (head-up display (HUD) vs. center console) on driver’s glance behavior during partially automated driving in a simulated car following task. Different automation system failures regarding the lateral and longitudinal control occurred while driving. Furthermore, participants’ reported advantages, disadvantages and preferences regarding the investigated display positions as well as regarding the secondary task engagement during partially automated driving in general. Mixed design ANOVAs revealed that the HUD yielded considerably longer eyes-on display time (total and mean glance durations) than the center console. Moreover, the text reading task resulted in longer total and mean glance durations than the SuRT. Similar to manual driving scenarios, the results showed a consistent effect of display position and secondary task on the driver’s glance behavior. Despite the longer eyes-on display time for the HUD, its proximity to the driving environment might enable a faster identification of and reaction to critical situations (e.g., due to system failures). Participants would prefer the HUD as display position compared to the center console. Regarding secondary task engagement during partially automated driving participants seemed to be aware of the benefits but also of the risks.     

Anticipatory and control processes in the interaction with secondary tasks while driving

The paper describes an experiment where anticipatory processes in the interaction with secondary tasks while driving could be explicitly identified and contrasted to control processes during the engagement in the secondary task. A special experimental set-up in a driving simulator environment was created that allows drivers to deliberately decide whether they want to be distracted or not depending on the driving situation and the expected development of that situation. As indicators for a situation-adaptive interaction with secondary tasks parameters from driving behaviour, secondary task performance and visual behaviour were analyzed. A study with 24 test drivers revealed that drivers are, in general, able to interact with a secondary task in a situation-aware manner. For example, drivers rejected more secondary tasks in already highly demanding situations or tried to delay the beginning of the task. During secondary task performance drivers observed the situational development with short control glances back to the road and adapted their speed. The analysis of driving errors revealed that rejecting a task in an already highly demanding driving situation is an effective strategy to maintain an adequate level of driving safety. However, some critical factors were identified that might hinder the driver from executing such strategies. Several recommendations for supporting the driver on this issue are given.     

Active Listening Delays Attentional Disengagement and Saccadic Eye Movements

Successful goal-directed visual behavior depends on efficient disengagement of attention. Attention must be withdrawn from its current focus before being redeployed to a new object or internal process. Previous research has demonstrated that occupying cognitive processes with a secondary cellular phone conversation impairs attentional functioning and driving behavior. For example, attentional processing is significantly impacted by concurrent cell phone use, resulting in decreased explicit memory for on-road information. Here, we examined the impact of a critical component of cell-phone use—active listening—on the effectiveness of attentional disengagement. In the gap task—a saccadic manipulation of attentional disengagement—we measured saccade latencies while participants performed a secondary active listening task. Saccadic latencies significantly increased under an active listening load only when attention needed to be disengaged, indicating that active listening delays a disengagement operation. Simple dual-task interference did not account for the observed results. Rather, active cognitive engagement is required for measurable disengagement slowing to be observed. These results have implications for investigations of attention, gaze behavior, and distracted driving. Secondary tasks such as active listening or cell-phone conversations can have wide-ranging impacts on cognitive functioning, potentially impairing relatively elementary operations of attentional function, including disengagement.     

Distracted driving caused by voice message apps: A series of experimental studies

Hand-free voice message apps are frequently used by young people while driving. Previous studies have identified voice message apps as a common source of driving distraction. To quantitatively evaluate the factors contributing to driving distractions, three simulated driving experiments were designed using a dual-task experimental paradigm. In Experiment 1, participants completed several common tasks related to voice messages in WeChat with or without manual operations (perceptual-motor distraction). Experiments 2 and 3 further took into consideration the cognitive distraction level, measured by task difficulty and task frequency. The results showed that, in comparison with undistracted driving, the perceptual-motor distraction related to voice message app use significantly (ps < 0.05) weakened young drivers’ driving performance with respect to the standard deviation of lateral position (SDLP) between two cars (0.24 m), response time (0.21 s) and error rate (0.12) to turning lights, and collision percentage (0.54%), similar to the effects induced by non-voice-based apps. There were also significant differences (ps < 0.05) between driving with secondary tasks with and without continuous manual operations in the SDLP between two cars (0.19 m) and in the response time (0.18 s) and error rate (0.10) to turning lights, which indicates that the distracting effect produced by voice-message apps comes from the related manual operations. The effects of cognitive distraction on driving performance mainly depended on task difficulty level. High-difficulty secondary tasks via a voice message app significantly (ps < 0.05) weakened the driving performance in response time (by 0.13 s and 0.13 s compared to low-difficulty and baseline conditions, respectively) and error rate (by 0.07 and 0.07 compared to low-difficulty and baseline conditions, respectively) to turning lights and collision percentage (by 0.90% and 0.80% compared to low-difficulty and baseline conditions, respectively). The findings provide a theoretical reference for analysing the distracting components of voice messages and suggest that drivers should limit the use of these kinds of apps during driving.     

Effects of primary task predictability and secondary task modality on lane maintenance

The objective of this work was to investigate the effects of auditory and visuo-spatial secondary tasks on variability in lane position in predictable and unpredictable driving conditions. Sixty-six participants drove a simulated vehicle. Predictability was manipulated by adding wind gusts, and the secondary task load by either an auditory task (Experiment 1) or a visuo-spatial task (Experiment 2). Results demonstrated that in the predictable driving condition, lane position variability decreased when auditory secondary task load was given, but it was not affected by visuo-spatial secondary task load. In the unpredictable driving condition; however, while the auditory secondary task load did not affect lane position variability, visuo-spatial secondary task load increased it. The data suggests that the effects of cognitive load on lane maintenance may depend on the type of the secondary task beside predictability of primary task.     

Secondary task engagement and disengagement in the context of highly automated driving

During highly automated driving (level 3 automation according to SAE International, 2014) people are likely to increase the frequency of secondary task interactions. However, the driver must still be able to take over control within a reasonable amount of time. Previous studies mainly investigated take-over behavior by forcing participants to engage in secondary tasks prior to take over, and barely addressed how drivers voluntarily schedule secondary task processing according to the availability and predictability of automated driving modes. In the current simulator study 20 participants completed a test drive with alternating sections of manual and highly automated driving. One group had a preview on the availability of the automated driving system in upcoming sections of the track (predictive HMI), while the other drivers served as a control group. A texting task was offered during both driving modes and also prior to take-over situations. Participants were free to accept or reject a given task, taking the situational demands into account. Drivers accepted more tasks during highly automated driving. Furthermore, tasks were rejected more often prior to take-over situations in the predictive HMI group. This was accompanied by safer take-over performance. However, once engaged in a task, drivers tended to continue texting even in take-over situations. The results indicate the need to discriminate different aspects of task handling regarding self-regulation: task engagement and disengagement.     

Looking and thinking when driving: The impact of gaze and cognitive load on steering

Driving around bends at high speeds is a task performed by many on a daily basis but the underlying mechanisms of steering control remain largely unknown. Previous research has shown that when steering, gaze direction can be a critical component of success. However, with increased use of in-vehicle information systems (IVIS), there is growing competition over the same resources that are needed to steer (gaze as well as associated attentional resources). Although it can be argued that locomotor steering is an automatic task that can be performed without recourse to conscious “cognitive” control, much simpler locomotor-related tasks, such as judging one’s heading, have been shown to be affected by concurrent attentional tasks (Wann, Swapp, & Rushton, 2000). Here we examined whether an attentional task placed at an offset fixation point influenced concurrent steering performance along a computer simulated road. The experiments either used gaze-fixation points that had similar properties to real-world road signs (i.e. moved relative to the vehicle) or were more akin to IVIS (i.e. fixed to the vehicle). Results showed that gaze fixation eccentric to future path caused systematic steering biases. The degree or type of cognitive load did not change the degree of steering bias, but there was some evidence of decreased lane variability when viewing the IVIS-type displays. No differences in steering performance were found between the different types of cognitive task. We conclude that where you look is critical for safe driving, and IVIS-type displays might make drivers more susceptible to cognitive interference.     

On-to-off-path gaze shift cancellations lead to gaze concentration in cognitively loaded car drivers: A simulator study exploring gaze patterns in relation to a cognitive task and the traffic environment

Appropriate visual behaviour is necessary for safe driving. Many previous studies have found that when performing non-visual cognitive tasks, drivers typically display an increased amount of on-path glances, along with a deteriorated visual scanning pattern towards potential hazards at locations outside their future travel path (off-path locations). This is often referred to as a gaze concentration effect. However, what has not been explored is more precisely how and when gaze concentration arises in relation to the cognitive task, and to what extent the timing of glances towards traffic-situation relevant off-path locations is affected. To investigate these specific topics, a driving simulator study was carried out. Car drivers’ visual behaviour during execution of a cognitive task (n-back) was studied during two traffic scenarios; one when driving through an intersection and one when passing a hidden exit. Aside from the expected gaze concentration effect, several novel findings that may explain this effect were observed. It was found that gaze shifts from an on-path to an off-path location were inhibited during increased cognitive load. However, gaze shifts in the other direction, that is, from an off-path to an on-path location, remained unaffected. This resulted in on-path glances increasing in duration, while off-path glances decreased in number. Furthermore, the inhibited off-path glances were typically not compensated for later. That is, off-path glances were cancelled, not delayed. This was the case both in relation to the cognitive task (near-term) and the traffic environment (far-term). There was thus a general reduction in the number of glances towards situationally relevant off-path locations, but the timing of the remaining glances was unaffected. These findings provide a deeper understanding of the mechanism behind gaze concentration and can contribute to both understanding and prediction of safety relevant effects of cognitive load in car drivers.