Challenges for automated vehicle driver training: A thematic analysis from manual and automated driving

Considerable research and resources are going into the development and testing of Automated Vehicles. They are expected to bring society a huge number of benefits (such as: improved safety, increased capacity, reduced fuel use and emissions). Notwithstanding these potential benefits, there have also been a number of high-profile collisions involving Automated Vehicles on the road. In the majority of these cases, the driver’s inattention to the vehicle and road environment was blamed as a significant causal factor. This suggests that solutions need to be developed in order to enhance the benefits and address the challenges associated with Automated Vehicles. One such solution is driver training. As drivers still require manual driving skills when operating Automated Vehicles on the road, this paper applied the grounded theory approach to identify eight “key” themes and interconnections that exist in current manual vehicle driver training. These themes were then applied to the limited literature available on Automated Vehicle driver training, and a ninth theme of trust emerged. This helped to identify a set of training requirements for drivers of Automated Vehicles, which suggests that a multifaceted approach (covering all nine themes and manual and Automated Vehicle driving skills) to driver training is required. This framework can be used to develop and test a training programme for drivers of Automated Vehicles.     

Keeping the driver in the loop in conditionally automated driving: A perception-action theory approach

In this paper we investigated if keeping the driver in the perception–action loop during automated driving can improve take-over behavior from conditionally automated driving. To meet this aim, we designed an experiment in which visual exposure (perception) and manual control exposure (action) were manipulated. In a dynamic driving simulator experiment, participants (n = 88) performed a non-driving related task either in a head-up display in the windshield (high visual exposure) or on a head-down display near the gear shift (low visual exposure). While driving, participants were either in an intermittent control-mode with four noncritical take-over situations (high manual control exposure), or in a continuous automation-mode throughout the ride (low manual control exposure). In all conditions, a critical take-over had to be carried out after an approximately 13 min ride. Measurements of take-over behavior showed that only high visual exposure had an effect on hands-on reaction time measurements. Both visual exposure and manual control exposure had small to medium sized main effects on time to system deactivation, the maximum velocity of the steering wheel, and the standard deviation of the steering wheel angle. The combined high visual – and high manual control exposure condition led to 0.55 s faster reaction time and 37% less steering variability in comparison to the worst case low visual – and low manual control exposure condition. Together, results corroborate that maintaining visual exposure and manual control exposure during automated driving can be efficacious and suggest that their positive effects are additive.     

Eye movements predict driver reaction time to takeover request in automated driving: A real-vehicle study

For automated driving at SAE level 3 or lower, driver performance in responding to takeover requests (TORs) is decisive in providing system safety. A driver state monitoring system that can predict a driver’s performance in a TOR event will facilitate a safer control transition from vehicle to driver. This experimental study investigated whether driver eye-movement measured before a TOR can predict driving performance in a subsequent TOR event. We recruited participants (N = 36) to obtain realistic results in a real-vehicle study. In the experiment, drivers rode in an automated vehicle on a test track for about 32 min, and a critical TOR event occurred at the end of the drive. Eye movements were measured by a camera-based driver monitoring system, and five measures were extracted from the last 2-min epoch prior to the TOR event. The correlations between each eye-movement measure and driver reaction time were examined, and a multiple regression model was built using a stepwise procedure. The results showed that longer reaction time could be significantly predicted by a smaller number of large saccades, a greater number of medium saccades, and lower saccadic velocity. The implications of these relationships are consistent with previous studies. The present real-vehicle study can provide insights to the automotive industry in the search for a safer and more flexible interface between the automated vehicle and the driver.     

A driving simulator study to assess driver performance during a car-following maneuver after switching from automated control to manual control

A review of the literature on autonomous vehicles has shown that they offer several benefits, such as reducing traffic congestion and emissions, and improving transport accessibility. Until the highest level of automation is achieved, humans will remain an important integral of the driving cycle, which necessitates to fully understand their role in automated driving. A difficult research topic involves an understanding of whether a period of automated driving is likely to reduce driver fatigue rather than increase the risk of distraction, particularly when drivers are involved in a secondary task while driving. The main aim of this research comprises assessing the effects of an automation period on drivers, in terms of driving performance and safety implications. A specific focus is set on the car-following maneuver. A driving simulator experiment has been designed for this purpose. In particular, each participant was requested to submit to a virtual scenario twice, with level-three driving automation: one drive consisting of Full Manual Control Mode (FM); the other comprising an Automated Control Mode (AM) activated in the midst of the scenario. During the automation mode, the drivers were asked to watch a movie on a tablet inside the vehicle. When the drivers had to take control of the vehicle, two car-following maneuvers were planned, by simulating a slow-moving vehicle in the right lane in the meanwhile a platoon of vehicles in the overtaking lane discouraged the passing maneuver. Various driving performances (speeds, accelerations, etc.) and surrogate safety measures (PET and TTC) were collected and analysed, focusing on car-following maneuvers. The overall results indicated a more dangerous behavior of drivers who were previously subjected to driving automation; the percentage of drivers who did not apply the brakes and headed into the overtaking lane despite the presence of a platoon of fast-moving vehicles with unsafe gaps between them was higher in AM drive than in FM drive. Conversely, for drivers who preferred to brake, it was noted that those who had already experienced automated driving, adopted a more careful behavior during the braking maneuver to avoid a collision. Finally, with regard to drivers who had decided to overtake the braking vehicle, it should be noted that drivers who had already experienced automated driving did not change their behavior whilst overtaking the stopped lead vehicle.     

Physiological indicators of driver workload during car-following scenarios and takeovers in highly automated driving

This driving simulator study, conducted as a part of Horizon2020-funded L3Pilot project, investigated how different car-following situations affected driver workload, within the context of vehicle automation. Electrocardiogram (ECG) and electrodermal activity (EDA)-based physiological metrics were used as objective indicators of workload, along with self-reported workload ratings. A total of 32 drivers were divided into two equal groups, based on whether they engaged in a non-driving related task (NDRT) during automation (SAE Level 3) or monitored the drive (SAE Level 2). Drivers in both groups were exposed to two counterbalanced experimental drives, lasting ∼ 18 min each, of Short (0.5 s) and Long (1.5 s) Time Headway conditions during automated car-following (ACF), which was followed by a takeover that happened with or without a lead vehicle. Results showed that driver workload due to the NDRT was significantly higher than both monitoring the drive during ACF and manual car-following (MCF). Furthermore, the results indicated that a lead vehicle maintain a shorter THW can significantly increase driver workload during takeover scenarios, potentially affecting driver safety. This warrants further research into understanding safe time headway thresholds to be maintained by automated vehicles, without placing additional cognitive or attentional demands on the driver. Our results indicated that ECG and EDA signals are sensitive to variations in workload, which warrants further investigation on the value of combining these two signals to assess driver workload in real-time, to help future driver monitoring systems respond appropriately to the limitations of the driver, and predict their performance in the driving task, if and when they have to resume manual control of the vehicle after a period of automated driving.     

Pedestrian detection using a moving camera: A novel framework for foreground detection

While background subtraction techniques have been widely applied to detect moving objects in a video stream captured by a static camera, detecting moving objects using a moving camera still represents a challenging task. In this context, pedestrian detection using a camera placed on the top of a vehicle’s windshield has been rarely investigated. This is mainly due to the background ego-motion. Since the scene captured by the camera seems in motion, it is very difficult to distinguish the moving pedestrians from the others that belong to the static part of the scene. For this reason, a compensation step is needed to suppress the ego-motion. This paper presents a study on the main challenges facing pedestrian detection systems as well as methods proposed to handle these challenges. A novel trajectory classification framework for detecting pedestrians even in challenging real-world environments is proposed. The proposed method models the background motion between two consecutive frames in order to compensate the camera motion. Then, it defines a classification process that differentiates between the background and the foreground in the frame. Using the defined foreground, we consequently identify the presence of pedestrians in the scene. The proposed method was validated on a public benchmark dataset: CVC-14 containing both visible and far infrared video sequences in day and night times. Experimental results confirm the effectiveness of the proposed approach in capturing the dynamic aspect between frames and therefore detecting the presence of pedestrians in the scene.     

The role of psychological factors on the choice of different driving controls: On manual,partial, and highly automated controls

The present study investigates the role of psychological factors on the choice of three controls (modes) in driving a vehicle, namely highly automated, partially automated, and manual control. Traditional driving habits, resistance to change, and behavioural beliefs were all assessed along with individual and socioeconomic variables. Using survey data (n = 595) of car users, a model was developed to predict the share of different driving controls and determine the effects of psychological variables. Results indicate that up to 55% of people prefer driving with highly automated control, and 30% prefer partially automated control. Behavioural beliefs (e.g., attitudes toward highly automated control) are not as critical to driving control as habits. People with stronger driving habits are less likely to use highly automated controls. A one-unit increase in worry could reduce driving in highly automated control by 5.5% and increase manual control by 4.5%, and those who welcome the new technologies are more likely to prefer highly automated control. Some practical policy solutions are also provided.     

Driver brake vs. steer response to sudden forward collision scenario in manual and automated driving modes

In autonomous vehicle operation, situations may arise when the driver is required to re-engage in manual control of the vehicle. Whether the control handoff from vehicle to human is done in a structured or unstructured manner, the process may be affected by the driver’s state, i.e. distracted or not. The study reported here was designed to measure a non-distracted driver’s response to a sudden forward collision (FC) event, in which the driver would assume manual control of the autonomous vehicle. Three driving scenarios were investigated: autonomous vehicle driven with full collision avoidance support, autonomous vehicle driven without collision avoidance support, and vehicle driven in manual mode.Forty-eight volunteers participated in a simulator study conducted in VIRTTEX. It was found that, at handoff, (1) drivers in manual mode tended to use evasive steering, rather than braking, compared to drivers in both the autonomous modes, (2) between subjects variations in speed were higher for the automation with collision support condition than for the other two scenarios, (3) for both autonomous driving scenarios, drivers reaction times were longer than for manual driving. In some cases the driver response was so late and the distance remaining so reduced that crash avoidance might be unfeasible. At a minimum, results of this study suggest that drivers may benefit from appropriate driver assistance technologies when a crash imminent situation is suddenly encountered.     

Effect of multimodal takeover request issued through A-pillar LED light,earcon, speech message,and haptic seat in conditionally automated driving

The driver of a conditionally automated vehicle equivalent to level 3 of the SAE is obligated to accept a takeover request (TOR) issued by the vehicle. Considerable research has been conducted on the TOR, especially in terms of the effectiveness of multimodal methods. Therefore, in this study, the effectiveness of various multimodalities was compared and analyzed. Thirty-six volunteers were recruited to compare the effects of the multimodalities, and vehicle and physiological data were obtained using a driving simulator. Eight combinations of TOR warnings, including those implemented through LED lights on the A-pillar, earcon, speech message, or vibrations in the back support and seat pan, were analyzed to clarify the corresponding effects. When the LED lights were implemented on the A-pillar, the driver reaction was faster (p = 0.022) and steering deviation was larger (p = 0.024) than those in the case in which no LED lights were implemented. The speech message resulted in a larger steering deviation than that in the case of the earcon (p = 0.044). When vibrations were provided through the haptic seat, the reaction time (p < 0.001) was faster, and the steering deviation (p = 0.001) was larger in the presence of vibrations in the haptic seat than no vibration. An interaction effect was noted between the visual and auditory modalities; notably, the earcon resulted in a small steering deviation and skin conductance response amplitude (SCR amplitude) when implemented with LED lights on the A-pillar, whereas the speech message led to a small steering deviation and SCR amplitude without the LED lights. In the design of a multimodal warning to be used to issue a TOR, the effects of each individual modality and corresponding interaction effects must be considered. These effects must be evaluated through application to various takeover situations.     

Driver trust in and training for advanced driver assistance systems in Real-World driving

Two-hundred and twenty-three participants completed an online survey regarding their experiences with advanced driver assistance systems (ADAS) on their personal vehicles, with focus on 1) drivers’ trust in 13 ADAS technologies, and 2) perceived effectiveness of currently used methods of training. Eighteen drivers participated in focus groups designed to probe more deeply into survey responses. Results of the survey showed that participant ratings of trust increased significantly with longer vehicle ownership, but participants who experienced unexpected ADAS technology behavior rated their trust over time significantly lower on ADAS technologies with the exception of rear collision avoidance. The majority (75.8%) of participants reported receiving some ADAS instruction at their vehicle dealership, but only 16.6% indicated it was formal. Participants who received formalized training reported it to be significantly more effective than those who received informal overviews of their systems. Use of trial and error and the owner’s manual were the most frequently reported methods of learning outside of dealership training. Responses indicated that the lack of content tailored to trim-specific vehicle features in owner’s manuals was a barrier to effective use.     

The impact of a dedicated lane for connected and automated vehicles on the behaviour of drivers of manual vehicles

Connected and automated vehicles (CAVs) are expected to enhance traffic efficiency by driving at shorter time headways, and traffic safety by shorter reaction times. However, one of the main concerns regarding their deployment is the mixed traffic situation, in which CAVs and manually driven vehicles (MVs) share the same road.This study investigates the behavioural adaptation of MV drivers in car-following and lane changing behaviour when they drive next to a dedicated lane (DL) for CAVs and compares that to a mixed traffic situation. The expectation is that in a mixed traffic situation, the behavioural adaptation of MV drivers is negligible due to lower exposure time and scarce platoons, while concentrating the CAVs on one dedicated lane may cause significant behavioural adaptation of MV drivers due to a higher exposure time and conspicuity of CAV platoons.Fifty-one participants were asked to drive an MV on a 3-lane motorway in three different traffic scenarios, in a fixed-base driving simulator: (1) Base, only MVs were present in traffic, (2) Mixed, platoons of 2–3 CAVs driving on any lane and mixed with MVs, (3) DL, platoons of 2–3 CAVs driving only on a DL. The DL was recognizable by road signs and a buffer demarcation which separated the DL from the other lanes. A moderate penetration rate of 43% was assumed for CAVs.During the drives, the car following headways and the accepted merging gaps by participants were collected and used for comparisons of driving behaviour in different scenarios.Based on the results, we conclude that there is no significant difference in the driving behaviour between Base and Mixed scenarios at tested penetration rate, confirming our research expectation. However, in DL scenario, MV drivers drove closer to their leaders specially when driving on the middle lane next to the platoons and accepted shorter gaps (up to 12.7% shorter at on-ramps) in lane changing manoeuvres. Dedicating a lane to CAVs increases the density of CAV platoons on one lane and consequently their conspicuity becomes higher. As a result, MV drivers are influenced by CAV platoons on a DL and imitate their behaviour.The literature suggests that dedicating a lane to CAVs improves the traffic efficiency by providing more possibilities for platooning. This study shows that implementing such a solution will affect the driving behaviour of human drivers. This should be taken into consideration when evaluating the impacts of dedicated lanes on traffic efficiency and traffic safety.     

Learning from experience: Familiarity with ACC and responding to a cut-in situation in automated driving

Response times to risky events have been seen to increase with the use of adaptive cruise control (ACC). It has been unclear whether driver experience with ACC mediates this increase. We compare driving in a cut-in event in a simulator both with and without system support, studying ACC as well as ACC with automatic steering. 31 participants were tested in a repeated-measures design, 10 novices and 21 previously experienced with ACC. There was no difference between responding to ACC with and without automatic steering for either group. As expected, we found an increase in response times when driving with system support for both ACC-experienced drivers and ACC-novices. However, this effect was significantly lower for those previously experienced with ACC. This indicates that there is an element of learning involved not only in knowing about system limitations, but also in responding to potential hazards.     

Self-concept as a risky driver: Mediating the relationship between racing video games and on-road driving violations in a community-based sample

PurposeThe mediating relationship of self-conceptions as a risky driver on self-reported driving violations was examined for players of “drive’em up” and “circuit” racing video games using an Internet survey of automobile and racing club members. Structural equation modelling (SEM) tested Fischer et al. (2012) extended socio-cognitive model on the effects of risk-glorifying media on cognitions and actions.MethodAn Internet questionnaire was developed and relied upon validated instruments or questions derived from previous surveys. Driver club members were asked about: (1) their frequency of video game playing, (2) self-perceptions as a risky driver and (3) self-reported driving violations. SEM was performed to examine mediating effects of racing video game playing on self-reported driving violations.ResultsPlaying “drive‘em up” video games positively predicted risky self-concept (β = .15, t = 2.26), which in turn, positively predicted driving violations (β = .73, t = 8.63), while playing “circuit racing” games did not predict risky self-concept, although risky self-concept did predict driving violations (β = .72, t = 8.67).ConclusionsSelf-concept as a risky driver mediated the relationship between racing video game playing and self-reported driving violations for “drive’em up”, but not for “circuit racing” video games. These findings are congruent with Fischer and colleagues’ experimental model that self-concept as a reckless driver mediated the relationship between racing video game playing for “drive’em up”, but not for “circuit racing” games and risk-taking behavior in a video of road traffic scenarios.     

Effect of a retention interval between pre-exposure and conditioning on latent inhibition in humans using a blink conditioning procedure

Latent inhibition, retarded learning after pre-exposure to the to-be-conditioned stimulus, was examined using a blink conditioned procedure in humans. Experiment 1 showed that the procedure is suited to inducing the latent inhibition effect. In Experiment 2, the introduction of a 3-minute interval between pre-exposure and conditioning phases attenuated latent inhibition. These results contribute to identify the mechanisms involved in pre-exposure and subsequent conditioning of a stimulus, which is particularly important if we bear in mind that latent inhibition has been used repeatedly as an instrument to analyze the course of attentional processes in normal and pathological populations.     

The different effects of personality on prosocial and aggressive driving behaviour in a Chinese sample

Dangerous driving behaviours, as a direct cause of accidents and death, are the focus of considerable research attention. However, unlike unsafe driving behaviours, few studies have explored safe driving behaviours and their effects on road traffic. This study aims to verify the Chinese version of the Prosocial and Aggressive Driving Inventory (PADI) and then investigate the relationship between personality and aggressive/prosocial driving behaviours. A total of 303 licensed drivers were recruited, and they voluntarily and anonymously completed the PADI, the Driving Behaviours Questionnaire (DBQ), and personality scales (anger, sensation-seeking and altruism). The results of this research confirmed the reliability and validity of the Chinese PADI. Most importantly, it was found that different relationships between different personalities and aggressive/prosocial driving behaviours. Specifically, individuals with high altruism exhibited more prosocial driving behaviours, while individuals with high sensation seeking presented more aggressive driving behaviours. The importance of these findings lies in two main potential implications: developing an effective measurement of prosocial driving behaviours in China and providing favourable evidence to guide drivers toward more prosocial driving behaviours.     

Fast face detection using a unified architecture for unconstrained and infrared face images

Human face detection along with its localization is a difficult task when the face is presented in the cluttered scene in an unconstrained scenario that might be with arbitrary pose variations, occlusions, random backgrounds and infrared (IR) environment. This paper proposes a novel face detection method which can address some of these issues and challenges quite successfully during face detection in unconstrained as well as infrared environments. It makes use of Fast Successive Mean Quantization Transform (FastSMQT) features for image enhancement and feature representation to deal with illumination and sensor insensitiveness. A split up Sparse Network of Winnows (SNoW) with Winnow updating rule is then exploited to speed up the original SNoW classifier. Finally, the features and classifiers are combined together with skin detection algorithm for face detection in crowd image and head orientation correction of near infrared faces. The proposed face detector is robust in handling pose, occlusion, illumination, blur and low image resolution. The experiment is performed on six challenging and publicly available databases, viz. BIOID, LFW, FDDB, UFI, WIDER FACE and IIT Delhi near infrared. The experimental results depict that the proposed method outperforms traditional as well as some advanced methods in detecting unconstrained and infrared faces under challenging situations.     

Effects of response bias and judgment framing on operator use of an automated aid in a target detection task

Automated diagnostic aids prone to false alarms often produce poorer human performance in signal detection tasks than equally reliable miss-prone aids. However, it is not yet clear whether this is attributable to differences in the perceptual salience of the automated aids' misses and false alarms or is the result of inherent differences in operators' cognitive responses to different forms of automation error. The present experiments therefore examined the effects of automation false alarms and misses on human performance under conditions in which the different forms of error were matched in their perceptual characteristics. Young adult participants performed a simulated baggage x-ray screening task while assisted by an automated diagnostic aid. Judgments from the aid were rendered as text messages presented at the onset of each trial, and every trial was followed by a second text message providing response feedback. Thus, misses and false alarms from the aid were matched for their perceptual salience. Experiment 1 found that even under these conditions, false alarms from the aid produced poorer human performance and engendered lower automation use than misses from the aid. Experiment 2, however, found that the asymmetry between misses and false alarms was reduced when the aid's false alarms were framed as neutral messages rather than explicit misjudgments. Results suggest that automation false alarms and misses differ in their inherent cognitive salience and imply that changes in diagnosis framing may allow designers to encourage better use of imperfectly reliable automated aids.     

Commentary: a different lens for working with affairs: using social constructionist and attachment theory

A combined social constructivist and attachment theory approach can help in work with affairs. Using past empirical research, a case is made that vulnerability to affairs derives primarily from pressures on partnerships stemming from the dominant discourse of modern partnerships. This discourse exalts sharing and joining, subjugating the more common experience of couples that other meaningful relationships (including non‐sexual ones) augment and fill in gaps. Affairs may be seen as, perhaps misguided, attempts to address untenable goals of the dominant discourse. Concomitantly, attachment theory explains both the need to be in a variety of relationships, normally within a hierarchy (‘multiple attachments’) and also the extreme wounds attendant to the discovery of secret affairs. Using both these perspectives can be useful preceding attempts at forgiveness. It is argued that they help therapists to establish an empathic positioning toward both partners in the couple, rather than the moralistic one too frequently attendant in work with affairs.     

Response facilitation and inhibition in manual, vocal, and oculomotor performance: evidence for a modality-unspecific mechanism

In 2 experiments (N = 10, Experiment 1; N = 16, Experiment 2), the authors investigated whether evidence for response facilitation and subsequent inhibition elicited by masked prime stimuli can be observed for output modalities other than manual responding. Masked primes were followed by target stimuli that required a 2-choice manual, saccadic, or vocal response. Performance was measured for compatible trials in which primes and targets were identical and for incompatible trials in which they were mapped to opposite responses. When primes were presented centrally, performance benefits were obtained for incompatible trials; whereas for peripherally presented primes, performance benefits were found in compatible trials. That pattern of results was obtained for manual responses and for saccadic eye movements (Experiment 1), demonstrating that those effects are not mediated by specialized dorsal pathways involved in visuomanual control. An analogous pattern of effects was found when manual and vocal responses were compared (Experiment 2). Because vocal responding is controlled by the inferotemporal cortex, that result shows that prime-target compatibility effects are not primarily mediated by the dorsal stream, but appear to reflect modality-unspecific visuomotor links that allow rapid activation of motor responses that may later be subject to inhibition.     

Inspecting pictures for information to verify a sentence: Eye movements in general encoding and in focused search

When we see combinations of text and graphics, such as photographs and their captions in printed media, how do we compare the information in the two components? Two experiments used a sentence-picture verification task in which statements about photographs of natural scenes were read in order to make a true/false decision about the validity of the sentence, and in which eye movements were recorded. In Experiment 1 the sentence and the picture were presented concurrently, and objects and words could be inspected in any order. In Experiment 2 the two components were presented one after the other, either picture first or sentence first. Fixation durations on pictures were characteristically longer than those on sentences in both experiments, and fixations on sentences varied according to whether they were being encoded as abstract propositions or as coreferents of objects depicted in a previously inspected picture. The decision time data present a difficulty for existing models of sentence verification tasks, with an inconsistent pattern of differences between true and false trials.