NASA TLX: Software for assessing subjective mental workload

The NASA Task Load Index (TLX) is a popular technique for measuring subjective mental workload. It relies on a multidimensional construct to derive an overall workload score based on a weighted average of ratings on six subscales: mental demand, physical demand, temporal demand, performance, effort, and frustration level. A program for implementing a computerized version of the NASA TLX is described. The software version assists in simplifying collection, postprocessing, and storage of raw data. The program collects raw data from the subject and calculates the weighted (or unweighted) workload score, which is output to a text file. The program can also be tailored to a specific experiment using a simple input text file, if desired. The program was designed in Visual Studio 2005 and is capable of running on a Pocket PC with Windows CE or on a PC with Windows 2000 or higher. The NASA TLX program is available for free download.     

The effect of gender and time-of-day on time perception and mental workload

Two experiments are reported which investigated how subject gender and time-of-day influenced the estimation of duration and the perception of task-related mental workload. In the first experiment, 24 subjects performed a filled time-estimation task in a constant blacked-out, noise-reduced environment at 0800h, 1200h, 1600h, and 2000h, respectively. In the second experiment, 12 different subjects performed an unfilled time estimation task in similar conditions at 0900h, 1400h, and 1900h. At the termination of all experimental sessions, participants completed the NASA Task Load Index workload assessment questionnaire as a measure of perceived mental workload. Results indicated that physiological response, reflected in body temperature change, followed an expected pattern of sequential increase with time-of-day. However, estimates of duration and the perception of mental workload showed no significant effects for time-of-day. In each of the experiments there were significant differences in time estimation and mental workload response contingent on the gender of the participant. These results are interpreted in light of the previous positive findings for circadian fluctuation in performance efficiency and the equivocal findings of a gender difference in time estimation. A unifying account of these collective results is given based on gender by time-of-day interactional effects. An earlier version of this paper was presented at the 32nd Annual Meeting of the Human Factors Society, Anaheim, CA, October, 1988.     

The effect of gender and time-of-day on time perception and mental workload

Two experiments are reported which investigated how subject gender and time-of-day influenced the estimation of duration and the perception of task-related mental workload. In the first experiment, 24 subjects performed a filled time-estimation task in a constant blacked-out, noise-reduced environment at 0800h, 1200h, 1600h, and 2000h, respectively. In the second experiment, 12 different subjects performed an unfilled time estimation task in similar conditions at 0900h, 1400h, and 1900h. At the termination of all experimental sessions, participants completed the NASA Task Load Index workload assessment questionnaire as a measure of perceived mental workload. Results indicated that physiological response, reflected in body temperature change, followed an expected pattern of sequential increase with time-of-day. However, estimates of duration and the perception of mental workload showed no significant effects for time-of-day. In each of the experiments there were significant differences in time estimation and mental workload response contingent on the gender of the participant. These results are interpreted in light of the previous positive findings for circadian fluctuation in performance efficiency and the equivocal findings of a gender difference in time estimation. A unifying account of these collective results is given based on gender by time-of-day interactional effects. An earlier version of this paper was presented at the 32nd Annual Meeting of the Human Factors Society, Anaheim, CA, October, 1988.     

State anxiety, performance accuracy, and work pace in a simulated pharmacy dispensing task

This study examined the effects of scores on state anxiety on overall accuracy of performance and speed of working in a simulated pharmacy dispensing task. 75 undergraduates worked in a simulated pharmacy environment, designed by Schell and Grasha in 1998, to fill 42 mock orders for simulated pharmacy items. Participants' accuracy and work pace in the simulation, presimulation stress, and postsimulation perceived workload and state anxiety were measured. Analysis indicated that state anxiety and overall accuracy were strongly related. State anxiety appears to be one of the best predictors of errors in the simulated pharmacy dispensing task found so far, while the lack of relationship between work pace and accuracy was confirmed. Work pace predicted accuracy, indirectly, but only after statistically removing the effects of anxiety, task frustration, significant-other stress, and grade point average.     

The impact of navigation system display size and environmental illumination on young driver mental workload

Driver distraction is a major cause of road crashes and has a great influence on road safety. In vehicles, one of the common distracting sources is navigation systems (NSs). The navigation system (NS) can distract the driver due to following directions and reading the provided information through its display. These tasks take the driver’s attention from the primary task of driving and may cause poor driving performance, increasing the risk of crashes. In this paper, the effect of the environment (i.e., urban areas and rural areas), the navigation system display (NSD) size, environmental illumination, and gender on young drivers between the ages of 18 and 29 years mental workload was investigated using a simulated driving experiment. To evaluate each driving condition, the NASA-TLX (NASA Task Load Index) workload assessment tool, and a distraction evaluation element, were introduced and used to assess the overall workload, the workload subscales and the distraction by the NSD. The assessment showed a higher perceived overall workload for urban areas and night driving as compared to a rural areas and daytime driving. Moreover, the results showed a greater perceived distraction by the NSD in urban areas compared to driving in rural areas. The subjects also felt distracted when using the small NS compared to using the large NS. The study concluded that urban areas driving, and night driving creates higher perceived workload than rural areas and daytime driving. Furthermore, small NSD leads to more perceived distraction than large NSD while driving. The NSD designers may utilize this research findings to optimize NSD designs to improve driving safety, performance and comfort. Moreover, this study contributes to our understanding of the effect of the NSD size on driving workload and distraction.     

The Stationary-Gaze Task Should Not Be Systematically Used as the Control Task in Studies of Postural Control

In studies of postural control, a control task is often used to understand significant effects obtained with experimental manipulations. This task should be the easiest task and (therefore) engage the lowest behavioral variability and cognitive workload. Since 1983, the stationary-gaze task is considered as the most relevant control task. Instead, the authors expected that free looking at small targets (white paper or images; visual angle: 12°) could be an easier task. To verify this assumption, 16 young individuals performed stationary-gaze, white-panel, and free-viewing 12° tasks in steady and relaxed stances. The stationary-gaze task led to significantly higher cognitive workload (mean score in the National Aeronotics and Space Administration Task Load Index questionnaire), higher interindividual body (head, neck, and lower back) linear variability, and higher interindividual body angular variability—not systematically yet—than both other tasks. There was more cognitive workload in steady than relaxed stances. The authors also tested if a free-viewing 24° task could lead to greater angular displacement, and hence greater body sway, than could the other tasks in relaxed stance. Unexpectedly, the participants mostly moved their eyes and not their body in this task. In the discussion, the authors explain why the stationary-gaze task may not be an ideal control task and how to choose this neutral task.     

Comparing eye-tracking metrics of mental workload caused by NDRTs in semi-autonomous driving

The objective of this study was to verify the effectiveness of eye-tacking metrics in indicating driver’s mental workload in semi-autonomous driving when the driver is engaged in different non-driving related tasks (NDRTs). A driving simulator was developed for three scenarios (high-, medium-, and low-mental workload presented by SAE (Society of Automotive Engineers) Levels 0, 1, and 2) and three uni-modality secondary tasks. Thirty-six individuals participated in the driving simulation experiment. NASA-TLX (Task Load Index), secondary task performance, and eye-tracking metrics were used as indicators of mental workload. The subjective rating using the NASA-TLX showed a main effect of autonomous level on mental workload in both visual and auditory tasks. Correlation-matrix calculation and principal-component extraction indicated that pupil diameter change, number of saccades, saccade duration, fixation duration, and 3D gaze entropy were effective indicators of a driver’s mental workload in the visual and auditory multi-tasking situations of semi-autonomous driving. The accuracy of predicting the mental-workload level using the K-Nearest Neighbor (KNN) classifier was 88.9% with bootstrapped data. These results can be used to develop an adaptive multi-modal interface that issues efficient and safe takeover requests.     

Driving mental workload and performance of ageing drivers

The World Health Organization stated that the global ageing population is increasing rapidly as well as the case of road accidents involving ageing drivers. This study presents the driving mental workload and performance model of ageing drivers in the context of real-time road driving. Twenty paid participants (ten males) with a mean age of 57.8 years old (SD = 2.7) and mean driving experience of 29.6 years (SD = 8.5) took part in driving experiments with three complexity levels of situation: simple situation (SS), moderately complex situation (MCS), and very complex situation (VCS). The subjective ratings using NASA Task Load Index (NASA-TLX), physiological measure using electroencephalogram, number of traffic violations (NTVs), speed variability, and reaction time of peripheral detection task were measured. The driving experiments reveal the following: (1) The subjective workload ratings on mean physical demand score were the highest. (2) The electroencephalogram results show that situation complexity had significant effects on theta relative power and alpha relative power of two channel locations (3) The highest mean NTVs was in VCS. (4) The mean speed variability in the MCS was significantly lower than that of in the SS and VCS. (5) The maximum reaction time was recorded in VCS while the minimum reaction time was recorded in the MCS. The overall driving performance score regression models were developed based on the strong correlation and linear relationship between mental workload and driving performance elements. The models may benefit as a reference for designers, manufacturers, developers, and policymakers in designing better driving environment for ageing drivers by integrating safety and transportation, thus, optimizing and sustaining the driving performance of ageing drivers.     

Drivers’ performances and their subjective feelings about their driving during a 40-min test on a circuit versus a dynamic simulator

Car manufacturers expect driving simulators to be reliable research and development tools. Questions arise, however, as to whether drivers’ behavior on simulators exactly matches that observed when they are driving real cars. Drivers’ performances and their subjective feelings about their driving were compared between two groups during a 40-min driving test on the same circuit in a real car (n = 20) and a high-fidelity dynamic simulator (n = 27). Their speed and its variability, the braking force and the engine revolutions per minute (rpm) were recorded five times on a straight line and three times on a curve. The differences observed in these measurements between circuit driving (CD) and simulator driving (SD) from the 6th to 40th minute showed no significant changes during the drive. The drivers also completed the NASA Raw Task Load Index (NASA RTLX) questionnaire and the Simulator Sickness Questionnaire (SSQ) and estimated the ease and standard of their own driving performances. These subjective feelings differed significantly between the two groups throughout the experiment. The SD group’s scores on the NASA RTLX and SSQ questionnaires increased with time and the CD group’s perceived driving quality and ease increased with time, reaching non-significantly different levels from their usual car driving standards by the end of the drive. These findings show the existence of a fairly good match between real-life and simulated driving, which stabilized six minutes after the start of the test, regardless of whether the road was straight or curved. These objective findings and subjective assessments suggest possible ways of improving the match between drivers’ performances on simulators and their real-life driving behavior.     

Fine and gross motor skills: The effects on skill-focused dual-tasks

Dual-task methodology often directs participants’ attention towards a gross motor skill involved in the execution of a skill, but researchers have not investigated the comparative effects of attention on fine motor skill tasks. Furthermore, there is limited information about participants’ subjective perception of workload with respect to task performance. To examine this, the current study administered the NASA-Task Load Index following a simulated shooting dual-task. The task required participants to stand 15 feet from a projector screen which depicted virtual targets and fire a modified Glock 17 handgun equipped with an infrared laser. Participants performed the primary shooting task alone (control), or were also instructed to focus their attention on a gross motor skill relevant to task execution (gross skill-focused) and a fine motor skill relevant to task execution (fine skill-focused). Results revealed that workload was significantly greater during the fine skill-focused task for both skill levels, but performance was only affected for the lesser-skilled participants. Shooting performance for the lesser-skilled participants was greater during the gross skill-focused condition compared to the fine skill-focused condition. Correlational analyses also demonstrated a significant negative relationship between shooting performance and workload during the gross skill-focused task for the higher-skilled participants. A discussion of the relationship between skill type, workload, skill level, and performance in dual-task paradigms is presented.     

认知负荷主观评价量表比较

运用双任务实验范式,比较了三种认知负荷主观评价量表的灵敏度与效度。结果发现：在本研究任务条件下,次任务反应时的稳定性、抗干扰性较好,可以作为认知负荷主观评价的标尺;WP量表与PAAS量表的敏感性均较好,其中WP量表的敏感性、诊断性高于后者,TLX量表的敏感性较弱;WP量表与PAAS量表的效度较好,且好于TLX量表。综合各项指标,在中低难度任务下,WP量表是目前认知负荷较为理想的测量工具。     

Individual differences in response to automation: the five factor model of personality

This study examined the relationship of operator personality (Five Factor Model) and characteristics of the task and of adaptive automation (reliability and adaptiveness-whether the automation was well-matched to changes in task demand) to operator performance, workload, stress, and coping. This represents the first investigation of how the Five Factors relate to human response to automation. One-hundred-sixty-one college students experienced either 75% or 95% reliable automation provided with task loads of either two or four displays to be monitored. The task required threat detection in a simulated uninhabited ground vehicle (UGV) task. Task demand exerted the strongest influence on outcome variables. Automation characteristics did not directly impact workload or stress, but effects did emerge in the context of trait-task interactions that varied as a function of the dimension of workload and stress. The pattern of relationships of traits to dependent variables was generally moderated by at least one task factor. Neuroticism was related to poorer performance in some conditions, and all five traits were associated with at least one measure of workload and stress. Neuroticism generally predicted increased workload and stress and the other traits predicted decreased levels of these states. However, in the case of the relation of Extraversion and Agreeableness to Worry, Frustration, and avoidant coping, the direction of effects varied across task conditions. The results support incorporation of individual differences into automation design by identifying the relevant person characteristics and using the information to determine what functions to automate and the form and level of automation.     

Mental workload and visual impairment: differences between pupil, blink, and subjective rating

This research has two aims: (a) To study the concurrent validity of three measures of mental workload, NASA TLX rating scale, pupil dilation and blink rate, testing the hypothesis that they will provide convergent results using a single-task, and dissociative results for dual-task; and (b) To analyse their capability to predict visual search impairment. These three measures were analyzed in the same cognitive tasks in single-task and dual-task (cognitive task and visual search) conditions in a within-subjects experiment with twenty-nine participants. Mental workload measures showed concurrent validity under single-task condition, but a complex pattern of results arose in the dual-task condition: it is suggested that NASA TLX would be a subjective addition of the rating of each task; pupil dilation would measure the average arousal underlying the cognitive tasks; and the blink rate would produce opposite effects: whereas mental workload of cognitive tasks would increase blink rate, visual demand would inhibit it. All three measures were good predictors of visual impairment. The soundness of these measures is discussed with regard to the applied field of driving and other activities.     

Effects of adaptive cruise control and highly automated driving on workload and situation awareness: A review of the empirical evidence

Adaptive cruise control (ACC), a driver assistance system that controls longitudinal motion, has been introduced in consumer cars in 1995. A next milestone is highly automated driving (HAD), a system that automates both longitudinal and lateral motion. We investigated the effects of ACC and HAD on drivers’ workload and situation awareness through a meta-analysis and narrative review of simulator and on-road studies. Based on a total of 32 studies, the unweighted mean self-reported workload was 43.5% for manual driving, 38.6% for ACC driving, and 22.7% for HAD (0% = minimum, 100 = maximum on the NASA Task Load Index or Rating Scale Mental Effort). Based on 12 studies, the number of tasks completed on an in-vehicle display relative to manual driving (100%) was 112% for ACC and 261% for HAD. Drivers of a highly automated car, and to a lesser extent ACC drivers, are likely to pick up tasks that are unrelated to driving. Both ACC and HAD can result in improved situation awareness compared to manual driving if drivers are motivated or instructed to detect objects in the environment. However, if drivers are engaged in non-driving tasks, situation awareness deteriorates for ACC and HAD compared to manual driving. The results of this review are consistent with the hypothesis that, from a Human Factors perspective, HAD is markedly different from ACC driving, because the driver of a highly automated car has the possibility, for better or worse, to divert attention to secondary tasks, whereas an ACC driver still has to attend to the roadway.     

The power and sensitivity of four core driver workload measures for benchmarking the distraction potential of new driver vehicle interfaces

This study evaluated the power and sensitivity of several core driver workload measures in order to better understand their use as a component of future driver distraction potential evaluation procedures of the in-vehicle human machine interface (HMI). Driving is a task that requires visual, manual and cognitive resources to perform. Secondary tasks, such as mobile phone use and interaction with in-built navigation, which load onto any of these three processing resources increase driver workload and can lead to impaired driving. Because workload and distraction potential are interrelated, a comprehensive method to assess driver workload that produces valid and predictive results is needed to advance the science of distraction potential evaluation. It is also needed to incorporate into New Car Assessment Program (NCAP) testing regimes. Workload measures of cognitive (DRT [Detection Response Task] Reaction Time), visual (DRT Miss Rate), subjective (NASA-TLX [driver workload questionnaire]), and temporal demand (Task Interaction Time) were collected as participants drove one of 40 vehicles while completing a variety of secondary tasks with varying interaction requirements. Of the evaluated measures, variance and power analyses demonstrated that Task Interaction Time is the most sensitive in detecting differences in driver workload between different in-vehicle HMIs, followed by DRT Miss Rate, NASA-TLX and finally DRT Reaction Time. There were relatively weak correlations between each of the four measures. These results suggest that Task Interaction Time, coupled with a reliable visual demand metric such as DRT Miss Rate, eye glance coding, or visual occlusion, more efficiently detect differences in driver workload between different HMIs compared to DRT Reaction Time and the NASA-TLX questionnaire. These results can be used to improve the understanding of the utility of each of these core driver workload measures in assessing driver distraction potential.     

Spatial and temporal task characteristics as stress: a test of the dynamic adaptability theory of stress, workload, and performance

The goal for this study was to test assertions of the dynamic adaptability theory of stress, which proposes two fundamental task dimensions, information rate (temporal properties of a task) and information structure (spatial properties of a task). The theory predicts adaptive stability across stress magnitudes, with progressive and precipitous changes in adaptive response manifesting first as increases in perceived workload and stress and then as performance failure. Information structure was manipulated by varying the number of displays to be monitored (1, 2, 4 or 8 displays). Information rate was manipulated by varying stimulus presentation rate (8, 12, 16, or 20 events/min). A signal detection task was used in which critical signals were pairs of digits that differed by 0 or 1. Performance accuracy declined and workload and stress increased as a function of increased task demand, with a precipitous decline in accuracy at the highest demand levels. However, the form of performance change as well as the pattern of relationships between speed and accuracy and between performance and workload/stress indicates that some aspects of the theory need revision. Implications of the results for the theory and for future research are discussed.