A novel blink detection method based on pupillometry noise

Pupillometry (or the measurement of pupil size) is commonly used as an index of cognitive load and arousal. Pupil size data are recorded using eyetracking devices that provide an output containing pupil size at various points in time. During blinks the eyetracking device loses track of the pupil, resulting in missing values in the output file. The missing-sample time window is preceded and followed by a sharp change in the recorded pupil size, due to the opening and closing of the eyelids. This eyelid signal can create artificial effects if it is not removed from the data. Thus, accurate detection of the onset and the offset of blinks is necessary for pupil size analysis. Although there are several approaches to detecting and removing blinks from the data, most of these approaches do not remove the eyelid signal or can result in a relatively large amount of data loss. The present work suggests a novel blink detection algorithm based on the fluctuations that characterize pupil data. These fluctuations (“noise”) result from measurement error produced by the eyetracker device. Our algorithm finds the onset and offset of the blinks on the basis of this fluctuation pattern and its distinctiveness from the eyelid signal. By comparing our algorithm to three other common blink detection methods and to results from two independent human raters, we demonstrate the effectiveness of our algorithm in detecting blink onset and offset. The algorithm’s code and example files for processing multiple eye blinks are freely available for download (https://osf.io/jyz43).     

Spontaneous eye blinks during creative task correlate with divergent processing

Creativity consists of divergent and convergent thinking, with both related to individual eye blinks at rest. To assess underlying mechanisms between eye blinks and traditional creativity tasks, we investigated the relationship between creativity performance and eye blinks at rest and during tasks. Participants performed an alternative uses and remote association task while eye blinks were recorded. Results showed that the relationship between eye blinks at rest and creativity performance was compatible with those of previous research. Interestingly, we found that the generation of ideas increased as a function of eye blink number during the alternative uses task. On the other hand, during the remote association task, accuracy was independent of eye blink number during the task, but response time increased with it. Moreover, eye blink changes in participants who responded quickly during the remote association task were different depending on their resting state eye blinks; that is, participants with many eye blinks during rest showed little increasing eye blinks and achieved solutions quickly. Positive correlations between eye blinks during creative tasks and yielding ideas on the alternative uses task and response time on the remote association task suggest that eye blinks during creativity tasks relate to divergent thinking processes such as conceptual reorganization.     

Where does attention go when you blink?

Many studies have shown that covert visual attention precedes saccadic eye movements to locations in space. The present research investigated whether the allocation of attention is similarly affected by eye blinks. Subjects completed a partial-report task under blink and no-blink conditions. Experiment 1 showed that blinking facilitated report of the bottom row of the stimulus array: Accuracy for the bottom row increased and mislocation errors decreased under blink, as compared with no-blink, conditions, indicating that blinking influenced the allocation of visual attention. Experiment 2 showed that this was true even when subjects were biased to attend elsewhere. These results indicate that attention moves downward before a blink in an involuntary fashion. The eyes also move downward during blinks, so attention may precede blink-induced eye movements just as it precedes saccades and other types of eye movements.     

Blinking predicts enhanced cognitive control

Recent models have suggested an important role for neuromodulation in explaining trial-to-trial adaptations in cognitive control. The adaptation-by-binding model (Verguts & Notebaert, Psychological review, 115(2), 518–525, 2008), for instance, suggests that increased cognitive control in response to conflict (e.g., incongruent flanker stimulus) is the result of stronger binding of stimulus, action, and context representations, mediated by neuromodulators like dopamine (DA) and/or norepinephrine (NE). We presented a flanker task and used the Gratton effect (smaller congruency effect following incongruent trials) as an index of cognitive control. We investigated the Gratton effect in relation to eye blinks (DA related) and pupil dilation (NE related). The results for pupil dilation were not unequivocal, but eye blinks clearly modulated the Gratton effect: The Gratton effect was enhanced after a blink trial, relative to after a no-blink trial, even when controlling for correlated variables. The latter suggests an important role for DA in cognitive control on a trial-to-trial basis.     

A data-driven algorithm for offline pupil signal preprocessing and eyeblink detection in low-speed eye-tracking protocols

Event detection is the conversion of raw eye-tracking data into events—such as fixations, saccades, glissades, blinks, and so forth—that are relevant for researchers. In eye-tracking studies, event detection algorithms can have a serious impact on higher level analyses, although most studies do not accurately report their settings. We developed a data-driven eyeblink detection algorithm (Identification-Artifact Correction [I-AC]) for 50-Hz eye-tracking protocols. I-AC works by first correcting blink-related artifacts within pupil diameter values and then estimating blink onset and offset. Artifact correction is achieved with data-driven thresholds, and more reliable pupil data are output. Blink parameters are defined according to previous studies on blink-related visual suppression. Blink detection performance was tested with experimental data by visually checking the actual correspondence between I-AC output and participants’ eye images, recorded by the eyetracker simultaneously with gaze data. Results showed a 97% correct detection percentage.     

The effect of blinks and saccadic eye movements on visual reaction times

Vision is suppressed during blinks and saccadic eye movements. We hypothesized that visual reaction times (RTs) in a vigilance test would be significantly increased when a blink or a saccade happened to coincide with the stimulus onset. Thirty healthy volunteers each performed a visual RT test for 15 min while their eye and eyelid movements were monitored by a system of infrared reflectance oculography. RTs increased significantly, many by more than 200 msec, when a blink occurred between 75 msec before and up to 150 msec after the stimulus onset. A similar result was observed with saccades that started 75 to 150 msec after the stimulus. Vision or attention was evidently inhibited before each blink and for longer than the saccades lasted. We suggest that visual suppression is involved in this process, which could explain some of the normal variability in RTs over periods of seconds that has not been adequately explained before.     

The effects of driving environment complexity and dual tasking on drivers’ mental workload and eye blink behavior

The goal of the present study was to assess the effectiveness of eye blink behavior in measuring drivers’ mental workload. Previous research has shown that when mental workload increases with the primary task difficulty, blink frequency drops. On the opposite, the number of blinks increases when a cognitive secondary task has to be performed concurrently. However, the combined effects of the primary task difficulty and dual-tasking on blink rate have not been investigated. The present study was thus designed to vary systematically both the primary driving task and the cognitive secondary task demand to examine their combined effects on blink rate. The driving task was manipulated by varying the complexity of a simulated driving environment. The cognitive load was manipulated using a concurrent simple reaction time task or a complex calculation task. The results confirmed that eye blink frequency was a sensitive measure to elicit increased mental workload level coming from the driving environment. They also confirmed that blink rate increased with the introduction of a cognitive secondary task while blink duration was not affected. However, eye blink behavior did not provide a clear mental workload signature when driving task demands and dual-task conditions were varied simultaneously. The overall picture goes against the suitability of eye blink behavior to monitor drivers’ states at least when external and internal demands interact.     

Short-term memory across eye blinks

The effect of eye blinks on short-term memory was examined in two experiments. On each trial, participants viewed an initial display of coloured, oriented lines, then after a retention interval they viewed a test display that was either identical or different by one feature. Participants kept their eyes open throughout the retention interval on some blocks of trials, whereas on others they made a single eye blink. Accuracy was measured as a function of the number of items in the display to determine the capacity of short-term memory on blink and no-blink trials. In separate blocks of trials participants were instructed to remember colour only, orientation only, or both colour and orientation. Eye blinks reduced short-term memory capacity by approximately 0.6–0.8 items for both feature and conjunction stimuli. A third, control, experiment showed that a button press during the retention interval had no effect on short-term memory capacity, indicating that the effect of an eye blink was not due to general motoric dual-task interference. Eye blinks might instead reduce short-term memory capacity by interfering with attention-based rehearsal processes.     

Blinks as an index of cognitive activity during reading

Horizontal and vertical EOG recordings of eye movements were analyzed to determine the spatial and temporal distribution of blinks and the patterns of eye movements (saccades and fixation pauses) exhibited by six subjects during the reading of stories presented in two formats (on paper and on a VDT). The frequency and placement of blinks was not affected by the presentation condition. Blinks were determined to be non-randomly distributed during reading. Significantly more blinks (36%) occurred in conjunction with saccades than the proportion of time consumed by saccades (12%) would predict. Significantly more blinks (36%) occurred in the vicinity of line change saccades, which accounts for 15% of reading time, and with fixation pauses associated with regressions (42%), which accounts for 26% of reading time, than with fixation pauses during normal reading (22%), which accounts for 60% of reading time. The results of the study suggest that blink behavior during reading is under perceptual and cognitive control.     

Change detection and occlusion modes in road-traffic scenarios

Change blindness phenomena are widely known in cognitive science, but their relation to driving is not quite clear. We report a study where subjects viewed colour video stills of natural traffic while eye movements were recorded. A change could occur randomly in three different occlusion modes—blinks, blanks and saccades—or during a fixation (as control condition). These changes could be either relevant or irrelevant with respect to the traffic safety. We used deletions as well as insertions of objects. All occlusion modes were equivalent concerning detection rate and reaction time, deviating from the control condition only. The detection of relevant changes was both more likely and faster than that of irrelevant ones, particularly for relevant insertions, which approached the base line level. Even in this case, it took about 180 ms longer to react to changes when they occurred during a saccade, blink or blank. In a second study, relevant insertions and the blank occlusion were used in a driving simulator environment. We found a surprising effect in the dynamic setting: an advantage in change detection rate and time with blanks compared to the control condition. Change detection was also good during blinks, but not in saccades. Possible explanation of these effects and their practical implications are discussed.     

A signal-detection approach to subception: Concomitant verbal and finger-latency responses in metacontrast

Metacontrast conditions were created by the onset of flanking lights designed to mask the prior blink of an otherwise steady center light. For pseudometacontrast trials, the center light did not blink in advance of the flanking lights. Responses were an immediate finger-key release to the first detectable change in the visual display followed by a statement of whether the center light had been doused. A signal-detection analysis was used to avoid both threshold and response-criterion problems. Verbal report was, on the average, more sensitive than finger latency in detecting the masked blinks. However, there were large and consistent individual differences; reaction time was the more sensitive for a few subjects. Data analysis revealed that each response system showed detection with the other system “partialed out.” A model was offered in which verbal- and finger-response systems act in parallel, with uncorrelated variability between systems accounting for the subception effect.     

Reflex eyeblinks and visual suppression

Voluntary blinks are known to be accompanied by visual suppression that arises, at least in part, through nonretinal processes. We wanted to test the hypothesis that this suppression must arise as a corollary to the voluntary command to blink. We therefore examined visual sensitivity at the time of a blink that does not involve a voluntary command, namely the corneal reflex elicited by a puff of air. We found that sensitivity was reduced during these reflex blinks; the magnitude and time course of suppression were similar to those of voluntary blinks. We therefore adopt the view that visual suppression stems from signals from the common, efferent, blink pathway.     

Comparison of video- and EMG-based evaluations of the magnitude of children’s emotion-modulated startle response

We investigated the reliability and validity of a video-based method of measuring the magnitude of children’s emotion-modulated startle response when electromyographic (EMG) measurement is not feasible. Thirty-one children between the ages of 4 and 7 years were videotaped while watching short video clips designed to elicit happiness or fear. Embedded in the audio track of the video clips were acoustic startle probes. A coding system was developed to quantify from the video record the strength of the eye-blink startle response to the probes. EMG measurement of the eye blink was obtained simultaneously. Intercoder reliability for the video coding was high (Cohen’sκ = .90). The average within-subjects probe-by-probe correlation between the EMG- and video-based methods was .84. Group-level correlations between the methods were also strong, and there was some evidence of emotion modulation of the startle response with both the EMG- and the video-derived data. Although the video method cannot be used to assess the latency, probability, or duration of startle blinks, the findings indicate that it can serve as a valid proxy of EMG in the assessment of the magnitude of emotion-modulated startle in studies of children conducted outside of a laboratory setting, where traditional psychophysiological methods are not feasible.     

Cardiac modulation of startle: Effects on eye blink and higher cognitive processing

Cardiac cycle time has been shown to affect pre-attentive brainstem startle processes, such as the magnitude of acoustically evoked reflexive startle eye blinks. These effects were attributed to baro-afferent feedback mechanisms. However, it remains unclear whether cardiac cycle time plays a role in higher startle-related cognitive processes, as well. Twenty-five volunteers responded first by ’fast as possible’ button pushes (reaction time, RT), and second, rated perceived intensity of 60 acoustic startle stimuli (85, 95, or 105 dB; 50 ms duration; binaural; instantaneous rise time), which were presented either 230 or 530 ms after the R-wave, and eye blink responses were measured by EMG. RT was divided into evaluation and motor response time according to previous research. Increasing stimulus intensity enhanced startle eye blink, intensity ratings, and RT components. Eye blinks and intensity judgments were lower when startle was elicited at a latency of R + 230 ms, but RT components were differentially affected: the evaluative component was attenuated, and the motor component was accelerated when stimuli were presented 230 ms after the R-wave. We conclude that the cardiac cycle affects the attentive processing of acoustic startle stimuli.     

Effects of caffeine on the trigeminal blink reflex

The acoustic startle and trigeminal blink reflexes share the same motor output. Since caffeine has been shown to augment the startle reflex, it was proposed that caffeine would also increase the trigeminal blink reflex. In 6 humans, the effects of caffeine (100 mg) on the trigeminal blink reflex were investigated. Reflex blinks were elicited by stimulation of the supraorbital branch of the trigeminal nerve. Following ingestion of caffeinated coffee, reflex blinks increased in amplitude and duration and occurred at a shorter latency than reflex blinks following ingestion of decaffeinated coffee. Since the blink reflex is a brainstem reflex, these results suggest that the psychomotor effects of caffeine facilitate brainstem processing.     

Ocular signatures of proactive versus reactive cognitive control in young adults

During the execution of a cognitive task, the brain maintains contextual information to guide behavior and achieve desired goals. The AX-Continuous Performance Task is used to study proactive versus reactive cognitive control. Young adults tend to behave proactively in standard testing conditions. However, it remains unclear how interindividual variability (e.g., in cognitive and motivational factors) may drive people into more reactive or proactive control under the same task demands. We investigated the use of control strategies in a large population of healthy young adults. We computed the proactive behavioral index and consequently divided participants into proactive, reactive, and intermediate groups. We found that reactive participants were generally slower, presented lower context sensitivity, and larger response variability. Pupillary changes and blink rate index cognitive effort allocation. We measured, concomitantly to the task, the pupil size and frequency of blinks associated with the cue maintenance and response intervals. During the cue period, nonfrequent, nontarget cues led to increased pupil dilation and number of blinks in all participants. During the response interval, we found more errors and increased pupil dilation to the probe when all participants had to overcome a response bias generated by the frequent cue. Only reactive participants showed larger response-related pupil when they had to overcome a response bias related to the frequent probe. Contrary to expectations, groups did not differ in ocular measures in the cue period. In conclusion, interindividual differences in cognitive control between healthy adults can be mapped onto different patterns of effort allocation indexed by the pupil.     

Safe and sensible preprocessing and baseline correction of pupil-size data

Measurement of pupil size (pupillometry) has recently gained renewed interest from psychologists, but there is little agreement on how pupil-size data is best analyzed. Here we focus on one aspect of pupillometric analyses: baseline correction, i.e., analyzing changes in pupil size relative to a baseline period. Baseline correction is useful in experiments that investigate the effect of some experimental manipulation on pupil size. In such experiments, baseline correction improves statistical power by taking into account random fluctuations in pupil size over time. However, we show that baseline correction can also distort data if unrealistically small pupil sizes are recorded during the baseline period, which can easily occur due to eye blinks, data loss, or other distortions. Divisive baseline correction (corrected pupil size = pupil size/baseline) is affected more strongly by such distortions than subtractive baseline correction (corrected pupil size = pupil size ? baseline). We discuss the role of baseline correction as a part of preprocessing of pupillometric data, and make five recommendations: (1) before baseline correction, perform data preprocessing to mark missing and invalid data, but assume that some distortions will remain in the data; (2) use subtractive baseline correction; (3) visually compare your corrected and uncorrected data; (4) be wary of pupil-size effects that emerge faster than the latency of the pupillary response allows (within ±220 ms after the manipulation that induces the effect); and (5) remove trials on which baseline pupil size is unrealistically small (indicative of blinks and other distortions).     

Visual marking across eye blinks

Visual search for a conjunction target can be made efficient by presenting one initial set of distractors as a preview, prior to the onset of the other items in the search display Watson & Humphreys (Psychological Review 104:90–122, 1997). However, this “preview advantage” is lost if the initial items are offset for a brief period before onsetting again with the search display Kunar, Humphreys, & Smith (Psychological Science 14:181–185, 2003). Researchers have long disputed whether the preview advantage reflects a process of internally coding and suppressing the old items or of the onset of the new items capturing attention Donk & Theeuwes (Perception & Psychophysics 63:891–900, 2001). In this study, we assessed whether an internally driven blink (in which participants close their eyes) acts in the same manner as an external blink produced by offsetting and then onsetting the preview. In the novel blink conditions, participants searched feature, conjunction, and preview displays after being cued to blink their eyes. The search displays were presented during the eye blink, and so were immediately available once participants opened their eyes. Having participants make an eye blink generally slowed search but had no effect on the search slopes. In contrast, imposing an externally driven blink disrupted preview search. The data indicated that visual attention can compensate for internally driven blinks, and this does not lead to the loss of the representations of distractors across time. Moreover, efficient preview search occurred when the search items had no abrupt onsets, demonstrating that onsets of new search items are not critical for the preview benefit.     

An optoelectronic motion-sensing eye blink detector

A noninvasive eye blink detector can be built at low cost from a Sprague optoelectronic motion detection module, a voltage level detector, and a tone burst generator. The blink detector triggers reliably when used to detect motion appearing on the video monitor of a videotape recorder.