ProtoMatch: a tool for analyzing high-density, sequential eye gaze and cursor protocols

ProtoMatch is a software tool for integrating and analyzing fixed-location and movement eye gaze and cursor data. It provides a comprehensive collection of protocol analysis tools that support sequential data analyses for eye fixations and scanpaths as well as for cursor “fixations” (dwells at one location) and “cursorpaths” (movements between locations). ProtoMatch is modularized software that integrates both eye gaze and cursor protocols into a unified stream of data and provides an assortment of filters and analyses. ProtoMatch subsumes basic analyses (i.e., fixation duration, number of fixations, etc.) and introduces a method of objectively computing the similarity between scanpaths or cursorpaths using sequence alignment. The combination of filters, basic analyses, and sequence alignment in ProtoMatch provides researchers with a versatile system for performing both confirmatory and exploratory sequential data analyses (Sanderson & Fisher, 1994).     

An eye movement analysis system (EMAS) for the identification of cognitive processes on figural tasks

EMAS is a software system (written in VAX-11 FORTRAN) for the analysis of eye movement data recorded during the performance of figural tasks. Its main functions are: (1) Calibration of raw coordinates of eye movements to determine their actual position on the stimulus display. Different kinds of measurement distortion may be corrected. (2) Identification of eye fixations and the determination of their locations and durations. (3) Analysis of fixation sequences. The frequency of transitions of fixations among specified sectors of the stimulus display is computed. A sequential list is made of the successive fixations in which the fixated sector and the fixation duration are graphically indicated. (4) Plotting of raw or calibrated eye movement data and fixation points. The sequence of fixations in specified display sectors can also be plotted in real time. Applications of the programs to the Embedded Figures Test and the Hidden Figures Test are illustrated.     

Identifying Representational Competence With Multi-Representational Displays

Increasingly, multi-representational educational technologies are being deployed in science classrooms to support science learning and the development of representational competence. Several studies have indicated that students experience significant challenges working with these multi-representational displays and prefer to use only one representation while problem solving. Here, we examine the use of one such display, a multi-representational molecular mechanics animation, by organic chemistry undergraduates in a problem-solving interview. Using both protocol analysis and eye fixation data, our analysis indicates that students rely mainly on two visual–spatial representations in the display and do not make use of two accompanying mathematical representations. Moreover, we explore how eye fixation data complement verbal protocols by providing information about how students allocate their attention to different locations of a multi-representational display with and without concurrent verbal utterances. Our analysis indicates that verbal protocols and eye movement data are highly correlated, suggesting that eye fixations and verbalizations reflect similar cognitive processes in such studies.     

Measuring the effect of multiple eye fixations on memory for visual attributes.

Because of limited peripheral vision, many visual tasks depend on multiple eye fixations. Good performance in such tasks demonstrates that some memory must survive from one fixation to the next. One factor that must influence performance is the degree to which multiple eye fixations interfere with the critical memories. In the present study, the amount of interference was measured by comparing visual discriminations based on multiple fixations to visual discriminations based on a single fixation. The procedure resembled partial report, but used a discrimination measure. In the prototype study, two lines were presented, followed by a single line and a cue. The cue pointed toward one of the positions of the first two lines. Observers were required to judge if the single line in the second display was longer or shorter than the cued line of the first display. These judgments were used to estimate a length threshold. The critical manipulation was to instruct observers either to maintain fixation between the lines of the first display or to fixate each line in sequence. The results showed an advantage for multiple fixations despite the intervening eye movements. In fact, thresholds for the multiple-fixation condition were nearly as good as those in a control condition where the lines were foveally viewed without eye movements. Thus, eye movements had little or no interfering effect in this task. Additional studies generalized the procedure and the stimuli. In conclusion, information about a variety of size and shape attributes was remembered with essentially no interference across eye fixations.     

Measuring the effect of multiple eye fixations on memory for visual attributes

Because of limited peripheral vision, many visual tasks depend on multiple eye fixations. Good performance in such tasks demonstrates that some memory must survive from one fixation to the next. One factor that must influence performance is the degree to which multiple eye fixations interfere with the critical memories. In the present study, the amount of interference was measured by comparing visual discriminations based on multiple fixations to visual discriminations based on a single fixation. The procedure resembled partial report, but used a discrimination measure. In the prototype study, two lines were presented, followed by a single line and a cue. The cue pointed toward one of the positions of the first two lines. Observers were required to judge if the single line in the second display was longer or shorter than the cued line of the first display. These judgments were used to estimate a length threshold. The critical manipulation was to instruct observers either to maintain fixation between the lines of the first display or to fixate each line in sequence. The results showed an advantage for multiple fixations despite the intervening eye movements. In fact, thresholds for the multiple-fixation condition were nearly as good as those in a control condition where the lines were foveally viewed without eye movements. Thus, eye movements had little or no interfering effect in this task. Additional studies generalized the procedure and the stimuli. In conclusion, information about a variety of size and shape attributes was remembered with essentially no interference across eye fixations.     

An eye fixation analysis of multialternative choice

Eye fixations were recorded while subjects chose their most preferred of six used cars. Fixation sequences of the form X-Y-X-... were used to identify pair comparisons. This assumption was validated by verbal protocols and by a comparison between X-Y-X and X-Y-X-Y-... patterns. The results of three experiments showed: (1) that the multialternative choice process was composed primarily of pair comparisons; (2) that evaluative processing took placeonly during these comparisons; (3) that strategies for selecting the pairs were based primarily on information processing convenience; (4) that the sequence of pair comparisons was not used by the subjects as a sequential elimination process; and (5)that subjects changed their strategies to adapt to different task environments. These results have implications for current choice models and for the use of eye fixations in other complex tasks.     

Interdependence of Saccadic and Fixational Fluctuations

Recent research suggests that the different components of eye movements (fixations, saccades) are not strictly separate but are interdependent processes. This argument rests on observations that gaze-step sizes yield unimodal distributions and exhibit power-law scaling, indicative of interdependent processes coordinated across timescales. The studies that produced these findings, however, employed complex tasks (visual search, scene perception). Thus, the question is whether the observed interdependence is a fundamental property of eye movements or emerges in the interplay between cognitive processes and complex visual stimuli. In this study, we used a simple eye movement task where participants moved their eyes in a prescribed sequence at several different paces. We outlined diverging predictions for this task for independence versus interdependence of fixational and saccadic fluctuations and tested these predictions by assessing the spectral properties of eye movements. We found no clear peak in the power spectrum attributable exclusively to saccadic fluctuations. Furthermore, changing the pace of the eye movement sequence yielded a global shift in scaling relations evident in the power spectrum, not just a localized shift for saccadic fluctuations. These results support the conclusion that fixations and saccades are interdependent processes.     

The perception of eye contact

Ss made judgments of whether or not they were being looked in the eye as an E fixated several points on and off their faces.Distance between sender and receiver, sex, and whether the sender made a sequence of fixation sprior to the terminal fixation were variables of interest. The sender-receiver distance produced less effect than predicted from the hypothesis that receiver judgments were determined solely by the discrepancy between the sender’s iris/sclera configuration (ISC) during true eye contact and the current fixation point. There was no stable differencein accuracy as a function of whether the sender’s terminal fixation was preceded by other fixations or not. These findings cast doubt on the belief expressed in recent studies that the ISC can be used to accurately discriminate eye gazes from other gazes. The results also suggest that fixations on some portions of the face yield a judgment of eye contact which might be predicted on the basis of social experience.     

Eye movement contingent display control in studying reading

A computer system has been developed that permits experimental control over a CRT display contingent upon characteristics of the viewer’s eye movements. The display can be changed during specific saccadic eye movements or fixations. Uses of such a system for studying reading are described. The paper reviews hardware and software considerations in developing such a system.     

A comparison of concurrent and retrospective verbal protocol analysis

In verbal protocol analysis, verbalization can occur either during decision making (concurrent data) or after (retrospective data). Although both methods have advantages and disadvantages, no empirical research has focused on a direct comparison. This study compared the effectiveness of concurrent and retrospective data for revealing the human decision making process. In general, the concurrent protocol analysis method outperformed the retrospective method. Not only was the number of concurrent protocol segments elicited higher than that of retrospective protocol segments, but concurrent data provided more insights into the decision-making steps occurring between stimulus introduction and the final choice outcome. However, retrospective protocols offer an interesting advantage: More statements about the final choice are provided in retrospective protocols than in concurrent protocols.     

How the Eyes Tell Lies: Social Gaze During a Preference Task

Social attention is thought to require detecting the eyes of others and following their gaze. To be effective, observers must also be able to infer the person's thoughts and feelings about what he or she is looking at, but this has only rarely been investigated in laboratory studies. In this study, participants' eye movements were recorded while they chose which of four patterns they preferred. New observers were subsequently able to reliably guess the preference response by watching a replay of the fixations. Moreover, when asked to mislead the person guessing, participants changed their looking behavior and guessing success was reduced. In a second experiment, naïve participants could also guess the preference of the original observers but were unable to identify trials which were lies. These results confirm that people can spontaneously use the gaze of others to infer their judgments, but also that these inferences are open to deception.     

Usability evaluation with screen reader users: a video presentation of the PCTA’s experimental setting and rules

In the study entitled “Web usability evaluation with screen reader users: Implementation of the Partial Concurrent Thinking Aloud technique” (Federici et al. 2010), we have proposed a modified protocol of usability evaluation technique for blind users, which integrates the features of the concurrent and the retrospective techniques. This new technique, called partial concurrent thinking aloud (PCTA), while respecting the properties of classic verbal protocols, overcomes the structural interference and the limits of concurrent and retrospective protocols when used with screen reader users. In order to facilitate understanding and acquisition of the PCTA’s technique for practitioners and researchers, we have video recorded three different verbal protocols by visualizing five experimental sections. In the first two videos, we have compared a concurrent with a retrospective’s verbal protocol of a sighted user, showing the difference of the verbalizations provided by the user in these two conditions. The third video shows the structural interference of the screen reader, during a blind user concurrent thinking aloud. In the last two videos, we show the difference of a blind user behaviour when PCTA or retrospective protocol is adopted. The videos clearly visualize the advantage of the PCTA use in respect of the two other protocols. In conclusion, the visualization of the PCTA technique confirms that this new verbal protocol promotes and guarantees a more user-driven usability assessment with disabled people, by better involving screen reader users, overcoming the structural interference and the limits of the concurrent and retrospective protocols.     

Evaluation algorithm for eye movement patterns during a problem solving task]

Cognitive processes are accompanied by eye movements, which are interrupted by fixations of an almost infinite number of points in the visual field. To investigate these processes, a systematic analysis of those movements is required to structure the long sequences of fixations. This task is possible if there are interrelationships between the ongoing thought processes and the fixations. Such interrelationships may manifest themselves through certain patterns of eye movements. It often happens that subsets of visual points are being looked at repeatedly. This is because a cognitive connection among the subset's elements is being discovered or created by such repetition. The method presented here attempts to achieve automatic recognition of fixation point patterns. Its usefulness is demonstrated using data from an experiment where eye movements were recorded while subjects were solving Raven Test items. The method may be characterized as a variant of time series analysis for nominal data. It is suitable for non-numerical data in general.     

Uncovering the problem-solving process: cued retrospective reporting versus concurrent and retrospective reporting

This study investigated the amounts of problem-solving process information ("action," "why," "how," and "metacognitive") elicited by means of concurrent, retrospective, and cued retrospective reporting. In a within-participants design, 26 participants completed electrical circuit troubleshooting tasks under different reporting conditions. The method of cued retrospective reporting used the original computer-based task and a superimposed record of the participant's eye fixations and mouse-keyboard operations as a cue for retrospection. Cued retrospective reporting (with the exception of why information) and concurrent reporting (with the exception of metacognitive information) resulted in a higher number of codes on the different types of information than did retrospective reporting.     

The perceptual aspect of skilled performance in chess: Evidence from eye movements

Expert and intermediate chess players attempted to choose the best move in five chess positions while their eye movements were monitored. Experts were faster and more accurate than intermediates in choosing the best move. Experts made fewer fixations per trial and greater amplitude saccades than did intermediates, but there was no difference in fixation duration across skill groups. Examining the spatial distribution of the first five fixations for each position by skill group revealed that experts produced more fixations on empty squares than did intermediates. When fixating pieces, experts produced a greater proportion of fixations on relevant pieces than did intermediates. It is argued that expert chess players perceptually encode chess configurations, rather than individual pieces, and, consequently, parafoveal or peripheral processing guides their eye movements, producing a pattern of saccadic selectivity by piece saliency.     

Eye movements in action disorganization syndrome: A single case analysis

This study examines eye movements made by a patient with action disorganization syndrome (ADS) as everyday tasks are performed. Relative to both normal participants and control patients, the ADS patient showed normal time-locking of eye movements to the subsequent use of objects. However, there were proportionately more unrelated fixations, and more fixations concerned with locating objects irrelevant to the immediate action, compared with control participants. The data suggest a dissociation between normal eye movement patterns for control of visually guided actions such as reaching and grasping, and abnormal eye movements between object-related fixations. The implications for understanding ADS are discussed.     

Eye fixations and memory for emotional events.

Subjects watched either an emotional, neutral, or unusual sequence of slides containing 1 critical slide in the middle. Experiments 1 and 2 allowed only a single eye fixation on the critical slide by presenting it for 180 ms (Experiment 1) or 150 ms (Experiment 2). Despite this constraint, memory for a central detail was better for the emotional condition. In Experiment 3, subjects were allowed 2.70 s to view the critical slide while their eye movements were monitored. When subjects who had devoted the same number of fixations were compared, memory for the central detail of the emotional slide was again better. The results suggest that enhanced memory for detail information of an emotional event does not occur solely because more attention is devoted to the emotional information.