Confirming statements about pictures of natural scenes: evidence of the processing of gist from eye movements

Combined displays of graphics and text, such as figure captions in newspapers and books, lead to distinctive inspection patterns, or scanpaths. Readers characteristically look very briefly at the picture, and then read the caption, and then look again at the picture. The initial inspection of the picture is the focus of interest in the present experiment, in which we attempted to modify the inspection by giving participants advance knowledge of the subject of a sentence (the cued object) that was to be verified or denied on the basis of whether it correctly described some aspect of the scene shown in the picture. Eye fixations were recorded while the viewers looked at the picture and the sentence in whatever sequence they chose. By allowing viewers to know the subject of the sentence in advance, we asked whether patterns of fixations on the sentence and on the second inspection of the picture would reflect prior knowledge of the focus of the sentence. Providing advance information did not influence eye movements while reading the sentence. It did, however, increase the number of fixations in the initial inspection of the picture, and it also reduced the number and duration of the fixations on the pictures overall. The results suggest that cueing participants to the object allowed increased coding in the initial inspection of the picture, though the benefit of such coding only becomes apparent when the picture is inspected for the second time.