Weakening the robustness of perspective: Evidence for a modified theory of compensation in picture perception

Viewed from the center of projection, a perspective picture presents the pictorial depth information of a scene. Knowing the center of projection, one can reconstruct the depicted scene. Assuming another viewpoint is the center of projection will cause one to reconstruct a transformed scene. Despite these transformations, we appreciate pictures from other viewpoints. The compensation hypothesis states that the visible picture surface allows observers to compensate for transformations by locating the center of projection and experiencing pictorial space from there. We show that observers neither completely compensate for nor experience transformations of space as geometry would predict. We propose a modified compensation hypothesis according to which different degrees of visibility of the picture surface invoke different degrees of compensation.     

Is pictorial perception robust? The effect of the observer vantage point on the perceived depth structure of linear-perspective images

Every image of a scene produced in accord with the rules of linear perspective has an associated projection centre. Only if observed from that position does the image provide the stimulus which is equivalent to the one provided by the original scene. According to the perspective-transformation hypothesis, observing the image from other vantage points should result in specific transformations of the structure of the conveyed scene, whereas according to the vantage-point compensation hypothesis it should have little effect. Geometrical analyses illustrating the transformation theory are presented. An experiment is reported to confront the two theories. The results provide little support for the compensation theory and are generally in accord with the transformation theory, but also show systematic deviations from it, possibly due to cue conflict and asymmetry of visual angles.     

Ambiguity in pictorial depth

Pictorial space is the 3-D impression that one obtains when looking 'into' a 2-D picture. One is aware of 3-D 'opaque' objects. 'Pictorial reliefs' are the surfaces of such pictorial objects in 'pictorial space'. Photographs (or any pictures) do in no way fully specify physical scenes. Rather, any photograph is compatible with an infinite number of possible scenes that may be called 'metameric scenes'. If pictorial relief is one of these metameric scenes, the response may be considered 'veridical'. The conventional usage is more restrictive and is indeed inconsistent. Thus the observer has much freedom in arriving at such a 'veridical' response. To address this ambiguity, we determined the pictorial reliefs for eight observers, six pictures, and two psychophysical methods. We used 'methods of cross-sections' to operationalise pictorial reliefs. We find that linear regression of the depths of relief at corresponding locations in the picture for different observers often lead to very low (even insignificant) R2s. Thus the responses are idiosyncratic to a large degree. Perhaps surprisingly, we also observed that multiple regression of depth and picture coordinates at corresponding locations often lead to very high R2s. Often R2s increased from insignificant up to almost 1. Apparently, to a large extent 'depth' is irrelevant as a psychophysical variable, in the sense that it does not uniquely account for the relation of the response to the pictorial structure. This clearly runs counter to the bulk of the literature on pictorial 'depth perception'. The invariant core of interindividual perception proves to be of an 'affine' rather than a Euclidean nature; that is to say, 'pictorial space' is not simply the picture plane augmented with a depth dimension.     

Rotation of objects in pictures viewed at an angle: evidence for different properties of two types of pictorial space

As an observer views a picture from different viewing angles, objects in the picture appear to change orientation relative to the observer, but some objects change orientation more than others. This difference in rotation for different objects is called the differential rotation effect. The differential rotation is not, however, accompanied by corresponding changes in the perception of the spatial layout of objects in the picture. This lack of correspondence between the perception of rotation and the perception of spatial layout is a result of the fact that the information on a picture's surface defines two kinds of pictorial space with different properties. Rotation is perceived in terms of the pictorial space outside the picture, and spatial layout is perceived in terms of the pictorial space inside the picture.     

Foreshortened tiles in paths converging on an observer viewing a picture: Elevation and visual angle ratio determine perceived size

Observers were shown wide-angle pictures of tiles on a ground plane and were asked about the aspect ratios of the tiles. The observers viewed the pictures from a fixed center of projection. Some of the tiles were in a path coming straight toward the observer. In one picture, the path came from the center of the picture, and in two others the path came from the left side of the picture (one from 30° and one from 45° to the left of the center, from the observer’s point of view). The apparent aspect ratios were a function of the elevations of the tiles and the ratios of visual angles of the sides of the tiles. Judgments were identical for all three paths. The local slant of the picture surface was not a significant factor.     

Visual orientation in a mirror world tilted 90 degrees

Previously, we showed that many supine observers in a furnished room tilted 90 degrees perceive themselves and the room as upright. We called this the "levitation illusion" because the arms feel weightless when held out from the body. We now report that a familiar scene viewed by supine observers through a mirror at 45 degrees appears vertical when, optically, it is horizontal and above the head. However, the body feels pitched upright only partially. This visual-righting effect, like the levitation illusion, is due to the polarity axis of the scene being accepted as vertical even in the presence of conflicting information from the gravity sense organs. In experiment 1 we tested the potency of objects containing either intrinsic polarity (due to familiar tops and bottoms) or extrinsic polarity (due to support relationships) to generate a visual-righting illusion. To almost all observers, a blank surface seen in the mirror appeared like a ceiling. A scene containing an object with intrinsic polarity, such as a chair or person, seen in the mirror appeared vertical to almost all observers. A scene containing a pair of objects with only extrinsic polarity, such as a ball on a shelf (but not a ball under a shelf), also appeared vertical to most observers. In experiment 2 we found that a polarised scene was more likely to produce a visual-righting illusion when it was arranged as a view through a window rather than as a picture inside a room.     

Cast shadows in wide perspective

We investigated the apparent spatial layout of cast shadows up to very wide fields of view. We presented up to 130 degrees wide images in which two 'flat poles' were standing on a green lawn under a cloudless blue sky on a sunny day. The poles threw sharp cast shadows on the green, of which one was fixed. The observer's task was to adjust the azimuth of the shadow of the other pole such that it fitted the scene. The source elevation was kept constant. The two cast shadows are, of course, parallel in physical space, but generically not in the picture plane because of the wide perspective. We found that observers made huge systematic errors, indicating that, generically, they fail to account for these perspective effects. The systematic deviations could be well described by a weighted linear combination of the directions in the picture plane and in the physical space, with weights that depended on the positions of, and distance between, the poles.     

Mirror-reflecting a picture of an object: what happens to the shape percept?

Although a two-dimensional picture never fully specifies the actual layout of the depicted three-dimensional scene, one is still able to make a three-dimensional interpretation. When a picture is mirror-reflected, the range of plausible scenes possibly corresponding with the depicted scene has not changed with respect to the original depiction. We were curious to find out whether the inherent picture ambiguities would be solved the same way or differently. Participants performed local attitude settings on three sets of pictures: (1) original pictures, (2) left-right mirrored pictures, and (3) up-down mirrored pictures. Pairwise comparison of the pictorial reliefs of the depicted object, reconstructed from the raw settings, revealed dissimilarities. The differences, however, could be drastically diminished by conducting an affine transformation correction taking into account not only the depths, but also the picture plane coordinates. The inherent ambiguities seemed thus to be solved differently between conditions. By factoring out different solutions to the ambiguities, the pictorial reliefs were found to be equivalent.     

Perceived depth of 3-D objects in 3-D scenes

Effects of information specifying the position of an object in a 3-D scene were investigated in two experiments with twelve observers. To separate the effects of the change in scene position from the changes in the projection that occur with increased distance from the observer, the same projections were produced by simulating (a) a constant object at different scene positions and (b) different objects at the same scene position. The simulated scene consisted of a ground plane, a ceiling plane, and a cylinder on a pole attached to both planes. Motion-parallax scenes were studied in one experiment; texture-gradient scenes were studied in the other. Observers adjusted a line to match the perceived internal depth of the cylinder. Judged depth for objects matched in simulated size decreased as simulated distance from the observer increased. Judged depth decreased at a faster rate for the same projections shown at a constant scene position. Adding object-centered depth information (object rotation) increased judged depth for the motion-parallax displays. These results demonstrate that the judged internal depth of an object is reduced by the change in projection that occurs with increased distance, but this effect is diminished if information for change in scene position accompanies the change in projection.     

Ambiguity and the 'mental eye' in pictorial relief

Photographs of scenes do not determine scenes in the sense that infinitely many different scenes could have given rise to any given photograph. In psychophysical experiments, observers have (at least partially) to resolve these ambiguities. The ambiguities also allow them to vary their response within the space of 'veridical' responses. Such variations may well be called 'the beholder's share' since they do not depend causally on the available depth cues. We determined the pictorial relief for four observers, four stimuli, and four different tasks. In all cases we addressed issues of reliability (scatter on repeated trials) and consistency (how well the data can be explained via a smooth surface, any surface). All data were converted to depth maps which allows us to compare the relief from the different operationalisations. As expected, pictorial relief can differ greatly either between observers (same stimulus, same task) or between operationalisations (same observer, same stimulus). However, when we factor out the essential ambiguity, these differences almost completely vanish and excellent agreement over tasks and observers pertains. Thus, observers often resolve the ambiguity in idiosyncratic ways, but mutually agree--even over tasks--in so far as their responses are causally dependent on the depth cues. A change of task often induces a change in 'mental perspective'. In such cases, the observers switch the 'beholder's share', which resolves the essential ambiguity through a change in viewpoint of their 'mental eye'.     

The horizon-ratio relation as information for relative size in pictures

The horizon-ratio relation was found to be an effective source of information for relative size in pictures under some conditions: when the difference in image size of depicted “same real size” objects was not too great (Experiment 1), and when the horizon line was not too high or too low in the picture (Experiment 2). The latter finding seemed to be linked to the observers’ identification of the horizontal line as the horizon (and not as the edge of a finite surface). In addition, individual patterns of response were remarkably systematic even in the absence of a horizon, or any other pictorial information, (Experiment 3). It is suggested that in this case observers imposed a horizon on the picture on which to base their relative size judgments, possibly based on the observer’s own eye level or on the content of the picture. It is concluded that although the horizon-ratio relation provides the same kind of information as that available in the optic arrays from real scenes, pictorial information requires the satisfaction of additional constraints in order to be fully effective.     

Scene articulation: dependence of illuminant estimates on number of surfaces

The ability of observers to detect changes in illuminant over two scenes containing different random samples of reflecting surfaces was determined in an experiment with Mondrian-like patterns containing different numbers of coloured patches. Performance was found to improve as the number of patches increased from 9 to 49. In principle, observers could have used space-average scene colour as the cue ('grey-world' hypothesis) or the colour of the brightest surface in the scene ('bright-is-white' hypothesis), as the two cues generally covary. In a second experiment, observers matched illuminants across different patterns in which the space-average cue and the brightest-patch cue were independently manipulated. The articulation of the patterns was varied: the number of patches increased from 49 (patch width 1 deg visual angle) to over 50000 (patch width 0.03 deg), while the gamut of colours was held constant. Space-average colour was found to be the dominant cue with all patterns except for those with the largest patches.     

Scene recognition following locomotion around a scene

Effects of locomotion on scene-recognition reaction time (RT) and accuracy were studied. In experiment 1, observers memorized an 11-object scene and made scene-recognition judgments on subsequently presented scenes from the encoded view or different views (ie scenes were rotated or observers moved around the scene, both from 40 degrees to 360 degrees). In experiment 2, observers viewed different 5-object scenes on each trial and made scene-recognition judgments from the encoded view or after moving around the scene, from 36 degrees to 180 degrees. Across experiments, scene-recognition RT increased (in experiment 2 accuracy decreased) with angular distance between encoded and judged views, regardless of how the viewpoint changes occurred. The findings raise questions about conditions in which locomotion produces spatially updated representations of scenes.     

Face recognition in pictures is affected by perspective transformation but not by the centre of projection

Recognition of unfamiliar faces is susceptible to image differences caused by angular sizes subtended from the face to the camera. Research on perception of cubes suggests that apparent distortions of a shape due to large camera angle are correctable by placing the observer at the centre of projection, especially when visibility of the picture surface is low (Yang and Kubovy, 1999 Perception & Psychophysics 61 456-467). To explore the implication of this finding for face perception, observers performed recognition and matching tasks where face images with reduced visibility of picture surface were shown with observers either at the centre of projection or at other viewpoints. The results show that, unlike perception of cubes, the effect of perspective transformation on face recognition is largely unaffected by the centre of projection. Furthermore, the use of perspective cues is not affected by textured surfaces. The limitation of perspective in restoring 3-D information of faces suggests a stronger role for image-based, rather than model-based, processes in recognition of unfamiliar faces.     

Revisiting scene primes for object locations

Sanocki and Epstein (1997) provided evidence that an immediate prior experience of a scene, as a prime, can induce representations of its spatial layout, facilitating the subsequent spatial processing of objects in the target scene. In their experiments, observers responded to target scenes by indicating which of two critical objects was closer in the pictorial space. Reaction times to target scenes that were preceded by same-scene primes without the critical objects were faster than reaction times to target scenes that were preceded by different scene or control primes (geometrical figures). By manipulating the nature of the prime and the interval between prime and target, and by cueing the position of the critical objects, we obtain evidence that the facilitating effect of the same-scene primes can also be explained by the sudden appearance of the critical objects in the target scene. In same-scene conditions, the critical objects cause a local onset, whereas in different-scene and control conditions the entire target scene causes a global onset. As a result, the local onset in the same-scene condition produces a shift of attention towards the critical objects, resulting in faster processing of the critical objects.     

Picture perception: An analysis of visual compensation

Although geometric information is altered when a picture’s viewing point is changed, such changes often do not affect perception. Two experiments assessed pictorial perception relative to the distortions introduced by viewing point dislocation. Results provide a psychophysical demonstration of pictorial compensation and suggest that it is based on the discrepancy between the actual and an assumed-correct viewing position. An explanation of pictorial compensation is offered that could be applied to direct picture perception and to picture-in-a-picture perception.     

Visual distance estimation in static compared to moving virtual scenes

Visual motion is used to control direction and speed of self-motion and time-to-contact with an obstacle. In earlier work, we found that human subjects can discriminate between the distances of different visually simulated self-motions in a virtual scene. Distance indication in terms of an exocentric interval adjustment task, however, revealed linear correlation between perceived and indicated distances but with a profound distance underestimation. One possible explanation for this underestimation is the perception of visual space in virtual environments. Humans perceive visual space in natural scenes as curved, and distances are increasingly underestimated with increasing distance from the observer. Such spatial compression may also exist in our virtual environment. We therefore surveyed perceived visual space in a static virtual scene. We asked observers to compare two horizontal depth intervals, similar to experiments performed in natural space. Subjects had to indicate the size of one depth interval relative to a second interval. Our observers perceived visual space in the virtual environment as compressed, similar to the perception found in natural scenes. However, the nonlinear depth function we found can not explain the observed distance underestimation of visual simulated self-motions in the same environment.     

Theoretical analysis of the cognitive processing of lexical and pictorial stimuli: reading, naming, and visual and conceptual comparisons

This article reviews the research literature on the differences between word reading and picture naming. A theory for the visual and cognitive processing of pictures and words is then introduced. The theory accounts for slower naming of pictures than reading of words. Reading aloud involves a fast, grapheme-to-phoneme transformation process, whereas picture naming involves two additional processes: (a) determining the meaning of the pictorial stimulus and (b) finding a name for the pictorial stimulus. We conducted a reading-naming experiment, and the time to achieve (a) and (b) was determined to be approximately 160 ms. On the basis of data from a second experiment, we demonstrated that there is no significant difference in time to visually compare two pictures or two words when size of the stimuli is equated. There is no difference in time to make the two types of cross-modality conceptual comparisons (picture first, then word, or word first, then picture). The symmetry of the visual and conceptual comparison results supports the hypothesis that the coding of the mind is neither intrinsically linguistic nor imagistic, but rather it is abstract. There is a potent stimulus size effect, equal for both pictorial and lexical stimuli. Small stimuli take longer to be visually processed than do larger stimuli. For optimal processing, stimuli should not only be equated for size, but should subtend a visual angle of at least 3 degrees. The article ends with the presentation of a mathematical theory that jointly accounts for the data from word-reading, picture-naming visual comparison, and conceptual-comparison experiments.     

Looking at scenes while searching for numbers: dividing attention multiplies space

Observers tend to remember seeing a greater expanse of a scene than was shown (boundary extension [BE]). Is undivided visual attention necessary for BE? In Experiment 1, 108 observers viewed photographs with superimposed numerals (2s and 5s). Each appeared for 750 msec, followed by a masked interval and a test picture (same, closer up, or wider angled). Test pictures were rated as the same, closer, or wider angled on a 5-point scale. Visual attention was manipulated with a search task: The observers reported the number of 5s (zero, one, or two). The observers performed search only, picture rating only, or both (giving search priority). Search accuracy was unaffected by condition. BE occurred in both conditions but was greater with divided attention. The results were replicated using incidental BE tests (Experiments 2 and 3). We propose that anticipatory representation of layout occurs automatically during scene perception, with focal attention serving to constrain the boundary error.     

Pictorial surface attitude and local depth comparisons

We measured local surface attitude for monocular pictorial relief and performed pairwise depthcomparison judgments on the same picture. Both measurements were subject to internal consistency checks. We found that both measurements were consistent with a relief (continuous pictorial surface) interpretation within the session-to-session scatter. We reconstructed the pictorial relief from both measurements separately, and found results that differed in detail but were quite similar in their basic structures. Formally, one expects certain geometrical identities that relate range and attitude data. Because we have independent measurements of both, we can attempt an empirical verification of such geometrical identities. Moreover, we can check whether the statistical scatter in the data indicates that, for example, the surface attitudes are derivable from a depth map or vice versa. We estimate that pairwise depth comparisons are an order of magnitude less precise than might be expected from the attitude data. Thus, the surface attitudes cannot be derived from a depth map as operationally defined by our methods, although the reverse is a possibility.