Depth from binocular half-occlusions in stereoscopic images of natural scenes

Over the past two decades psychophysical experiments have firmly established that binocular half-occlusions are useful sources of information for the human visual system. The existing literature has focused on simplified stimuli that have no additional cues to depth, apart from stereopsis. From this large body of work we can be confident that the visual system is able to exploit binocular half-occlusions to aid depth perception; however, we do not know if this signal has any influence on perception when observers view complex stereoscopic stimuli with multiple sources of depth information. This issue is addressed here with the use of stereoscopic images of natural scenes, some of which have been digitally altered to manipulate a major half-occlusion signal. Our results show that depth-ordering judgments for these relatively complex stimuli are significantly affected by the nature of the half-occlusion signal, but only when highly textured surfaces are viewed. Under such conditions, the replacement of a binocular half-occlusion with background texture slows reaction time relative to performance when the occluded region is consistent with the foreground object. This result is specific to conditions when the depth ordering is correct (ie not reversed) and depends upon the size of the half-occlusion. The influence of the half-occlusion information in the presence of potent depth cues such as perspective, texture gradient, shading, and interposition is convincing evidence that this information plays a significant role in human depth perception.     

The first pictures: perceptual foundations of Paleolithic art.

Paleolithic representational art has a number of consistent characteristics: the subjects are almost always animals, depicted without scenic background, usually in profile, and mostly in outline; the means of representation are extremely economical, often consisting of only a few strokes that indicate the salient features of the animal which are sufficient to suggest the whole form; and it is naturalistic to a degree, but lacks anything like photographic realism. Two elementary questions are raised in this essay: (i) why did the earliest known attempts at depiction have just these characteristics and not others? and (ii) how are objects so minimally represented recognizable? The answers seem to lie with certain fundamental features of visual perception, especially figure-ground distinction, Gestalt principles of closure and good continuation, line surrogacy, component feature analysis, and canonical imaging. In the earliest pictures the graphic means used are such that they evoke the same visual responses as those involved in the perception of real-world forms, but eschew redundancies of color, texture, linear perspective, and completeness of representation.     

Infants' distance perception from linear perspective and texture gradients

This study investigated 5- and 7-month-old infants' abilities to perceive objects' distances from pictorial depth cues, the depth cues available to a stationary, monocular viewer. Infants viewed a display in which texture gradients and linear perspective, two pictorial depth cues, created an illusion of two objects resting at different distances on a textured surface. Under monocular viewing conditions, 7-month-olds reached preferentially for the apparently nearer object, indicating that they perceived the objects' relative distances specified by pictorial depth cues. Under binocular viewing conditions, these infants showed no reaching preference. This finding rules out interpretations of the results not based on the objects' perceived distances. The 5-month-olds' reaching preferences were not significantly different in the experimental (monocular) and control (binocular) conditions. These infants, therefore, did not show clear evidence of distance perception from pictorial depth cues.     

The linear perspective information in ground surface representation and distance judgment

Most ground surfaces contain various types of texture gradient information that serve as depth cues for space perception. We investigated how linear perspective, a type of texture gradient information on the ground, affects judged absolute distance and eye level. Phosphorescent elements were used to display linear perspective information on the floor in an otherwise dark room. We found that observers were remarkably receptive to such information. Changing the configuration of the linear perspective information from parallel to converging resulted in relatively larger judged distances and lower judged eye levels. These findings support the proposals that (1) the visual system has a bias for representing an image of converging lines as one of parallel lines on a downward-slanting surface and (2) the convergence point of a converging-lines image represents the eye level. Finally, we found that the visual system may be less sensitive to the manipulation of compression gradient information than of linear perspective information.     

Cue gradient and cue density interact in the detection and recognition of objects defined by motion,contrast, or texture

The human visual system is able to extract an object from its surrounding using a number of cues. These include foreground/background gradients in disparity, motion, texture, colour, and luminance. We have investigated normal subjects' ability to detect objects defined by either motion, texture, or luminance gradients. The effects of manipulating cue density and cue foreground/background gradient on both detection and recognition accuracy were also investigated. The results demonstrate a simple additive relationship between cue density and cue gradient across forms defined by motion, luminance, and texture. The results are interpreted as evidence for the notion that form parsing is achieved via a similar algorithm across anatomically distinct processing streams.     

On the Role of Vertical Texture Cues in Height Perception

When people judge height from the top looking down they tend to overestimate vertical distance. Initial findings suggest that this perceptual bias may be in part due to the experienced fear of falling. However, previous studies did not control for potentially relevant optical invariants, especially vertical texture gradient cues, that may inform such perceptual judgments. In this study, we examined to what degree texture gradient cues may account for the perceptual bias in height perception. To this end, in 2 different conditions in which vertical texture gradient cues were either present or absent, participants provided distance and size judgments from an elevated position (5.68 m) and from the ground. Results revealed a systematic overestimation of vertical extent and object size when judged from a height. Yet, the absence or presence of texture gradient cues did not differentially influence these judgments. Finally, although fear levels were reported to be larger at the elevated position than on the ground, fear levels did not correlate with the perceptual bias, thereby demanding further research into the mechanisms underlying the perceptual bias in height perception.     

Extremal edges: a powerful cue to depth perception and figure-ground organization

Extremal edges (EEs) are projections of viewpoint-specific horizons of self-occlusion on smooth convex surfaces. An ecological analysis of viewpoint constraints suggests that an EE surface is likely to be closer to the observer than the non-EE surface on the other side of the edge. In two experiments, one using shading gradients and the other using texture gradients, we demonstrated that EEs operate as strong cues to relative depth perception and figure-ground organization. Image regions with an EE along the shared border were overwhelmingly perceived as closer than either flat or equally convex surfaces without an EE along that border. A further demonstration suggests that EEs are more powerful than classical figure-ground cues, including even the joint effects of small size, convexity, and surroundedness.     

Perceptual biases for multimodal cues in chimpanzee (<Emphasis Type="Italic">Pan troglodytes</Emphasis>) affect recognition

The ability of organisms to discriminate social signals, such as affective displays, using different sensory modalities is important for social communication. However, a major problem for understanding the evolution and integration of multimodal signals is determining how humans and animals attend to different sensory modalities, and these different modalities contribute to the perception and categorization of social signals. Using a matching-to-sample procedure, chimpanzees discriminated videos of conspecifics' facial expressions that contained only auditory or only visual cues by selecting one of two facial expression photographs that matched the expression category represented by the sample. Other videos were edited to contain incongruent sensory cues, i.e., visual features of one expression but auditory features of another. In these cases, subjects were free to select the expression that matched either the auditory or visual modality, whichever was more salient for that expression type. Results showed that chimpanzees were able to discriminate facial expressions using only auditory or visual cues, and when these modalities were mixed. However, in these latter trials, depending on the expression category, clear preferences for either the visual or auditory modality emerged. Pant-hoots and play faces were discriminated preferentially using the auditory modality, while screams were discriminated preferentially using the visual modality. Therefore, depending on the type of expressive display, the auditory and visual modalities were differentially salient in ways that appear consistent with the ethological importance of that display's social function.     

Rolling perception without rolling motion

The texture of a rolling circle depicts the translational and rotational components of its motion. In the case of a homogeneous circle, however, visual cues to the rotational component of motion are absent. To examine how the visual system resolves undetermined motion cues, optically neutral circles were displaced so that changes in their orientation were invisible. Contextual cues systematically triggered the perception of illusory rotation, suggesting that the visual system uses contextual cues along with intrinsic surface cues to compute percepts of rolling objects. This might also explain why people rarely experience the perception of ambiguous motion.     

Depth perception in Mongolian gerbils (Meriones unguiculatus) and spiny mice (Acomys russatus and A. cahirinus)

Depth perception in gerbils and spiny mice was studied with a modified visual cliff which varied the height of a platform from 5.08 cm to 25.4 cm and presented animals with an apparent drop-off to a patterned or a white field. Time to descend from the platform and orienting response frequency were recorded. For gerbils neither measure varied significantly between the platforms. Both Acomys species increased descent time and orienting response frequency as platform height increased. The results suggest that both gerbils and spiny mice can perceive depth, though there appears to be a difference between the species' use of sensory cues in descending from a visual cliff. The spiny mice appear to rely more on visual cues than do the gerbils. The data also reveal subtle behavioral differences between the Acomys species that may relate to their successful sympatry. The results are discussed by alluding to ecological differences between these species in their natural habitats.     

Multiple spatial learning strategies in goldfish (Carassius auratus)

There is a considerable amount of evidence that mammals and birds can use different spatial learning strategies based on multiple learning and memory systems. Unfortunately, only a few studies have investigated spatial learning and memory mechanisms in other vertebrates. This study aimed to identify the strategies used by goldfish to solve two different spatial tasks in a series of three experiments. In experiment 1, two groups of goldfish (Carassius auratus) were trained either in a spatial constancy task (SC), in which visual cues signalled the goal indirectly, or in a directly cued task (DC) in which similar cues signalled the goal directly. Transfer tests were conducted to study the effects of discrete cue deletion on the performance in both tasks. In these transfer tests the performance of the animals trained in the DC task dropped to chance level when the cue that signalled the goal directly was removed. In contrast, the removal of any single cue did not disrupt SC performance. In experiment 2, fish trained in the SC or the DC task were trained with the goal reversed. Goldfish in the SC group needed fewer sessions to master the reversal task than DC animals. Finally, experiment 3 investigated the effects of a substantial modification of the geometrical features of the apparatus on the performance of animals trained in the SC or in the DC condition. The performance of DC goldfish was not affected, whereas the same change disrupted performance in the SC animals despite the presence of the visual cues. These results suggest that there are separate spatial learning and memory systems in fish. Whereas the DC animals used a typical guidance strategy, relying only on the cue that signalled the goal directly, SC fish relied on a strategy with the properties of an actual spatial mapping system. Thus, the comparative approach points to the generality of these learning strategies among vertebrates. Received: 10 October 1998 / Accepted after revision: 16 April 1999     

The relative contribution of relative height, linear perspective, and texture gradients to pictorial depth perception in 7-month-old infants

In the current preferential-reaching experiments, 7-month-olds were tested for their ability to respond to a combination of relative height and texture gradients. The infants were presented with a display in which these pictorial depth cues specified that two toys were at different distances. The experimental displays differed from the textured surfaces employed in earlier studies in that linear perspective of the contours of the texture elements was omitted. Experiment A shows that the infants still preferred to reach for the apparently nearer toy under monocular, but not binocular, viewing conditions, indicating that they responded to the pictorial depth cues. In experiment B, relative height and texture provided the infants with conflicting information for depth. Here, relative height outperformed texture information. A statistical comparison between the experiments as well as systematic comparisons with experimental conditions from an earlier study (Hemker et al, 2010 Infancy 15 6-27) revealed that texture gradients, unlike linear perspective, neither enhanced nor weakened the effect exerted by relative height. In sum, 7-month-old infants are obviously more sensitive to relative height and to the linear perspective of the surface contours than to the texture gradients of compression, perspective, and density.     

Number, texture and crowding

A recent study shows that selectivity for numerosity emerges as a natural property in deep networks of hierarchical generative models of visual perception. These results, together with recent conceptualizations of crowding and texture, suggest that the 'sense of number' is a fundamental visual property, independent of texture and seemingly related attributes.     

The role of top-down knowledge about environmental context in egocentric distance judgments

Judgments of egocentric distances in well-lit natural environments can differ substantially in indoor versus outdoor contexts. Visual cues (e.g., linear perspective, texture gradients) no doubt play a strong role in context-dependent judgments when cues are abundant. Here we investigated a possible top-down influence on distance judgments that might play a unique role under conditions of perceptual uncertainty: assumptions or knowledge that one is indoors or outdoors. We presented targets in a large outdoor field and in an indoor classroom. To control visual distance and depth cues between the environments, we restricted the field of view by using a 14-deg aperture. Evidence of context effects depended on the response mode: Blindfolded-walking responses were systematically shorter indoors than outdoors, whereas verbal and size gesture judgments showed no context effects. These results suggest that top-down knowledge about the environmental context does not strongly influence visually perceived egocentric distance. However, this knowledge can operate as an output-level bias, such that blindfolded-walking responses are shorter when observers’ top-down knowledge indicates that they are indoors and when the size of the room is uncertain.

     

Monocular stereopsis with and without head movement

Random dots moving with various velocity gradients were presented to observers; the motion was yoked to head movement in one condition and to no head movement in another. In Experiment 1, 12 observers were shown motion gradients with sine, triangle, sawtooth, and square waveforms with amplitudes (equivalent disparities) of 12' and 1 degrees 53'. In Experiment 2, 48 observers were shown only the sinewave or square-wave gradient of 1 degrees 53' disparity either with or without head movement so that the observers' expectation to see depth in one condition did not transfer to another. The main findings were: (1) with 12' disparity, the head-movement condition produced perceived depth but almost no perceived motion, whereas the no-head-movement condition produced both perceived depth and perceived motion; (2) with 1 degrees 53' disparity, both conditions produced perceived depth and perceived motion; and (3) when the expectation to see depth was removed, the no-head-movement condition with the square-wave gradient produced no perceived depth, only motion. We suggest that monocular stereopsis with head movement can be achieved without perception of motion but monocular stereopsis without head movement requires perception of motion.     

Monocular without stereopsis with and head movement

Random dots moving with various velocity gradients were presented to observers; the motion was yoked to head movement in one condition and to no head movement in another. In Experiment 1, 12 observers were shown motion gradients with sine, triangle, sawtooth, and square waveforms with amplitudes (equivalent disparities) of 12′ and 1° 53′. In Experiment 2, 48 observers were shown only the sinewave or square-wave gradient of 1° 53′ disparity either with or without head movement so that the observers’ expectation to see depth in one condition did not transfer to another. The main findings were: (1) with 12′ disparity, the head-movement condition produced perceived depth but almost no perceived motion, whereas the no-head-movement condition produced both perceived depth and perceived motion; (2) with 1° 53′ disparity, both conditions produced perceived depth and perceived motion; and (3) when the expectation to see depth was removed, the no-head-movement condition with the square-wave gradient produced no perceived depth, only motion. We suggest that monocular stereopsis with head movement can be achieved without perception of motion but monocular stereopsis without head movement requires perception of motion.     

Texture gradient effectiveness in the perception of surface slant

To assess the development of monocular slant perception as well as the relative effectiveness of different sources of information, children in first, third, and fifth grades and college adults were asked to make judgments of surface slant on the basis of monocular texture gradient information. Accuracy of judgment increased with increasing age. In addition, differences in gradient effectiveness were found. Compression gradients were relatively ineffective sources of information, whereas perspective and multiple gradients resulted in greater accuracy. The results suggest limitations on the specificity of certain forms of gradients.     

Control of the corridor illusion in baboons (Papio papio) by gradient and linear-perspective depth cues

The corridor illusion was recently demonstrated in baboons with background pictures containing rich depth information (Barbet and Fagot 2002, Behavioural Brain Research 132 111-115). In the current research we determined the contribution of gradient texture and perspective lines to that illusion. In experiment 1, the corridor illusion was tested in two baboons, with pictures of a hallway as backgrounds, or the same image of the hallway represented by perspective lines. Findings confirmed that the baboons experience the corridor illusion with the picture of the hallway, and showed that this illusion remained with the perspective-line backgrounds. The same procedure was adopted in experiment 2, but with a hallway drawn from gradient textures. The two baboons experienced the illusion in this experiment too. Thus both gradient and perspective line cues convey sufficient information to control the corridor illusion in baboons. In baboons, the processing of these two kinds of depth cues interacts with the perception of object size, suggesting homologous processes of pictorial depth perception in humans and non-human primates.     

The contribution of monocular depth cues to scene perception by pigeons

The contributions of different monocular depth cues to performance of a scene perception task were investigated in 4 pigeons. They discriminated the sequential depth ordering of three geometric objects in computer-rendered scenes. The orderings of these objects were specified by the combined presence or absence of the pictorial cues of relative density, occlusion, and relative size. In Phase 1, the pigeons learned the task as a direct function of the number of cues present. The three monocular cues contributed equally to the discrimination. Phase 2 established that differential shading on the objects provided an additional discriminative cue. These results suggest that the pigeon visual system is sensitive to many of the same monocular depth cues that are known to be used by humans. The theoretical implications for a comparative psychology of picture processing are considered.     

INFANT MACAQUE MONKEYS RESPOND TO PICTORIAL DEPTH

Abstract— The studies described here are the first to demonstrate that a nonhuman primate species is capable of responding to pictorial depth information during infancy. In two experiments, pigtailed macaque (Macaca nemestrina) infants were tested for responsivity to the pictorial depth cites of texture gradient/linear perspective and relative size. The procedures were adapted from human studies and are based on the proclivity of infants to reach more frequently to closer objects than to objects that are farther away. The stimulus displays included two equidistant objects that, when viewed monocularly, appear separated in space because of an illusion created by pictorial depth cues. When presented with these displays, animals reached significantly more often to the apparently closer objects under monocular conditions than under binocular conditions. These findings suggest that infant macaques are sensitive to pictorial depth information, the implication being that this ability has ancient phytogenetic origins and ix not learned from exposure to the conventions of Western art.