Shape from shading from images rendered with various surface types and light fields

Shape constancy is referred to as the tendency for the perceived shape of an object to remain unchanged even under changed viewing and illumination conditions. We investigated, in two experiments, whether shape constancy would hold for images of 3-D solid objects defined by shading only, whose renderings differed in terms of surface material type (bi-directional reflectance distribution functions), light field, light direction, shape, and specularity. Observers were presented with the image of a sphere or an ellipsoid and required to set perceived orientation and cross-section profile on designated points of the image. Results showed that shape judgments varied with all the aforementioned variables except specularity. Shape estimates were more precise with specular than asperity scattering surfaces, collimated than hemispherical diffuse lighting conditions, lower than higher elevations, spherical than ellipsoidal shapes, but not different between surfaces having differing specularity. These results suggest that shape judgments are made largely on the basis of the overall intensity distribution of shading, and that the portions of intensity distribution that are due to nonstructural variables such as surface material type or light field are not excluded in the process of shape estimation, as if being due to structural components. It is concluded that little constancy is expected in the perception of shape from shading.     

Perception of illumination direction in images of 3-D convex objects: influence of surface materials and light fields

We investigated the perception of illumination direction in images of 3-D convex objects under variations of light field and surface material properties. In a first experiment, we used an illumination-matching procedure in order to measure observers' ability to estimate the direction of illumination in images of 3-D polyhedra rendered under different light fields and illumination directions. Match deviations were larger in frontal direction than in rear directions, mainly counterclockwise in azimuth component, and diverged, in elevation component, from the image plane. In a second experiment, we examined whether the direction estimate was affected by the surface material type (BRDF), the light field, and the illumination direction. Angular deviations varied with material surface type and were largest in the test elevation direction 0 degrees. Elevation component deviations also differed with surface type and were larger in hemispherical diffuse lighting than in collimated lighting. These results suggest that the direction estimation is better with images of evenly distributed intensity gradients than with those of drastically varying gradients, and that the visual system may not take intensity variations due to the surface material or the light field into account in estimating the direction of illumination.     

The perception of 3-D shape from shadows cast onto curved surfaces

In a natural environment, cast shadows abound. Objects cast shadows both upon themselves and upon background surfaces. Previous research on the perception of 3-D shape from cast shadows has only examined the informativeness of shadows cast upon flat background surfaces. In outdoor environments, however, background surfaces often possess significant curvature (large rocks, trees, hills, etc.), and this background curvature distorts the shape of cast shadows. The purpose of this study was to determine the extent to which observers can “discount” the distorting effects of curved background surfaces. In our experiments, observers viewed deforming or static shadows of naturally shaped objects, which were cast upon flat and curved background surfaces. The results showed that the discrimination of 3-D object shape from cast shadows was generally invariant over the distortions produced by hemispherical background surfaces. The observers often had difficulty, however, in identifying the shadows cast onto saddle-shaped background surfaces. The variations in curvature which occur in different directions on saddle-shaped background surfaces cause shadow distortions that lead to difficulties in object recognition and discrimination.     

Material-illumination ambiguities and the perception of solid objects

The appearance of objects depends on their material, shape, and on the illumination conditions. Conversely, object appearance provides us with cues about the illumination and the material. This so-called inverse problem is basically underdetermined and therefore we expect that material and illumination perception are confounded. To gain insight into the relevant mechanisms, we rendered a set of artificial spheres for vastly different canonical light fields and reflectance functions. We used four physics-based bidirectional reflectance distribution functions (BRDFs) representing glossy, pitted, velvety, and matte material. The six illumination conditions were collimated illumination from four directions, hemispherical diffuse illumination, and fully diffuse (Ganzfeld) illumination. In three sub-experiments we presented pairs of stimuli and asked human observers to judge whether the material was the same, whether the illumination was the same, and for a subset in which either the illumination or the material was the same to judge which of the two was constant. We found that observers made many errors in all sub-experiments. In experiment 2 the illumination direction was chosen at random. Using an interactive interface, we asked human observers to match the illumination direction of a sphere of one of the four materials with that of a Lambertian sphere. We found systematical material-dependent deviations from veridical performance. Theoretical analysis of the radiance patterns suggests that judgments were based mainly on the position of the shadow edge. In conclusion, we found no evidence for 'material constancy' for perception of smooth rendered spheres despite vast quantitative and qualitative differences in illumination and in BRDF between the stimuli. Although human observers demonstrated some 'illumination constancy', they made systematic errors depending on the material reflectance, suggesting that they used mainly the location of the shadow edge. Our results suggest that material perception and light-field perception are basically confounded.     

Lightness constancy in the presence of specular highlights

Abstract— Visible surfaces in a natural environment often have multiple components of reflectance, including a diffuse component, by which light is scattered isotropically in all possible directions, and a specular component, by which light is reflected anisotropically within a limited range of directions. The research described in the present article was designed to investigate how these different components of reflectance influence the perception of lightness. Human observers were presented with shaded images of smoothly curved surfaces and asked to compare the relative lightness of different surface regions whose diffuse and specular components of luminance were independently manipulated. The results revealed that observers are able to discount the presence of specular highlights so that the relative lightness among different regions is determined almost entirely by the diffuse component of surface reflectance.     

Perceptual limits on low-dimensional models of Munsell reflectance spectra

Some experimental and theoretical approaches to surface-colour perception depend on approximating surface reflectance spectra by low-dimensional models. In the psychophysical experiment reported here, observers had to discriminate between patterns of Munsell surfaces and their spectral approximations under either the same or different illuminants. The approximations were produced by principal component analysis, by independent component analysis, and by artificial neural networks trained with a supervised-learning rule. In all experimental conditions, observers required, on average, at least 5 basis functions for discrimination performance to be at chance, thus placing a lower limit on the dimensionality of models of Munsell reflectance spectra.     

Recognising faces: effects of lighting direction, inversion, and brightness reversal.

When information about three-dimensional shape obtained from shading and shadows is ambiguous, the visual system favours an interpretation of surface geometry which is consistent with illumination from above. If pictures of top-lit faces are rotated the resulting stimulus is both figurally inverted and illuminated from below. In this study the question of whether the effects of figural inversion and lighting orientation on face recognition are independent or interactive is addressed. Although there was a clear inversion effect for faces illuminated from the front and above, the inversion effect was found to be reduced or eliminated for faces illuminated from below. A strong inversion effect for photographic negatives was also found but in this case the effect was not dependent on the direction of illumination. These findings are interpreted as evidence to suggest that lighting faces from below disrupts the formation of surface-based representations of facial shape.     

The perception of lightness in 3-D curved objects

Lightness constancy in complex scenes requires that the visual system take account of information concerning variations of illumination falling on visible surfaces. Three experiments on the perception of lightness for three-dimensional (3-D) curved objects show that human observers are better able to perform this accounting for certain scenes than for others. The experiments investigate the effect of object curvature, illumination direction, and object shape on lightness perception. Lightness constancy was quite good when a rich local gray-level context was provided. Deviations occurred when both illumination and reflectance changed along the surface of the objects. Does the perception of a 3-D surface and illuminant layout help calibrate lightness judgments? Our results showed a small but consistent improvement between lightness matches on ellipsoid shapes, relative to flat rectangle shapes, under illumination conditions that produce similar image gradients. Illumination change over 3-D forms is therefore taken into account in lightness perception.     

Shape from shading in pigeons

Light is the origin of vision. The pattern of shading reflected from object surfaces is one of several optical features that provide fundamental information about shape and surface orientation. To understand how surface and object shading is processed by birds, six pigeons were tested with differentially illuminated convex and concave curved surfaces in five experiments using a go/no-go procedure. We found that pigeons rapidly learned this type of visual discrimination independent of lighting direction, surface coloration and camera perspective. Subsequent experiments varying the pattern of the lighting on these surfaces through changes in camera perspective, surface height, contrast, material specularity, surface shape, light motion, and perspective movement were consistent with the hypothesis that the pigeons were perceiving these illuminated surfaces as three-dimensional surfaces containing curved shapes. The results suggest that the use of relative shading for objects in a visual scene creates highly salient features for shape processing in birds.     

Discrimination of 3-D shape and 3-D curvature from motion in active vision

We examined the ability of human observers to discriminate between different 3-D quadratic surfaces defined by motion, and with head position fed back to the stimulus to provide an up-to-date dynamical perspective view. We tested whether 3-D shape or 3-D curvature would affect discrimination performance. It appeared that discrimination of 3-D quadratic shape clearly depended on shape but not on the amount of curvature. Even when the amount of curvature was randomized, subjects’ performance was not altered. On the other hand, the discrimination of 3-D curvature clearly depended linearly on curvature with Weber fractions of 20% on the average and, to a small degree, on 3-D shape. The experiment shows that observers can easily separate 3-D shape and 3-D curvature, and that Koenderink’s shape index and curvedness provide a convenient way to specify shape. These results warn us against using just any arbitrary 3-D shape in 3-D shape perception tasks and indicate, for example, that emphasizing 3-D shape in computer displays by exaggerating curvature does not have any effect.     

Aging and the discrimination of 3-D shape from motion and binocular disparity

Two experiments evaluated the ability of younger and older adults to visually discriminate 3-D shape as a function of surface coherence. The coherence was manipulated by embedding the 3-D surfaces in volumetric noise (e.g., for a 55?% coherent surface, 55?% of the stimulus points fell on a 3-D surface, while 45?% of the points occupied random locations within the same volume of space). The 3-D surfaces were defined by static binocular disparity, dynamic binocular disparity, and motion. The results of both experiments demonstrated significant effects of age: Older adults required more coherence (tolerated volumetric noise less) for reliable shape discrimination than did younger adults. Motion-defined and static-binocular-disparity-defined surfaces resulted in similar coherence thresholds. However, performance for dynamic-binocular-disparity-defined surfaces was superior (i.e., the observers?? surface coherence thresholds were lowest for these stimuli). The results of both experiments showed that younger and older adults possess considerable tolerance to the disrupting effects of volumetric noise; the observers could reliably discriminate 3-D surface shape even when 45?% of the stimulus points (or more) constituted noise.     

The stereoscopic anisotropy: individual differences and underlying mechanisms

Observers are more sensitive to variations in the depth of stereoscopic surfaces in a vertical than in a horizontal direction; however, there are large individual differences in this anisotropy. The authors measured discrimination thresholds for surfaces slanted about a vertical axis or inclined about a horizontal axis for 50 observers. Orientation and spatial frequency discrimination thresholds were also measured. For most observers, thresholds were lower for inclination than for slant and lower for orientation than for spatial frequency. There was a positive correlation between the 2 anisotropies, resulting from positive correlations between (a) orientation and inclination thresholds and (b) spatial frequency and slant thresholds. These results support the notion that surface inclination and slant perception is in part limited by the sensitivity of orientation and spatial frequency mechanisms.     

Matching illumination of solid objects

The appearance of objects is determined by their surface reflectance and roughness and by the light field. Conversely, human observers might derive properties of the light field from the appearance of objects. The inverse problem has no unique solution, so perceptual interactions between reflectance, roughness, and lightfield are to be expected. In two separate experiments, we tested whether observers are able to match the illumination of spheres under collimated illumination only (matching of illumination direction) and under more or less diffuse illumination (matching of illumination direction and directedness of the beam). We found that observers are quite able to match collimated illumination directions of two rendered Lambertian spheres. Matching of the collimated beam directions of a Lambertian sphere and that of a real object with arbitrary reflectance and roughness properties resulted in similar results for the azimuthal angle, but in higher variance for the polar angle. Translucent objects and a tennis ball were found to be systematic outliers. If the directedness of the beam was also varied, the direction settings showed larger variance for more diffuse illumination. The directedness settings showed an overall quite large variance and, interestingly, interacted with the polar angle settings. We discuss possible photometrical mechanisms behind these effects.     

Is pigmentation important for face recognition? Evidence from contrast negation

It is extraordinarily difficult to recognize a face in an image with negated contrast, as in a photographic negative. The variation among faces can be partitioned into two general sources: (a) shape and (b) surface reflectance, here termed 'pigmentation'. To determine whether negation differentially affects the processing of shape or pigmentation, we made two sets of faces where the individual faces differed only in shape in one set and only in pigmentation in the other. Surprisingly, matching performance was significantly impaired by contrast negation only when the faces varied in pigmentation. This provides evidence that the perception of pigmentation, not shape, is selectively disrupted by negation and, by extension, that pigmentation contributes to the neural representation of face identity.     

Face recognition in pictures is affected by perspective transformation but not by the centre of projection

Recognition of unfamiliar faces is susceptible to image differences caused by angular sizes subtended from the face to the camera. Research on perception of cubes suggests that apparent distortions of a shape due to large camera angle are correctable by placing the observer at the centre of projection, especially when visibility of the picture surface is low (Yang and Kubovy, 1999 Perception & Psychophysics 61 456-467). To explore the implication of this finding for face perception, observers performed recognition and matching tasks where face images with reduced visibility of picture surface were shown with observers either at the centre of projection or at other viewpoints. The results show that, unlike perception of cubes, the effect of perspective transformation on face recognition is largely unaffected by the centre of projection. Furthermore, the use of perspective cues is not affected by textured surfaces. The limitation of perspective in restoring 3-D information of faces suggests a stronger role for image-based, rather than model-based, processes in recognition of unfamiliar faces.     

Minimalist surface-colour matching

Some theories of surface-colour perception assume that observers estimate the illuminant on a scene so that its effects can be discounted. A critical test of this interpretation of colour constancy is whether surface-colour matching is worse when the number of surfaces in a scene is so small that any illuminant estimate is unreliable. In the experiment reported here, observers made asymmetric colour matches between pairs of simultaneously presented Mondrian-like patterns under different daylights. The patterns had either 49 surfaces or a minimal 2 surfaces. No significant effect of number was found, suggesting that illuminant estimates are unnecessary for surface-colour matching.     

Aging and the visual, haptic, and cross-modal perception of natural object shape

One hundred observers participated in two experiments designed to investigate aging and the perception of natural object shape. In the experiments, younger and older observers performed either a same/different shape discrimination task (experiment 1) or a cross-modal matching task (experiment 2). Quantitative effects of age were found in both experiments. The effect of age in experiment 1 was limited to cross-modal shape discrimination: there was no effect of age upon unimodal (ie within a single perceptual modality) shape discrimination. The effect of age in experiment 2 was eliminated when the older observers were either given an unlimited amount of time to perform the task or when the number of response alternatives was decreased. Overall, the results of the experiments reveal that older observers can effectively perceive 3-D shape from both vision and haptics.     

Effective 3-D shape discrimination survives retinal blur

A single experiment evaluated observers’ ability to visually discriminate 3-D object shape, where the 3-D structure was defined by motion, texture, Lambertian shading, and occluding contours. The observers’ vision was degraded to varying degrees by blurring the experimental stimuli, using 2.0-, 2.5-, and 3.0-diopter convex lenses. The lenses reduced the observers’ acuity from ?0.091 LogMAR (in the no-blur conditions) to 0.924 LogMAR (in the conditions with the most blur; 3.0-diopter lenses). This visual degradation, although producing severe reductions in visual acuity, had only small (but significant) effects on the observers’ ability to discriminate 3-D shape. The observers’ shape discrimination performance was facilitated by the objects’ rotation in depth, regardless of the presence or absence of blur. Our results indicate that accurate global shape discrimination survives a considerable amount of retinal blur.     

Real-world face recognition: the importance of surface reflectance properties

The face recognition task we perform [corrected] most often in everyday experience is the identification of people with whom we are familiar. However, because of logistical challenges, most studies focus on unfamiliar-face recognition, wherein subjects are asked to match or remember images of unfamiliar people's faces. Here we explore the importance of two facial attributes -shape and surface reflectance-in the context of a familiar-face recognition task. In our experiment, subjects were asked to recognise color images of the faces of their friends. The images were manipulated such that only reflectance or only shape information was useful for recognizing any particular face. Subjects were actually better at recognizing their friends' faces from reflectance information than from shape information. This provides evidence that reflectance information is important for face recognition in ecologically relevant contexts.