The visual and haptic perception of natural object shape

In this study, we evaluated observers' ability to compare naturally shaped three-dimensional (3-D) objects, using their senses of vision and touch. In one experiment, the observers haptically manipulated 1 object and then indicated which of 12 visible objects possessed the same shape. In the second experiment, pairs of objects were presented, and the observers indicated whether their 3-D shape was the same or different. The 2 objects were presented either unimodally (vision-vision or haptic-haptic) or cross-modally (vision-haptic or haptic-vision). In both experiments, the observers were able to compare 3-D shape across modalities with reasonably high levels of accuracy. In Experiment 1, for example, the observers' matching performance rose to 72% correct (chance performance was 8.3%) after five experimental sessions. In Experiment 2, small (but significant) differences in performance were obtained between the unimodal vision-vision condition and the two cross-modal conditions. Taken together, the results suggest that vision and touch have functionally overlapping, but not necessarily equivalent, representations of 3-D shape.     

Aging and the haptic perception of 3D surface shape

Two experiments evaluated the ability of older and younger adults to perceive the three-dimensional (3D) shape of object surfaces from active touch (haptics). The ages of the older adults ranged from 64 to 84 years, while those of the younger adults ranged from 18 to 27 years. In Experiment 1, the participants haptically judged the shape of large (20 cm diameter) surfaces with an entire hand. In contrast, in Experiment 2, the participants explored the shape of small (5 cm diameter) surfaces with a single finger. The haptic surfaces varied in shape index (Koenderink, Solid shape, 1990; Koenderink, Image and Vision Computing, 10, 557-564, 1992) from -1.0 to +1.0 in steps of 0.25. For both types of surfaces (large and small), the participants were able to judge surface shape reliably. The older participants' judgments of surface shape were just as accurate and precise as those of the younger participants. The results of the current study demonstrate that while older adults do possess reductions in tactile sensitivity and acuity, they nevertheless can effectively perceive 3D surface shape from haptic exploration.     

Symmetry in haptic and in visual shape perception

Four experiments tested the hypothesis that bilateral symmetry is an incidental encoding property in vision, but can also be elicited as an incidental effect in touch, provided that sufficient spatial reference information is available initially for haptic inputs to be organized spatially. Experiment 1 showed that symmetry facilitated processing in vision, even though the task required judgments of stimulus closure rather than the detection of symmetry. The same task and stimuli failed to show symmetry effects in tactual scanning by one finger (Experiment 2). Experiment 3 found facilitating effects for vertically symmetric open stimuli, although not for closed patterns, in two-forefinger exploration when the fore-fingers had previously been aligned to the body midaxis to provide body-centered spatial reference. The one-finger exploration condition again failed to show symmetry effects. Experiment 4 replicated the facilitating effects of symmetry for open symmetric shapes in tactual exploration by the two (previously aligned) forefingers. Closed shapes again showed no effect. Spatial-reference information, finger movements, and stimulus factors in shape perception by touch are discussed.     

Anisotropy in haptic curvature and shape perception

An investigation was undertaken into whether haptic comparison of curvature and of shape is influenced by the length/width ratio of the hand. For this purpose three experiments were conducted to test the curvature matching of curved strips (experiment 1), the curvature matching of cylindrically curved hand-sized surfaces (experiment 2), and the shape discrimination of elliptically curved hand-sized surfaces (experiment 3). The orientation of the stimuli with respect to the fingers was varied. The results of the two matching experiments showed that a given curvature is judged to be more curved when touched along the fingers than when touched across the fingers. The phenomenal flatness along and across the fingers was found to be different and subject dependent. The results of the shape-discrimination experiment showed that the orientation of ellipsoidal surfaces influences the judgments of the shapes of these surfaces. This influence could be predicted on the basis of results of the second matching experiment. It is concluded that similar mechanisms underlie the (anisotropic) perception of curvature and shape. For the major part the trends in the results can be explained by the length/width ratio of the hand and the phenomenal flatnesses.     

Influence of shape on haptic curvature perception

Curvature discrimination of hand-sized doubly curved surfaces by means of static touch was investigated. Stimuli consisted of hyperbolical, cylindrical, elliptical and spherical surfaces of various curvatures. In the first experiment subjects had to discriminate the curvature along a specified orientation (the discrimination orientation) of a doubly curved surface from a flat surface. The curvature to be discriminated was oriented either along the middle finger or across the middle finger of the right hand. Independent of the shape of the surface, thresholds were found to be about 1.6 times smaller along the middle finger than across the middle finger. Discrimination biases were found to be strongly influenced by the shape of the surface; subjects judged a curvature to be more convex when the perpendicular curvature was convex than when this curvature was concave. With the results of the second experiment it could be ruled out that the influence of shape on curvature perception was simply due to a systematic error made by the subject regarding the discrimination orientation.     

Viewpoint dependence in visual and haptic object recognition

On the whole, people recognize objects best when they see the objects from a familiar view and worse when they see the objects from views that were previously occluded from sight. Unexpectedly, we found haptic object recognition to be viewpoint-specific as well, even though hand movements were unrestricted. This viewpoint dependence was due to the hands preferring the back "view" of the objects. Furthermore, when the sensory modalities (visual vs. haptic) differed between learning an object and recognizing it, recognition performance was best when the objects were rotated back-to-front between learning and recognition. Our data indicate that the visual system recognizes the front view of objects best, whereas the hand recognizes objects best from the back.     

The perception and recognition of natural object shape from deforming and static shadows

In this study of the informativeness of shadows for the perception of object shape, observers viewed shadows cast by a set of natural solid objects and were required to discriminate between them. In some conditions the objects underwent rotation in depth while in other conditions they remained stationary, thus producing both deforming and static shadows. The orientation of the light source casting the shadows was also varied, leading to further alterations in the shape of the shadows. When deformations in the shadow boundary were present, the observers were able to reliably recognize and discriminate between the objects, invariant over the shadow distortions produced by movements of the light source. The recognition performance for the static shadows depended critically upon the content of the specific views that were shown. These results support the idea that there are invariant features of shadow boundaries that permit the recognition of shape (cf Koenderink, 1984 Perception 13 321-330).     

Infants' haptic perception of object unity in rotating displays

Four-month-old infants were allowed to manipulate, without vision, two rings attached to a bar that permitted each ring to undergo rotary motion against a fixed surface. In different conditions, the relative motions of the rings were rigid, independent, or opposite, and they circled either the same fixed point outside the zone of manipulation or spatially separated points. Infants' perception of the ring assemblies were affected by the nature of the rotary motion in two ways. First, infants perceived a unitary object when the felt ends of the object underwent a common, rigid rotary motion; perception of object unity was stronger in this condition than when the ends underwent either independent or opposite rotary motions. Second, infants perceived two distinct objects when the felt ends of the objects underwent independent rotary motions that centred on distinct fixed points. Perception of the distinctness of the objects was less clear when the ends underwent opposite or independent rotary motions that centred on a common fixed point. These findings provide the first evidence that infants are sensitive to rotary motion patterns and can extrapolate a global pattern of rigid motion from the distinct, local velocities that they produce and experience at their two hands.     

Salient features in 3-D haptic shape perception

Shape is an important cue for recognizing an object by touch. Several features, such as edges, curvature, surface area, and aspect ratio, are associated with 3-D shape. To investigate the saliency of 3-D shape features, we developed a haptic search task. The target and distractor items consisted of shapes (cube, sphere, tetrahedron, cylinder, and ellipsoid) that differed in several of these features. Exploratory movements were left as unconstrained as possible. Our results show that this type of haptic search task can be performed very efficiently (25 msec/item) and that edges and vertices are the most salient features. Furthermore, very salient local features, such as edges, can also be perceived through enclosure, an exploratory procedure usually associated with global shape. Since the subjects had to answer as quickly as possible, this suggests that speed may be a factor in selecting the appropriate exploratory procedure.     

The role of contour polarity,objectness, and regularities in haptic and visual perception

Regularities like symmetry (mirror reflection) and repetition (translation) play an important role in both visual and haptic (active touch) shape perception. Altering figure-ground factors to change what is perceived as an object influences regularity detection. For vision, symmetry is usually easier to detect within one object, whereas repetition is easier to detect across two objects. For haptics, we have not found this interaction between regularity type and objectness (Cecchetto & Lawson, Journal of Experimental Psychology: Human Perception and Performance, 43, 103–125, 2017; Lawson, Ajvani, & Cecchetto, Experimental Psychology, 63, 197–214, 2016). However, our studies used repetition stimuli with mismatched concavities, convexities, and luminance, and so had mismatched contour polarities. Such stimuli may be processed differently to stimuli with matching contour polarities. We investigated this possibility. For haptics, speeded symmetry and repetition detection for novel, planar shapes was similar. Performance deteriorated strikingly if contour polarity mismatched (keeping objectness constant), whilst there was a modest disadvantage for between-2objects:facing-sides compared to within-1object:outer-sides comparisons (keeping contour polarity constant). For the same task for vision, symmetry detection was similar to haptics (strong costs for mismatched contour polarity, weaker costs for between-2objects:facing-sides comparisons), but repetition detection was very different (weak costs for mismatched contour polarity, strong benefits for between-2objects:facing-sides comparisons). Thus, objectness was less influential than contour polarity for both haptic and visual symmetry detection, and for haptic repetition detection. However, for visual repetition detection, objectness effects reversed direction (within-1object:outer-sides comparisons were harder) and were stronger than contour polarity effects. This pattern of results suggests that regularity detection reflects information extraction as well as regularity distributions in the physical world.     

Light source position in the perception of object shape

The apparent relief of monocularly viewed surfaces reversed when the order of light and shade was reversed relative to the position of a lamp observed the moment earlier. The pattern of shading was reversed either by illuminating from a direction opposite to that of the apparent direction of illumination or by inverting the illuminating image relative to the light source. The combination of both of these manipulations restores the original juxtaposition of light source and shading and reestablished accurate perception of relief. These results demonstrate that the perception of the relief of physical surfaces depends upon the remembered position of an apparent light source.     

The experience of force: the role of haptic experience of forces in visual perception of object motion and interactions, mental simulation, and motion-related judgments

Forces are experienced in actions on objects. The mechanoreceptor system is stimulated by proximal forces in interactions with objects, and experiences of force occur in a context of information yielded by other sensory modalities, principally vision. These experiences are registered and stored as episodic traces in the brain. These stored representations are involved in generating visual impressions of forces and causality in object motion and interactions. Kinematic information provided by vision is matched to kinematic features of stored representations, and the information about forces and causality in those representations then forms part of the perceptual interpretation. I apply this account to the perception of interactions between objects and to motions of objects that do not have perceived external causes, in which motion tends to be perceptually interpreted as biological or internally caused. I also apply it to internal simulations of events involving mental imagery, such as mental rotation, trajectory extrapolation and judgment, visual memory for the location of moving objects, and the learning of perceptual judgments and motor skills. Simulations support more accurate judgments when they represent the underlying dynamics of the event simulated. Mechanoreception gives us whatever limited ability we have to perceive interactions and object motions in terms of forces and resistances; it supports our practical interventions on objects by enabling us to generate simulations that are guided by inferences about forces and resistances, and it helps us learn novel, visually based judgments about object behavior.     

Curvature and the visual perception of shape: theory on information along object boundaries and the minima rule revisited

Previous empirical studies have shown that information along visual contours is known to be concentrated in regions of high magnitude of curvature, and, for closed contours, segments of negative curvature (i.e., concave segments) carry greater perceptual relevance than corresponding regions of positive curvature (i.e., convex segments). Lately, Feldman and Singh (2005, Psychological Review, 112, 243-252) proposed a mathematical derivation to yield information content as a function of curvature along a contour. Here, we highlight several fundamental errors in their derivation and in its associated implementation, which are problematic in both mathematical and psychological senses. Instead, we propose an alternative mathematical formulation for information measure of contour curvature that addresses these issues. Additionally, unlike in previous work, we extend this approach to 3-dimensional (3D) shape by providing a formal measure of information content for surface curvature and outline a modified version of the minima rule relating to part segmentation using curvature in 3D shape.     

A theory of visual interpolation in object perception

We describe a new theory explaining the perception of partly occluded objects and illusory figures, from both static and kinematic information, in a unified framework. Three ideas guide our approach. First, perception of partly occluded objects, perception of illusory figures, and some other object perception phenomena derive from a single boundary interpolation process. These phenomena differ only in respects that are not part of the unit formation process, such as the depth placement of units formed. Second, unit formation from static and kinematic information can be treated in the same general framework. Third, spatial and spatiotemporal discontinuities in the boundaries of optically projected areas are fundamental to the unit formation process. Consistent with these ideas, we develop a detailed theory of unit formation that accounts for most cases of boundary perception in the absence of local physical specification. According to this theory, discontinuities in the first derivative of projected edges are initiating conditions for unit formation. A formal notion of relatability is defined, specifying which physically given edges leading into discontinuities can be connected to others by interpolated edges. Intuitively, relatability requires that two edges be connectable by a smooth, monotonic curve. The roots of the discontinuity and relatability notions in ecological constraints on object perception are discussed. Finally, we elaborate our approach by discussing related issues, some new phenomena, connections to other approaches, and issues for future research.     

The visual perception of 3D shape

A fundamental problem for the visual perception of 3D shape is that patterns of optical stimulation are inherently ambiguous. Recent mathematical analyses have shown, however, that these ambiguities can be highly constrained, so that many aspects of 3D structure are uniquely specified even though others might be underdetermined. Empirical results with human observers reveal a similar pattern of performance. Judgments about 3D shape are often systematically distorted relative to the actual structure of an observed scene, but these distortions are typically constrained to a limited class of transformations. These findings suggest that the perceptual representation of 3D shape involves a relatively abstract data structure that is based primarily on qualitative properties that can be reliably determined from visual information.     

Influence of exploration time on haptic and visual matching of complex shape

The time adult Ss were allowed to explore stimuli was varied during intra- and cross-model equivalence matching involving vision and touch. Increasing time to explore either each standard, each comparison, or both standard and comparison from 4 to 16 sec significantly improved haptic intramodel matching. However, cross-modal matching, from either vision to touch or touch to vision, improved significantly only when time to explore each standard was increased. Videotape recordings of Ss’ hand movements revealed use of a greater variety of haptic scanning strategies by Ss in groups where increased exploration time enhanced accuracy. The difference in effects of exploration time on intra- compared to cross-model shape matching was discussed in terms of possible differences in requirements between the two tasks.     

Sound enhances visual perception: cross-modal effects of auditory organization on vision

Six experiments demonstrated cross-modal influences from the auditory modality on the visual modality at an early level of perceptual organization. Participants had to detect a visual target in a rapidly changing sequence of visual distractors. A high tone embedded in a sequence of low tones improved detection of a synchronously presented visual target (Experiment 1), but the effect disappeared when the high tone was presented before the target (Experiment 2). Rhythmically based or order-based anticipation was unlikely to account for the effect because the improvement was unaffected by whether there was jitter (Experiment 3) or a random number of distractors between successive targets (Experiment 4). The facilitatory effect was greatly reduced when the tone was less abrupt and part of a melody (Experiments 5 and 6). These results show that perceptual organization in the auditory modality can have an effect on perceptibility in the visual modality.     

Auditory,visual, and cross-modal negative priming

Auditory, visual, and cross-modal negative priming was investigated using a task in which participants judged whether stimuli were animals or musical instruments. Negative priming was observed, but only if the attended and the ignored primes evoked different responses. This pattern—negative priming after conflict, but not after nonconflict, primes—was demonstrated with visual stimuli and replicated with auditory stimuli, as well as across modalities, both auditory to visual and visual to auditory. Implications for theories of negative priming are discussed.