Intersensory conflict between vision and touch: the response modality dominates when precise, attention-riveting judgments are required

In four experiments, reducing lenses were used to minify vision and generate intersensory size conflicts between vision and touch. Subjects made size judgments, using either visual matching or haptic matching. In visual matching, the subjects chose from a set of visible squares that progressively increased in size. In haptic matching, the subjects selected matches from an array of tangible wooden squares. In Experiment 1, it was found that neither sense dominated when subjects exposed to an intersensory discrepancy made their size estimates by using either visual matching or haptic matching. Size judgments were nearly indentical for conflict subjects making visual or haptic matches. Thus, matching modality did not matter in Experiment 1. In Experiment 2, it was found that subjects were influenced by the sight of their hands, which led to increases in the magnitude of their size judgments. Sight of the hands produced more accurate judgments, with subjects being better able to compensate for the illusory effects of the reducing lens. In two additional experiments, it was found that when more precise judgments were required and subjects had to generate their own size estimates, the response modality dominated. Thus, vision dominated in Experiment 3, where size judgments derived from viewing a metric ruler, whereas touch dominated in Experiment 4, where subjects made size estimates with a pincers posture of their hands. It is suggested that matching procedures are inadequate for assessing intersensory dominance relations. These results qualify the position (Hershberger & Misceo, 1996) that the modality of size estimates influences the resolution of intersensory conflicts. Only when required to self-generate more precise judgments did subjects rely on one sense, either vision or touch. Thus, task and attentional requirements influence dominance relations, and vision does not invariably prevail over touch.     

Cross-sensory transfer of reference frames in spatial memory

Two experiments investigated whether visual cues influence spatial reference frame selection for locations learned through touch. Participants experienced visual cues emphasizing specific environmental axes and later learned objects through touch. Visual cues were manipulated and haptic learning conditions were held constant. Imagined perspective taking when recalling touched objects was best from perspectives aligned with visually-defined axes, providing evidence for cross-sensory reference frame transfer. These findings advance spatial memory theory by demonstrating that multimodal spatial information can be integrated within a common spatial representation.     

The Müller-Lyer illusion in touch and vision: implications for multisensory processes

In six experiments, we used the Müller-Lyer illusion to investigate factors in the integration of touch, movement, and spatial cues in haptic shape perception, and in the similarity with the visual illusion. Latencies provided evidence against the hypothesis that scanning times explain the haptic illusion. Distinctive fin effects supported the hypothesis that cue distinctiveness contributes to the illusion, but showed also that it depends on modality-specific conditions, and is not the main factor. Allocentric cues from scanning an external frame (EF) did not reduce the haptic illusion. Scanning elicited downward movements and more negative errors for horizontal convergent figures and more positive errors for vertical divergent figures, suggesting a modality-specific movement effect. But the Müller-Lyer illusion was highly significant for both vertical and horizontal figures. By contrast, instructions to use body-centered reference and to ignore the fins reduced the haptic illusion for vertical figures in touch from 12.60% to 1.7%. In vision, without explicit egocentric reference, instructions to ignore fins did not reduce the illusion to near floor level, though external cues were present. But the visual illusion was reduced to the same level as in touch with instructions that included the use of body-centered cues. The new evidence shows that the same instructions reduced the Müller-Lyer illusion almost to zero in both vision and touch. It suggests that the similarity of the illusions is not fortuitous. The results on touch supported the hypothesis that body-centered spatial reference is involved in integrating inputs from touch and movement for accurate haptic shape perception. The finding that explicit egocentric reference had the same effect on vision suggests that it may be a common factor in the integration of disparate inputs from multisensory sources.     

Similar mechanisms underlie curvature comparison by static and dynamic touch

In four experiments, we tested whether haptic comparison of curvature ranging from ?41m to +41m is qualitatively the same for static and for dynamic touch. In Experiments 1 and 3, we tested whether static and dynamic curvature discrimination are based on height differences, attitude (slope) differences, curvature differences, or a combination of these geometrical variables. It was found that both static and dynamic haptic curvature discrimination are based on attitude differences. In Experiments 2 and 4, we tested whether this mechanism leads to errors in the comparison of stimuli with different lengths for static and dynamic touch, respectively. If the judgments are based on attitude differences, subjects will make systematic errors in these comparisons. In both experiments, we found that subjects compared the curvatures of strips of the same length veridically, whereas they made systematic errors if they were required to compare the curvatures of strips of different lengths. Longer stimuli were judged to be more curved than shorter stimuli with the same curvature. We conclude that similar mechanisms underlie static and dynamic haptic curvature comparison. Moreover, additional data comparison showed that static and dynamic curvature comparison is not only qualitatively, but also quantitatively similar.     

Haptic experiences influence visually acquired memories: Reference frames during multimodal spatial learning

In two experiments, we investigated whether reference frames acquired through touch could influence memories for locations learned through vision. Participants learned two objects through touch, and haptic egocentric (Experiment 1) and environmental (Experiment 2) cues encouraged selection of a specific reference frame. Participants later learned eight new objects through vision. Haptic cues were manipulated, whereas visual learning was held constant in order to observe any potential influence of the haptically experienced reference frame on memories for visually learned locations. When the haptically experienced reference frame was defined primarily by egocentric cues, cue manipulation had no effect on memories for objects learned through vision. Instead, visually learned locations were remembered using a reference frame selected from the visual study perspective. When the haptically experienced reference frame was defined by both egocentric and environmental cues, visually learned objects were remembered in the context of the haptically experienced reference frame. These findings support the common reference frame hypothesis, which proposes that locations learned through different sensory modalities are represented within a common reference frame.     

Visual size perception and haptic calibration during development

It is still unclear how the visual system perceives accurately the size of objects at different distances. One suggestion, dating back to Berkeley’s famous essay, is that vision is calibrated by touch. If so, we may expect different mechanisms involved for near, reachable distances and far, unreachable distances. To study how the haptic system calibrates vision we measured size constancy in children (from 6 to 16 years of age) and adults, at various distances. At all ages, accuracy of the visual size perception changes with distance, and is almost veridical inside the haptic workspace, in agreement with the idea that the haptic system acts to calibrate visual size perception. Outside this space, systematic errors occurred, which varied with age. Adults tended to overestimate visual size of distant objects (over‐compensation for distance), while children younger than 14 underestimated their size (under‐compensation). At 16 years of age there seemed to be a transition point, with veridical perception of distant objects. When young subjects were allowed to touch the object inside the haptic workspace, the visual biases disappeared, while older subjects showed multisensory integration. All results are consistent with the idea that the haptic system can be used to calibrate visual size perception during development, more effectively within than outside the haptic workspace, and that the calibration mechanisms are different in children than in adults.     

Similar mechanisms underlie curvature comparison by static and dynamic touch.

In four experiments, we tested whether haptic comparison of curvature ranging from -4/m to +4/m is qualitatively the same for static and for dynamic touch. In Experiments 1 and 3, we tested whether static and dynamic curvature discrimination are based on height differences, attitude (slope) differences, curvature differences, or a combination of these geometrical variables. It was found that both static and dynamic hepatic curvature discrimination are based on attitude differences. In Experiments 2 and 4, we tested whether this mechanism leads to errors in the comparison of stimuli with different lengths for static and dynamic touch, respectively. If the judgments are based on attitude differences, subjects will make systematic errors in these comparisons. In both experiments, we found that subjects compared the curvatures of strips of the same length vertically, whereas they made systematic errors if they were required to compare the curvatures of strips of different lengths. Longer stimuli were judged to be more curved than shorter stimuli with the same curvature. We conclude that similar mechanisms underlie static and dynamic haptic curvature comparison. Moreover, additional data comparison showed that static and dynamic curvature comparison is not only qualitatively, but also quantitatively similar.     

Touch dominates haptic estimates of discordant visual-haptic size

Observers (72 college students) estimated the size of plastic squares that they held in their fingers and simultaneously viewed through a reducing lens that halved the squares’ visual size. The squares were grasped from below through a cloth that prevented direct sight of the hand. Each estimate was a match selected later, either haptically or visually, from a set of comparison squares. Vision dominated the visual estimates and touch dominated the haptic estimates, whether or not the observers knew in advance which type of estimate they would be asked to make. Neither modality inherently dominates perceived size.     

What aspects of vision facilitate haptic processing?

We investigate how vision affects haptic performance when task-relevant visual cues are reduced or excluded. The task was to remember the spatial location of six landmarks that were explored by touch in a tactile map. Here, we use specially designed spectacles that simulate residual peripheral vision, tunnel vision, diffuse light perception, and total blindness. Results for target locations differed, suggesting additional effects from adjacent touch cues. These are discussed. Touch with full vision was most accurate, as expected. Peripheral and tunnel vision, which reduce visuo-spatial cues, differed in error pattern. Both were less accurate than full vision, and significantly more accurate than touch with diffuse light perception, and touch alone. The important finding was that touch with diffuse light perception, which excludes spatial cues, did not differ from touch without vision in performance accuracy, nor in location error pattern. The contrast between spatially relevant versus spatially irrelevant vision provides new, rather decisive, evidence against the hypothesis that vision affects haptic processing even if it does not add task-relevant information. The results support optimal integration theories, and suggest that spatial and non-spatial aspects of vision need explicit distinction in bimodal studies and theories of spatial integration.     

The role of haptic versus visual volume cues in the size-weight illusion

Three experiments establish the size-weight illusion as a primarily haptic phenomenon, despite its having been more traditionally considered an example of vision influencing haptic processing. Experiment 1 documents, across a broad range of stimulus weights and volumes, the existence of a purely haptic size-weight illusion, equal in strength to the traditional illusion. Experiment 2 demonstrates that haptic volume cues are both sufficient and necessary-for a full-strength illusion. In contrast, visual volume cues are merely sufficient, and produce a relatively weaker effect. Experiment 3 establishes that congenitally blind subjects experience an effect as powerful as that Of blindfolded sighted observers, thus demonstrating that visual imagery is also unnecessary for a robust size-weight illusion. The results are discussed in terms of their implications for both sensory and cognitive theories of the size-weight illusion. Applications of this work to a human factors design and to sensor-based systems for robotic manipulation are also briefly considered.     

触动觉对词汇学习和回忆的作用

本研究通过两个实验采用单因素被试内设计研究了触动觉对无意义词汇学习和回忆的作用。自变量设置了3 个水平,视觉学习—视觉回忆(水平1),视觉触动觉结合学习—视觉回忆(水平2),视觉触动觉结合学习—视觉触动觉结合回忆(水平3)。结果表明,在即时回忆条件下,触动觉对3字符词汇的学习和回忆都没有明显的促进作用。但是对5 字符词汇而言,在学习过程中,加入触动觉会明显提高学习成绩;在回忆过程中加不加触动觉对成绩影响不大。     

The effects of curvature on haptic judgments of extent in sighted and blind people

A series of experiments was carried out to examine the effect of curvature on haptic judgments of extent in sighted and blind individuals. Experiment 1 showed that diameters connecting the endpoints of semicircular lines were underestimated with respect to straight lines, but failed to show an effect of visual experience on length judgments. In experiment 2 we tested are lengths. The effects of curvature on perceived path length were weaker, but were still present in this experiment. Visual experience had no effect on path length judgments. Another experiment was performed to examine the effect of repeated tracing (1, 5, 9, or unlimited number of traces) on judgments of the lengths of straight lines and diameters of semicircles. Judgments of extent were more accurate when subjects engaged in larger numbers of traces. There was no effect of number of traces on curve-height judgments, suggesting that subjects were not using height estimates to judge diameters of semicircles. In a further experiment we tested the effect of number of traces on curves that varied in height. Restricting subjects to a single trace magnified the effect of path length on judgments of the distance between the endpoints of curves. Additional experiments showed that curvature effects on diameter judgments were not eliminated when stimuli were in the frontal plane or when the curves were explored with the use of two hands. Arm support had no effect on judged length in experiment 7. A final experiment showed a robust horizontal vertical illusion in haptic perception of convex curves, with overestimation of the heights of the curves compared with their widths. The practical and theoretical implications of the results are discussed.     

Euclidean metric representations of haptically explored triangles

This study explored whether people create Euclidean representations of 2-dimensional right triangles from touch and use them to make spatial inferences in accord with Euclidean distance axioms. Blindfolded participants who were instructed to form visual images of triangles felt the vertical and horizontal sides of right triangles, then estimated the lengths (but not the angles) of the 3 triangle sides. In these 3 experiments, length estimates conformed closely to the Euclidean metric when evaluated on application of the Pythagorean theorem. Participants who used a visual imaging strategy were accurate more often than those who used visual imagery less often. In Experiments 2 and 3, a hypotenuse inference was as accurate as a direct haptic judgment of the hypotenuse. These results demonstrated similar accuracy of the hypotenuse judgments when participants made verbal rather than haptic estimates. The findings indicate that participants can form Euclidean representations under certain conditions from felt 2-dimensional right triangles based on visual images.     

Self-training of dynamic touch: Striking improves judgment by wielding

In traditional theories of perceptual learning, sensory modalities support one another. A good example comes from research on dynamic touch, the wielding of an unseen object to perceive its properties. Wielding provides the haptic system with mechanical information related to the length of the object. Visual feedback can improve the accuracy of subsequent length judgments; visual perception supports haptic perception. Such cross-modal support is not the only route to perceptual learning. We present a dynamic touch task in which we replaced visual feedback with the instruction to strike the unseen object against an unseen surface following length judgment. This additional mechanical information improved subsequent length judgments. We propose a self-organizing perspective in which a single modality trains itself.     

Haptic distal spatial perception mediated by strings: haptic "looming"

Five experiments tested a haptic analog of optical looming, demonstrating string-mediated haptic distal spatial perception. Horizontally collinear hooks supported a weighted string held taut by a blindfolded participant's finger midway between the hooks. At the finger, the angle between string segments increased as the finger approached collinearity with the hooks, just as the optical angle subtended by an approaching object increases. The vertical force component at the finger is potentially informative for perception, approaching zero at finger-hook collinearity. In Experiment 1, participants judged hook height reasonably accurately. To retain force relationships but eliminate immediate skin contact, Experiment 2 employed a hand-held rod; results replicated those of Experiment 1. Experiment 3 replicated Experiment 2 using a ring instead of a rod. In all three experiments, estimated hook height closely paralleled actual height, r > .9. Experiments 4 and 5 showed that participants could project impending finger-hook collinearity when finger contact with the string was interrupted during its traverse. Perceivers' estimate errors were nearly perfectly predicted by height-force ratio relationships (Rs > .96). Outcomes are related to optical and acoustic looming, dynamic touch, tau theory, and Gibsonian perceptual theory.     

Roughness perception of unfamiliar dot pattern textures

Both vision and touch yield comparable results in terms of roughness estimation of familiar textures as was shown in earlier studies. To our knowledge, no research has been conducted on the effect of sensory familiarity with the stimulus material on roughness estimation of unfamiliar textures.The influence of sensory modality and familiarity on roughness perception of dot pattern textures was investigated in a series of five experiments. Participants estimated the roughness of textures varying in mean center-to-center dot spacing in experimental conditions providing visual, haptic and visual–haptic combined information.The findings indicate that roughness perception of unfamiliar dot pattern textures is well described by a bi-exponential function of inter-dot spacing, regardless of the sensory modality used. However, sensory modality appears to affect the maximum of the psychophysical roughness function, with visually perceived roughness peaking for a smaller inter-dot spacing than haptic roughness. We propose that this might be due to the better spatial acuity of the visual modality. Individuals appeared to use different visual roughness estimation strategies depending on their first sensory experience (visual vs. haptic) with the stimulus material, primarily in an experimental context which required the combination of visual and haptic information in a single bimodal roughness estimate. Furthermore, the similarity of findings in experimental settings using real and virtual visual textures indicates the suitability of the experimental setup for neuroimaging studies, creating a more direct link between behavioral and neuroimaging results.     

Vertical and bisection bias in active touch

We investigated the conditions that underlie the vertical and bisection illusion in touch, in order to understand the basis of their similarity to visual illusions, and the means of reducing the biases in length perception by active touch. Movement, speed, and spatial reference cues were tested. Movements in scanning L-shapes in ipsilateral and contralateral (across the body midline) table-top space produced significant underestimation of the vertical line with the right hand, but not with the left hand. Right-handed scanning of L-shapes showed no significant bias when the vertical line in the figure was aligned to the body midline, suggesting that spatial cues were involved. The vertical line was overestimated in inverted T-shapes, but underestimated in rotated T-shapes, implicating line bisection. Holding scanning latencies constant reduced the vertical error for inverted T-shapes, but could not explain the bisection bias. Sectioning biases were predicted by the location of junctions on sectioned lines, showing that junction points act as misleading anchor cues for movement extents. The illusion was significantly reduced when reference information was added by instructing subjects to relate two-handed scanning of the figure to an external frame and to body-centred cues. It is argued that disparities in spatial reference (anchor) cues for movement extents are involved in vertical and bisection biases in active touch. The hypothesis that length illusions depend on disparities in spatial reference information can also account for the similarity of the tactile to the visual horizontal-vertical illusion.     

Haptic estimates of discordant visual-haptic size vary developmentally.

In two size-conflict experiments, children viewed various squares through a reducing (1/2) lens while manually grasping them through a hand-concealing cloth. Then, using either vision or touch, they selected a match from a set of comparison squares. Forty 6-, 9-, and 12-year-olds participated in Experiment 1. Vision dominated the visual estimates of all three age groups; however, for the haptic estimates, the dominant modality varied developmentally: Vision dominated the 6-year-olds' haptic estimates, whereas neither modality dominated the 9-year-olds' haptic estimates, and touch dominated the 12-year-olds' haptic estimates. In Experiment 2, 24 six-year-olds were tested, as in Experiment 1; however, half of them were shown the size-distorting effects of the lens just prior to testing. Although this reduced the visual dominance of their haptic estimates, the effect was weak and short-lived. The haptic dominance seen in the data of the 12-year-olds was conspicuously absent.     

Haptic estimates of discordant visual—haptic size vary developmentally

In two size-conflict experiments, children viewed various squares through a reducing (1/2) lens while manually grasping them through a hand-concealing cloth. Then, using either vision or touch, they selected a match from a set of comparison squares. Forty 6-, 9-, and 12-year-olds participated in Experiment 1. Vision dominated the visual estimates of all three age groups; however, for the haptic estimates, the dominant modality varied developmentally: Vision dominated the 6-year-olds’ haptic estimates, whereas neither modality dominated the 9-year-olds’ haptic estimates, and touch dominated the 12-ear-olds’ haptic estimates. In Experiment 2, 24 six-year-olds were tested, as in Experiment 1; however, half of them were shown the size-distorting effects of the lens just prior to testing. Although this reduced the visual dominance of their haptic estimates, the effect was weak and short-lived. The haptic dominance seen in the data of the 12-year-olds was conspicuously absent.     

Attenuating the haptic horizontal—vertical curvature illusion

In a number of experiments, blindfolded subjects traced convex curves whose verticals were equal to their horizontal extent at the base. Overestimation of verticals, as compared with horizontals, was found, indicating the presence of a horizontal-vertical illusion with haptic curves, as well as with visible curves. Experiment 1 showed that the illusion occurred with stimuli in the frontal plane and with stimuli that were flat on the table surface in vision and touch. In the second experiment, the stimuli were rotated, and differences between vision and touch were revealed, with a stronger illusion in touch. The haptic horizontal-vertical illusion was virtually eliminated when the stimuli were bimanually touched using free exploration at the body midline, but a strong illusion was obtained when curves were felt with two index fingers or with a single hand at the midline. Bimanual exploration eliminated the illusion for smaller 2.5- through 10.2-cm stimuli, but a weakened illusion remained for the largest 12.7-cm patterns. The illusion was present when the stimuli were bimanually explored in the left and right hemispace. Thus, the benefits of bimanual exploration derived from the use of the two hands at the body midline combined with free exploration, rather than from bimanual free exploration per se. The results indicate the importance of haptic exploration at the body midline, where the body can serve as a familiar reference metric for size judgments. Alternative interpretations of the results are discussed, including the impact of movement-based heuristics as a causal factor for the illusion. It was suggested that tracing the curve’s peak served to bisect the curve in haptics, because of the change in direction.