Haptic interfaces for virtual environments: perceived instability and force constancy in haptic sensing of virtual surfaces.

Haptic interfaces are becoming more commonplace in virtual environment and teleoperation systems. There is a growing need to not only continue to improve hardware platforms and rendering algorithms, but evaluate human performance with haptic interfaces. This review summarizes two recent studies inspired by perception problems in using haptic interfaces to interact with virtual environments. The first study evaluated perceived quality of virtual haptic textures and discovered several types of perceived instability and their sources. We found that the buzzing type of perceived instability was most likely due to the mechanical resonance of the haptic interface hardware, and the aliveness type of perceived instability due to our inability to sense the slight movements of our hands in free space. The second study focused on the motor strategy employed during interaction with a virtual surface via a force-feedback haptic interface. We found that users tended to maintain a constant penetration force into a virtual surface when interacting with the surface. This can result in a reversal in perceived relative surface heights if the taller surface is rendered with a lower stiffness, thereby resulting in an erroneous perception of the virtual environment being rendered. For both studies, possible solutions to improving human perception of virtual and remote objects via hardware and/or software are discussed.     

Delayed memory for visual—haptic exploration of familiar objects

Long-term memory of haptic, visual, and cross-modality information was investigated. In Experiment 1, subjects briefly explored 40 commonplace objects visually or haptically and then received a recognition test with categorically similar foils in the same or the alternative modality both immediately and after 1 week. Recognition was best for visual input and test, with haptic memory still apparent after a week's delay. Recognition was poorest in the cross-modality conditions, with performance on the haptic-visual and visual-haptic cross-modal conditions being nearly identical. Visual and haptic information decayed at similar rates across a week delay. In Experiment 2, subjects simultaneously viewed and handled the same objects, and transfer was tested in a successive cue-modality paradigm. Performance with the visual modality again exceeded that with the haptic modality. Furthermore, initial errors on the haptic test were often corrected when followed by the visual presentation, both immediately and after 1 week. However, visual test errors were corrected by haptic cuing on the immediate test only. These results are discussed in terms of shared information between the haptic and visual modalities, and the ease of transfer between these modalities immediately and after a substantial delay.     

Understanding or feeling? Motor and sensory processes in recall of haptic experiences

Effects of learning can show in a direct, i.e., explicit way, or they can be expressed indirectly, i.e., in an implicit way. It is investigated whether hepatic information shows implicit effect, and whether implicit haptic memory effects are based primarily on motor or on sensory memory components. In the first phase blindfolded subjects had to palpate objects in order to answer questions about the objects' distinct properties as fast as possible. In the following phase this task was repeated with the same objects and additional control items. Additionally, recognition judgements were required. Results demonstrate reliable effects of implicit memory for haptic information in terms of reaction times to old vs. new objects. Subjects who had to wear plastic gloves in the first stage showed comparable effects of repetition priming. Changing the questions--and, thus, hand movements--during the palpitation of objects known from the first stage, however, abolishes implicit memory expression. It is concluded, therefore, that implicit memory for haptic information is based on motor processes. On the other hand, explicit memory is hampered in subjects wearing gloves during the first phase, as revealed in terms of recognition performance while changing the questions about objects' properties has no effect on recognition judgements. Thus, explicit memory for haptic information seems to be based on the sensory processes when touching objects.     

Evidence for a shared mechanism used in multiple-object tracking and subitizing

It has been proposed that the mechanism that supports the ability to keep track of multiple moving objects also supports subitizing--the ability to quickly and accurately enumerate a small set of objects. To test this hypothesis, we investigated the effects on subitizing when human observers were required to perform a multiple object tracking task and an enumeration task simultaneously. In three experiments, participants (Exp. 1, N = 24; Exp. 2, N = 11; Exp. 3, N = 37) enumerated sets of zero to nine squares that were flashed while they tracked zero, two, or four moving discs. The results indicated that the number of items participants could subitize decreased by one for each item they tracked. No such pattern was seen when the enumeration task was paired with an equally difficult, but nonvisual, working memory task. These results suggest that a shared visual mechanism supports multiple object tracking and subitizing.     

Visual memory and visual perception: when memory improves visual search

This study examined the relationship between memory and perception in order to identify the influence of a memory dimension in perceptual processing. Our aim was to determine whether the variation of typical size between items (i.e., the size in real life) affects visual search. In two experiments, the congruency between typical size difference and perceptual size difference was manipulated in a visual search task. We observed that congruency between the typical and perceptual size differences decreased reaction times in the visual search (Exp. 1), and noncongruency between these two differences increased reaction times in the visual search (Exp. 2). We argue that these results highlight that memory and perception share some resources and reveal the intervention of typical size difference on the computation of the perceptual size difference.     

Psychophysical functions for perceived and remembered distance

Two separate groups of subjects made magnitude estimations of the distances to fifteen objects (from 20 ft to 14.28 miles away) situated in a landscape. On day 1 of the experiment both groups learned the names and locations of the objects while viewing them from the top of a small mountain. On day 2 the perception group (N = 19) judged the distances to the objects while viewing them from the top of the mountain, whereas the memory group (N = 18) judged the distances while visualizing the landscape from memory. The data for both groups were well fit by power functions; the exponent was reliably smaller for the memory group (0.596) than for the perception group (0.811). Both groups drew maps of the landscape (reproduction task) from memory and the exponents were 0.483 and 0.514 for the memory and perception groups, respectively. The results are discussed in light of possible transformations performed on the original stimulus inputs by the sensory/perceptual and the memorial systems.     

Visuo-haptic integration in object identification using novel objects

Although some studies have shown that haptic and visual identification seem to rely on similar processes, few studies have directly compared the two. We investigated haptic and visual object identification by asking participants to learn to recognize (Experiments 1, and 3), or to match (Experiment 2) novel objects that varied only in shape. Participants explored objects haptically, visually, or bimodally, and were then asked to identify objects haptically and/or visually. We demonstrated that patterns of identification errors were similar across identification modality, independently of learning and testing condition, suggesting that the haptic and visual representations in memory were similar. We also demonstrated that identification performance depended on both learning and testing conditions: visual identification surpassed haptic identification only when participants explored the objects visually or bimodally. When participants explored the objects haptically, haptic and visual identification were equivalent. Interestingly, when participants were simultaneously presented with two objects (one was presented haptically, and one was presented visually), object similarity only influenced performance when participants were asked to indicate whether the two objects were the same, or when participants had learned about the objects visually—without any haptic input. The results suggest that haptic and visual object representations rely on similar processes, that they may be shared, and that visual processing may not always lead to the best performance.     

Visual size perception and haptic calibration during development

It is still unclear how the visual system perceives accurately the size of objects at different distances. One suggestion, dating back to Berkeley’s famous essay, is that vision is calibrated by touch. If so, we may expect different mechanisms involved for near, reachable distances and far, unreachable distances. To study how the haptic system calibrates vision we measured size constancy in children (from 6 to 16 years of age) and adults, at various distances. At all ages, accuracy of the visual size perception changes with distance, and is almost veridical inside the haptic workspace, in agreement with the idea that the haptic system acts to calibrate visual size perception. Outside this space, systematic errors occurred, which varied with age. Adults tended to overestimate visual size of distant objects (over‐compensation for distance), while children younger than 14 underestimated their size (under‐compensation). At 16 years of age there seemed to be a transition point, with veridical perception of distant objects. When young subjects were allowed to touch the object inside the haptic workspace, the visual biases disappeared, while older subjects showed multisensory integration. All results are consistent with the idea that the haptic system can be used to calibrate visual size perception during development, more effectively within than outside the haptic workspace, and that the calibration mechanisms are different in children than in adults.     

Interactive processing among parts, configuration, and orientation of complex objects

The aim of the present study was to investigate whether the representation of parts and the configuration and orientation of complex objects were separate or interdependent. Participants were asked to match parts and configuration of novel 3-D objects with constant or varied features in irrelevant dimensions. The results of both Exp. 1 (sequential matching) and Exp. 2 (simultaneous matching) showed that both sensitivity and response time (RT) of part matching were influenced by whether the irrelevant configuration remained constant or varied. Both sensitivity and RT for configuration matching were affected by the variation of irrelevant parts in Exp. 2, but only sensitivity was affected in Exp. 1. These results demonstrated an interdependent pattern of representations of parts and relations but different interactive patterns in memory and perception. The results of Exp. 3 showed orientation dependency in a configuration-matching task but orientation invariance in a part-matching task. Consistent with a study done in 2000 by Edelman and Intrator, these results suggest an interactive processing between configuration and orientation but an independent processing between the parts and orientation.     

Multisensory cortical processing of object shape and its relation to mental imagery

Here, we used functional magnetic resonance imaging to investigate the multisensory processing of object shape in the human cerebral cortex and explored the role of mental imagery in such processing. Regions active bilaterally during both visual and haptic shape perception, relative to texture perception in the respective modality, included parts of the superior parietal gyrus, the anterior intraparietal sulcus, and the lateral occipital complex. Of these bimodal regions, the lateral occipital complexes preferred visual over haptic stimuli, whereas the parietal areas preferred haptic over visual stimuli. Whereas most subjects reported little haptic imagery during visual shape perception, experiences of visual imagery during haptic shape perception were common. Across subjects, ratings of the vividness of visual imagery strongly predicted the amount of haptic shape-selective activity in the right, but not in the left, lateral occipital complex. Thus, visual imagery appears to contribute to activation of some, but not all, visual cortical areas during haptic perception.     

Discrimination threshold for haptic volume perception of fingers and phalanges

Humans exhibit a remarkable ability to discriminate variations in object volume based on natural haptic perception. The discrimination thresholds for the haptic volume perception of the whole hand are well known, but the discrimination thresholds for haptic volume perception of fingers and phalanges are still unknown. In the present study, two psychophysical experiments were performed to investigate haptic volume perception in various fingers and phalanges. The configurations of both experiments were completely dependent on haptic volume perception from the fingers and phalanges. The participants were asked to blindly discriminate the volume variation of regular solid objects in a random order by using the distal phalanx, medial phalanx, and proximal phalanx of their index finger, middle finger, ring finger, and little finger. The discrimination threshold of haptic volume perception gradually decreases from the little finger to the index finger as well as from the proximal phalanx to the distal phalanx. Overall, both the shape of the target and the part of the finger in contact with the target significantly influence the precision of haptic perception of volume. This substantial data set provides detailed and compelling perspectives on the haptic system, including for discrimination of the spatial size of objects and for performing more general perceptual processes.     

Congruency effect of presentation modality on false recognition of haptic and visual objects

Studying a list of items related to an item that is not presented (lure item) produces a false memory. We investigated whether a haptic study/test results in false recognition and, if so, whether congruency of presentation modality between study and test reduces the false recognition. After haptic or visual study of lists of real objects that are related to a lure object, participants were asked to recognise whether the objects were presented haptically or visually. We obtained false recognition results with haptic study and/or test. False recognition was reduced when presentation and study modalities were congruent. After haptic study, false recognition was reduced in the haptic test, as compared to the visual test. In contrast, visual study always reduced visual false recognition. These results indicate that there is a general effect of retrieval cues that will reduce false recognition.     

Mapping the tip of the tongue--deprivation, sensory sensitisation, and oral haptics

We investigated the impact of food deprivation on oral and manual haptic size perception of food and non-food objects. From relevant theories (need-proportional perception, motivated perception, frustrative nonreward, perceptual defence, and sensory sensitisation) at least four completely different competing predictions can be derived. Testing these predictions, we found across four experiments that participants estimated the length of both non-food and food objects to be larger when hungry than when satiated, which was true only for oral haptic perception, while manual haptic perception was not influenced by hunger state. Subjectively reported hunger correlated positively with estimated object size in oral, but not in manual, haptic perception. The impact of food deprivation on oral perception vanished after oral stimulations even for hungry individuals. These results favour a sensory sensitisation account maintaining that hunger itself does not alter oral perception but the accompanying lack of sensory stimulation of the oral mucosa. Both oral and manual haptic perception tended to underestimate actual object size. Finally, an enhancing effect of domain-target matching was found, ie food objects were perceived larger by oral than by manual haptics, while non-food objects were perceived larger by manual than by oral haptics.     

Repetition priming and the haptic recognition of familiar and unfamiliar objects

In four experiments, we examined the haptic recognition of 3-D objects. In Experiment 1, blindfolded participants named everyday objects presented haptically in two blocks. There was significant priming of naming, but no cost of an object changing orientation between blocks. However, typical orientations of objects were recognized more quickly than nonstandard orientations. In Experiment 2, participants accurately performed an unannounced test of memory for orientation. The lack of orientation-specific priming in Experiment 1, therefore, was not because participants could not remember the orientation at which they had first felt an object. In Experiment 3, we examined haptic naming of objects that were primed either haptically or visually. Haptic priming was greater than visual priming, although significant cross-modal priming was also observed. In Experiment 4, we tested recognition memory for familiar and unfamiliar objects using an old-new recognition task. Objects were recognized best when they were presented in the same orientation in both blocks, suggesting that haptic object recognition is orientation sensitive. Photographs of the unfamiliar objects may be downloaded from www.psychonomic.org/archive.     

Limited capacity for contour curvature in iconic memory

We measured the difference threshold for contour curvature in iconic memory by using the cued discrimination method. The study stimulus consisting of 2 to 6 curved contours was briefly presented in the fovea, followed by two lines as cues. Subjects discriminated the curvature of two cued curves. The cue delays were 0 msec. and 300 msec. in Exps. 1 and 2, respectively, and 50 msec. before the study offset in Exp. 3. Analysis of data from Exps. 1 and 2 showed that the Weber fraction rose monotonically with the increase in set size. Clear set-size effects indicate that iconic memory has a limited capacity. Moreover, clear set-size effect in Exp. 3 indicates that perception itself has a limited capacity. Larger set-size effects in Exp. 1 than in Exp. 3 suggest that iconic memory after perceptual process has limited capacity. These properties of iconic memory at threshold level are contradictory to the traditional view that iconic memory has a high capacity both at suprathreshold and categorical levels.     

Discrimination thresholds for haptic perception of volume, surface area, and weight

The present study investigated the human ability to discriminate the size of 3-D objects by touch. Experiment 1 measured the just noticeable differences (JNDs) for three tasks: (1) discrimination of volume without availability of weight information, (2) discrimination of volume with weight information available, and (3) discrimination of surface area. Stimuli consisted of spheres, cubes, and tetrahedrons. For all shapes, two reference sizes were used (3.5 and 12 cm(3)). No significant effect of task on the discriminability of objects was found, but the effects of shape and size were significant, as well as the interaction between these two factors. Post hoc analysis revealed that for the small reference, the Weber fractions for the tetrahedron were significantly larger than the fractions for the cube and the sphere. In Experiment 2, the JNDs for haptic perception of weight were measured for the same objects as those used in Experiment 1. The shape of objects had no significant effect on the Weber fractions for weight, but the Weber fractions for the small stimuli were larger than the fractions for the large stimuli. Surprisingly, a comparison between the two experiments showed that the Weber fractions for weight were significantly larger than the fractions for volume with availability of weight information. Taken together, the results reveal that volume and weight information are not effectively combined in discrimination tasks. This study provides detailed insight into the accuracy of the haptic system in discriminating objects' size. This substantial set of data satisfies the need for more fundamental knowledge on haptic size perception, necessary for a greater understanding of the perception of related properties, as well as of more general perceptual processes.     

Interacting with objects compresses environmental representations in spatial memory

People perceive individual objects as being closer when they have the ability to interact with the objects than when they do not. We asked how interaction with multiple objects impacts representations of the environment. Participants studied multiple-object layouts, by manually exploring or simply observing each object, and then drew a scaled version of the environment (Exp. 1) or reconstructed a copy of the environment and its boundaries (Exp. 2) from memory. The participants who interacted with multiple objects remembered these objects as being closer together and reconstructed smaller environment boundaries than did the participants who looked without touching. These findings provide evidence that action-based perceptual distortions endure in memory over a moving observer’s multiple interactions, compressing not only representations between touched objects, but also untouched environmental boundaries.     

Developmental change in young children's use of haptic information in a visual task: the role of hand movements

Preschoolers who explore objects haptically often fail to recognize those objects in subsequent visual tests. This suggests that children may represent qualitatively different information in vision and haptics and/or that children’s haptic perception may be poor. In this study, 72 children (2½-5 years of age) and 20 adults explored unfamiliar objects either haptically or visually and then chose a visual match from among three test objects, each matching the exemplar on one perceptual dimension. All age groups chose shape-based matches after visual exploration. Both 5-year-olds and adults also chose shape-based matches after haptic exploration, but younger children did not match consistently in this condition. Certain hand movements performed by children during haptic exploration reliably predicted shape-based matches but occurred at very low frequencies. Thus, younger children’s difficulties with haptic-to-visual information transfer appeared to stem from their failure to use their hands to obtain reliable haptic information about objects.     

Source memory in Parkinson's disease

Three experiments (ns = 14 per group) are reported which investigated the ability of Parkinson patients to remember the characteristics of conditions under which a memory was acquired. In Exp. 1, subjects were required to indicate for each item in a recognition memory test whether it was spoken by Experimenter 1 or by Experimenter 2 (external-external source memory). In Exp. 2, subjects had to indicate for each item whether it was generated by themselves or by the experimenter (internal-external source memory). In Exp. 3, subjects had to judge whether an item was generated by themselves in saying or in thinking (internal-internal source memory). We found that patients with Parkinson's disease were not impaired in the previous two kinds of source memory (Exp. 1 and 2) but were impaired in internal-internal source memory (Exp. 3) relative to the age-matched control groups. In addition, both groups' performance could be improved when given distinctive cues, i.e., perceptual cues in Exp. 1 and different-domain cues in Exp. 2. These results suggest that the availability of cues was critical for Parkinson's disease in source memory. Finally, the result of Exp. 2 also showed generation effects for patients with Parkinson's disease. The generation effect refers to better memory of information by people when they had to produce it, e.g., producing associates to a word, compared with memory of information given to them.     

Haptic classification of common objects: Knowledge-driven exploration

In Experiment 1, haptically available object properties that would be diagnostic for constrained common object classification at the basic and subordinate levels were elicited in a questionnaire. The results are considered in terms of the nature of the haptically derived representations of common objects. Initial data are also presented regarding knowledge of the natural co-occurrence of properties in haptic object perception. In Experiment 2, the hand movements executed during haptic classification of manipulable common objects were examined. Manual exploration consisted of a two-stage sequence, an initial generalized "grasp-and -lift" routine, followed by a series of more specialized hand-movement patterns strongly driven by knowledge of the property diagnosticity for the specific object (obtained in Experiment 1). The current results may guide computational models of human haptic object classification and the development of perceptual systems for robots equipped with sensate dextrous hands, capable of intelligent exploration, recognition, and manipulation of concrete objects.