Controlled interaction: Strategies for using virtual reality to study perception

Immersive virtual reality systems employing head-mounted displays offer great promise for the investigation of perception and action, but there are well-documented limitations to most virtual reality systems. In the present article, we suggest strategies for studying perception/action interactions that try to depend on both scale-invariant metrics (such as power function exponents) and careful consideration of the requirements of the interactions under investigation. New data concerning the effect of pincushion distortion on the perception of surface orientation are presented, as well as data documenting the perception of dynamic distortions associated with head movements with uncorrected optics. A review of several successful uses of virtual reality to study the interaction of perception and action emphasizes scale-free analysis strategies that can achieve theoretical goals while minimizing assumptions about the accuracy of virtual simulations.     

The influence of sex and empathy on putting oneself in the shoes of others

We tested whether putting oneself in the shoes of others is easier for women, possibly as a function of individuals' empathy levels, and whether any sex difference might be modulated by the sex of presented figures. Participants (N=100, 50 women) imagined (a) being in the spatial position of front‐facing and back‐facing female and male figures (third person perspective (3PP) task) and (b) that the figures were their own mirror reflections (first person perspective (1PP) task). After mentally taking the figure's position, individuals decided whether the indicated hand of the figure would be their own left or right hand. Contrary to our hypothesis, results from the 3PP‐task showed higher rotational costs for women than men, suggesting that mental rotation rather than social strategies had been employed. However, faster responding by women with higher empathy scores would appear to indicate that some women engaged social perspective taking strategies irrespective of the figures' position. Figures' sex was relevant to task performance as higher rotational costs were observed for male figures in the 3PP‐task for both sexes and for female figures in the 1PP‐task for women. We argue that these latter findings indicate that performance was facilitated and/or inhibited towards figures associated with specific social and emotional implications.     

3-D shape perception

In this paper, we analyze and test three theories of 3-D shape perception: (1) Helmholtzian theory, which assumes that perception of the shape of an object involves reconstructing Euclidean structure of the object (up to size scaling) from the object’s retinal image after taking into account the object’s orientation relative to the observer, (2) Gibsonian theory, which assumes that shape perception involves invariants (projective or affine) computed directly from the object’s retinal image, and (3) perspective invariants theory, which assumes that shape perception involves a new kind of invariants of perspective transformation. Predictions of these three theories were tested in four experiments. In the first experiment, we showed that reliable discrimination between a perspective and nonperspective image of a random polygon is possible even when information only about the contour of the image is present. In the second experiment, we showed that discrimination performance did not benefit from the presence of a textured surface, providing information about the 3-D orientation of the polygon, and that the subjects could not reliably discriminate between the 3-D orientation of the textured surface and that of a shape. In the third experiment, we compared discrimination for solid shapes that either had flat contours (cuboids) or did not have visible flat contours (cylinders). The discrimination was very reliable in the case of cuboids but not in the case of cylinders. In the fourth experiment, we tested the effectiveness of planar motion in perception of distances and showed that the discrimination threshold was large and similar to thresholds when other cues to 3-D orientation were used. All these results support perspective invariants as a model of 3-D shape perception.     

Configural information is processed differently in human action

Recent evidence indicates that observers' sensitivity to configural information in dynamic human action is disrupted when action is inverted, whereas sensitivity to featural action information is not. The current research involved two experiments that expand upon this basic finding. Experiment 1 revealed that featural and configural action information are processed similarly in static representations of action as in dynamic action. Experiment 2 indicated that configural processing is uniquely sensitive to orientation only in human action as compared to a similar control stimulus. These findings further support the idea that the perception of action recruits specialized orientation-specific configural processing, and parallel similar findings in face perception and visual expertise.     

Manipulating perception versus action in recalibration tasks

We conducted six experiments to examine how manipulating perception versus action affects perception–action recalibration in real and imagined blindfolded walking tasks. Participants first performed a distance estimation task (pretest) and then walked through an immersive virtual environment on a treadmill for 10 min. Participants then repeated the distance estimation task (posttest), the results of which were compared with their pretest performance. In Experiments 1a, 2a, and 3a, participants walked at a normal speed during recalibration, but the rate of visual motion was either twice as fast or half as fast as the participants' walking speed. In Experiments 1b, 2b, and 3b, the rate of visual motion was kept constant, but participants walked at either a faster or a slower speed. During pre- and posttest, we used either a blindfolded walking distance estimation task or an imagined walking distance estimation task. Additionally, participants performed the pretest and posttest distance estimation tasks in either the real environment or the virtual environment. With blindfolded walking as the distance estimation task for pre- and posttest, we found a recalibration effect when either the rate of visual motion or the walking speed was manipulated during the recalibration phase. With imagined walking as the distance estimation task, we found a recalibration effect when the rate of visual motion was manipulated, but not when the walking speed was manipulated in both the real environment and the virtual environment. Discussion focuses on how spatial-updating processes operate on perception and action and on representation and action.     

Motor learning enhances perceptual judgment: a case for action-perception transfer

Transfer from perception to action is well documented, for instance in the form of observational learning. Transfer from action to perception, on the other hand, has not been researched. Such action-perception transfer (APT) is compatible with several learning theories and has been predicted within the framework of common coding of perceptual and motor events (Prinz, 1992, 1997). Our first experiment aimed at an empirical evaluation of APT and involved motor practice of timed two-cycle arm movements on verbal command without visual feedback. In a transfer test, visual judgments of similar patterns had to be made. In addition, transfer from the visual to the motor task was studied. In Experiment 2 we separated kinesthetic aspects of motor practice from preparatory and efferent contributions to APT. The experiments provide evidence that transfer between perception and action is bi-directional. Transfer from perception to action and, more importantly, from action to perception was found. Furthermore, APT was equally pronounced for participants who had actively practiced movements during training and for passive participants who had received merely kinesthetic feedback about the movement. This kinesthetic-visual transfer is likely to be achieved via visuomotor-kinesthetic matching or via timekeeping mechanisms that are involved in both motor and visual performance. Received: 2 June 1999 / Accepted: 20 June 2000     

Influence of cognitive stance and physical perspective on subjective and autonomic reactivity to observed pain and pleasure: An immersive virtual reality study

Observing others’ pain may induce a reaction called personal distress that may be influenced by top-down (imagine self or other in pain, i.e., self- vs other-oriented stance) and bottom-up (physical perspective of those who suffer, i.e., first vs third person perspective- 1PP vs 3PP) processes. The different contributions of these processes have not been teased apart. By capitalizing on the power of Immersive Virtual Reality, we explored how behavioural (subjective ratings) and physiological reactivity (skin conductance reactivity, SCR) to pain and pleasure delivered to an avatar was influenced by Cognitive stance and Physical perspective. Taking an Other-Oriented stance leads to attributing higher congruent valence (i.e. pain rated as unpleasant and pleasure as pleasant) and intensity to the stimuli and induces reduced SCR. Ownership over the virtual limb was maximal in 1PP where physiological reactivity to the stimuli was comparable. Our results highlight different components underpinning reactivity to pain and pleasure.     

Target-directed visual attention is a prerequisite for action-specific perception

Witt et al. (2008) have recently shown that golfers who putt with more success perceive the hole to be bigger than golfers who putt with less success. In three experiments, we systematically examined whether this phenomenon, labelled action-specific perception, depends on directing visual attention towards the action target. In Experiment 1 we replicated previously reported action-specific effects on perception in golf putting. In 3 and 4 we directly assessed whether action-specific effects on perception in golf putting are dependent on focusing visual attention on the target. To this end, the participants performed the putting task while visual attention towards the target was either completely withheld (Experiment 2) or divided over the target and other task-relevant objects (Experiment 3). No action-specific effects were found when visual attention towards the action target was occluded or partially diverted from the target. Together, our results provide evidence to suggest that focusing visual attention on the target while performing the action is a prerequisite for the emergence of action-specific perception.     

Pain modulation during drives through cold and hot virtual environments.

Evidence exists that virtual worlds reduce pain perception by providing distraction. However, there is no experimental study to show that the type of world used in virtual reality (VR) distraction influences pain perception. Therefore, we investigated whether pain triggered by heat or cold stimuli is modulated by "warm "or "cold " virtual environments and whether virtual worlds reduce pain perception more than does static picture presentation. We expected that cold worlds would reduce pain perception from heat stimuli, while warm environments would reduce pain perception from cold stimuli. Additionally, both virtual worlds should reduce pain perception in general. Heat and cold pain stimuli thresholds were assessed outside VR in 48 volunteers in a balanced crossover design. Participants completed three 4-minute assessment periods: virtual "walks " through (1) a winter and (2) an autumn landscape and static exposure to (3) a neutral landscape. During each period, five heat stimuli or three cold stimuli were delivered via a thermode on the participant's arm, and affective and sensory pain perceptions were rated. Then the thermode was changed to the other arm, and the procedure was repeated with the opposite pain stimuli (heat or cold). We found that both warm and cold virtual environments reduced pain intensity and unpleasantness for heat and cold pain stimuli when compared to the control condition. Since participants wore a head-mounted display (HMD) in both the control condition and VR, we concluded that the distracting value of virtual environments is not explained solely by excluding perception of the real world. Although VR reduced pain unpleasantness, we found no difference in efficacy between the types of virtual world used for each pain stimulus.     

Effects of perspective taking on courtroom decisions

Four experiments examined the hypothesis that perspective taking with a defendant would lead to greater empathy, which would mediate lowered perceptions of culpability, with lowered culpability mediating a lower probability of guilt and recidivism. Experiments 1 and 2 established that perspective taking leads to a lower probability of guilty verdicts and recidivism, mediated by a decreased perception of the defendant's culpability. Experiment 2 showed that it does so by increasing empathy. Experiment 3 showed that perspective taking also heightens the perception of culpability through increased empathy for the victim. Experiment 4 showed that decreased culpability is in part driven by leniency, which is also a function of empathy. We tie our findings into other research investigating links between empathy and perspective taking.     

Memory for target height is scaled to observer height

According to the embodied approach to visual perception, individuals scale the environment to their bodies. This approach highlights the central role of the body for immediate, situated action. The present experiments addressed whether body scaling—specifically, eye-height scaling—occurs in memory when action is not immediate. Participants viewed standard targets that were either the same height as, taller than, or shorter than themselves. Participants then viewed a comparison target and judged whether the comparison was taller or shorter than the standard target. Participants were most accurate when the standard target height matched their own heights, taking into account postural changes. Participants were biased to underestimate standard target height, in general, and to push standard target height away from their own heights. These results are consistent with the literature on eye-height scaling in visual perception and suggest that body scaling is not only a useful metric for perception and action, but is also preserved in memory.     

I might ease your pain,but only if you’re sad: The impact of the empathized emotion in the empathy-helping association

Ample research demonstrated that empathizing with someone in need promotes helping that person. Two studies examined whether this effect of empathy on helping behavior holds across different emotional reactions expressed by a target in need. Results of Study 1 indicate that perspective taking with a sad needy target increased empathic concern which, in turn, fostered helping the individual. This relation was not found for participants taking the perspective of angry or disgusted needy targets. Study 2 provides further support for the underlying mechanism of the results of Study 1. Perspective taking with a sad needy target increased empathizers’ empathic concern because perception of target neediness was increased. Again, this pattern was not found for perspective taking with an angry needy target. The findings correspond to theorizing on the role of emotions in person perception. Hence, the current research provides insights regarding the boundary conditions of the empathy-helping association.     

Eye gaze metrics reflect a shared motor representation for action observation and movement imagery

Action observation (AO) and movement imagery (MI) have been reported to share similar neural networks. This study investigated the congruency between AO and MI using the eye gaze metrics, dwell time and fixation number. A simple reach-grasp-place arm movement was observed and, in a second condition, imagined where the movement was presented from the first person perspective (1PP) and the third person perspective (3PP). Dwell time and number of fixations were calculated for whole scene and regions of interest (ROIs). For whole scene, no significant differences were found in the number of fixations for condition (AO, MI) or perspective. Dwell time, however, was significantly longer in AO than MI. For ROIs, the number of fixations was significantly greater in 1PP than 3PP. The data provide support for congruence between motor simulation states but also indicate some functional differences.     

More than just perception–action recalibration: Walking through a virtual environment causes rescaling of perceived space

Egocentric distances in virtual environments are commonly underperceived by up to 50 % of the intended distance. However, a brief period of interaction in which participants walk through the virtual environment while receiving visual feedback can dramatically improve distance judgments. Two experiments were designed to explore whether the increase in postinteraction distance judgments is due to perception–action recalibration or the rescaling of perceived space. Perception–action recalibration as a result of walking interaction should only affect action-specific distance judgments, whereas rescaling of perceived space should affect all distance judgments based on the rescaled percept. Participants made blind-walking distance judgments and verbal size judgments in response to objects in a virtual environment before and after interacting with the environment through either walking (Experiment 1) or reaching (Experiment 2). Size judgments were used to infer perceived distance under the assumption of size–distance invariance, and these served as an implicit measure of perceived distance. Preinteraction walking and size-based distance judgments indicated an underperception of egocentric distance, whereas postinteraction walking and size-based distance judgments both increased as a result of the walking interaction, indicating that walking through the virtual environment with continuous visual feedback caused rescaling of the perceived space. However, interaction with the virtual environment through reaching had no effect on either type of distance judgment, indicating that physical translation through the virtual environment may be necessary for a rescaling of perceived space. Furthermore, the size-based distance and walking distance judgments were highly correlated, even across changes in perceived distance, providing support for the size–distance invariance hypothesis.     

Use of static and dynamic media to simulate environments: a meta-analysis

This article is a review of studies on how strongly responses from static media (such as color slides) and dynamic media (such as virtual reality models) compared to onsite evaluations or to each other. Eighty-four empirical findings were found. 6,323 participants and 967 environments were included in the review. The overall correlation of subjective responses obtained either onsite or by viewing static color simulations was r = .86. The overall correlation of subjective responses obtained either onsite or by viewing dynamic simulations was r = .83. The overall correlation of subjective responses obtained either from static or dynamic media was r = .82. It is suggested that both types of simulation generate statistically equivalent results and so a choice of simulation media should be based on efficacy rather than concerns about validity.     

Perceptual resonance: action-induced modulation of perception

A direct relationship between perception and action implies bi-directionality, and predicts not only effects of perception on action but also effects of action on perception. Modern theories of social cognition have intensively examined the relation from perception to action and propose that mirroring the observed actions of others underlies action understanding. Here, we suggest that this view is incomplete, as it neglects the perspective of the actor. We will review empirical evidence showing the effects of self-generated action on perceptual judgments. We propose that producing action might prime perception in a way that observers are selectively sensitive to related or similar actions of conspecifics. Therefore, perceptual resonance, not motor resonance, might be decisive for grounding sympathy and empathy and, thus, successful social interactions.     

Expecting to lift a box together makes the load look lighter

The action abilities of an individual observer modulate his or her perception of spatial properties of the environment and of objects. The present study investigated how joint action abilities shape perception. Four experiments examined how the intention to lift an object with another individual affects perceived weight. In Experiments 1, 2a, and 2b, participants judged the perceived weight of boxes while expecting to lift them either alone or with a co-actor. In Experiment 3, the co-actor was healthy or injured. Participants intending to lift a box with a co-actor perceived the box as lighter than participants intending to lift the same box alone, provided that the co-actor appeared healthy and therefore capable of helping. These findings suggest that anticipated effort modulates the perception of object properties in the context of joint action. We discuss implications for the role of action prediction and action simulation processes in social interaction.