Goal-directed aiming under restricted viewing conditions with confirmatory sensory feedback

A substantial body of research has examined the speed-accuracy tradeoff captured by Fitts’ law, demonstrating increases in movement time that occur as aiming tasks are made more difficult by decreasing target width and/or increasing the distance between targets. Yet, serial aiming movements guided by internal spatial representations, rather than by visual views of targets have not been examined in this manner, and the value of confirmatory feedback via different sensory modalities within this paradigm is unknown. Here we examined goal-directed serial aiming movements (tapping back and forth between two targets), wherein targets were visually unavailable during the task. However, confirmatory feedback (auditory, haptic, visual, and bimodal combinations of each) was delivered upon each target acquisition, in a counterbalanced, within-subjects design. Each participant performed the aiming task with their pointer finger, represented within an immersive virtual environment as a 1 cm white sphere, while wearing a head-mounted display. Despite visual target occlusion, movement times increased in accordance with Fitts’ law. Though Fitts’ law captured performance for each of the sensory feedback conditions, the slopes differed. The effect of increasing difficulty on movement times was least influential in the haptic condition, suggesting more efficient processing of confirmatory haptic feedback during aiming movements guided by internal spatial representations.     

Comparison of grasping movements made by healthy subjects in a 3-dimensional immersive virtual versus physical environment

Virtual reality (VR) technology is being used with increasing frequency as a training medium for motor rehabilitation. However, before addressing training effectiveness in virtual environments (VEs), it is necessary to identify if movements made in such environments are kinematically similar to those made in physical environments (PEs) and the effect of provision of haptic feedback on these movement patterns. These questions are important since reach-to-grasp movements may be inaccurate when visual or haptic feedback is altered or absent. Our goal was to compare kinematics of reaching and grasping movements to three objects performed in an immersive three-dimensional (3D) VE with haptic feedback (cyberglove/grasp system) viewed through a head-mounted display to those made in an equivalent physical environment (PE). We also compared movements in PE made with and without wearing the cyberglove/grasp haptic feedback system. Ten healthy subjects (8 women, 62.1 ± 8.8 years) reached and grasped objects requiring 3 different grasp types (can, diameter 65.6 mm, cylindrical grasp; screwdriver, diameter 31.6 mm, power grasp; pen, diameter 7.5 mm, precision grasp) in PE and visually similar virtual objects in VE. Temporal and spatial arm and trunk kinematics were analyzed. Movements were slower and grip apertures were wider when wearing the glove in both the PE and the VE compared to movements made in the PE without the glove. When wearing the glove, subjects used similar reaching trajectories in both environments, preserved the coordination between reaching and grasping and scaled grip aperture to object size for the larger object (cylindrical grasp). However, in VE compared to PE, movements were slower and had longer deceleration times, elbow extension was greater when reaching to the smallest object and apertures were wider for the power and precision grip tasks. Overall, the differences in spatial and temporal kinematics of movements between environments were greater than those due only to wearing the cyberglove/grasp system. Differences in movement kinematics due to the viewing environment were likely due to a lack of prior experience with the virtual environment, an uncertainty of object location and the restricted field-of-view when wearing the head-mounted display. The results can be used to inform the design and disposition of objects within 3D VEs for the study of the control of prehension and for upper limb rehabilitation.     

Knowledge of Performance is Insufficient for Implicit Visuomotor Rotation Adaptation

The ability to adapt is a fundamental and vital characteristic of the motor system. The authors altered the visual environment and focused on the ability of humans to adapt to a rotated environment in a reaching task, in the absence of continuous visual information about their hand location. Subjects could not see their arm but were provided with post trial knowledge of performance depicting hand path from movement onset to final position. Subjects failed to adapt under these conditions. The authors sought to find out whether the lack of adaptation is related to the number of target directions presented in the task, and planned 2 protocols in which subjects were gradually exposed to 22.5° visuomotor rotation. These protocols differed only in the number of target directions: 8 and 4 targets. The authors found that subjects had difficulty adapting without the existence of continuous visual feedback of their performance regardless of the number of targets presented in task. In the 4-target protocol, some of the subjects noticed the rotation and explicitly aimed to the correct direction. The results suggest that real-time feedback is required for motor adaptation to visual rotation during reaching movements.     

Ipsilesional arm reaching movements are not affected by the postural configuration adopted by individuals with stroke

Individuals with stroke present several impairments in the ipsilesional arm reaching movements that can limit the execution of daily living activities. These impairments depend on the side of the brain lesion. The present study aimed to compare the arm reaching movements performed in sitting and standing positions and to examine whether the effects of the adopted posture configuration depend on the side of the brain lesion. Twenty right-handed individuals with stroke (half with right hemiparesis and a half with left hemiparesis) and twenty healthy adults (half used the left arm) reached toward a target displayed on a monitor screen placed in one of three heights (i.e., upper, central, or lower targets). Participants performed the reaches in sitting and standing positions under conditions where the target location was either well-known in advance (certainty condition) or unknown until the movement onset (uncertainty condition). The values of movement onset time, movement time, and constant error were compared across conditions (posture configuration and uncertainty) and groups for each target height. Individuals with stroke were slower and spent more time to start to move than healthy participants, mainly when they reached the superior target in the upright position and under the uncertainty condition. Individuals who have suffered a right stroke were more affected by the task conditions and those who suffered a left stroke showed less accurate reaches. Overall, these results were observed regardless of the adopted posture. The current findings suggested that ipsilesional arm reaching movements are not affected by the postural configuration adopted by individuals with stroke. The central nervous system modulates the reaching movements according to the target position, adopted posture, and the uncertainty in the final target position to be reached.     

Augmented Feedback Presented in a Virtual Environment Accelerates Learning of a Difficult Motor Task

One can use a number of techniques (e.g., from videotaping to computer enhancement of the environment) to augment the feedback that a subject usually receives during training on a motor task. Although some forms of augmented feedback have been shown to enhance performance on isolated isometric tasks during training, when the feedback has been removed subjects have sometimes not been able to perform as well in the "real-world" task as controls. Indeed, for realistic, nonisometric motor tasks, improved skill acquisition because of augmented feedback has not been demonstrated. In the present experiments, subjects (Experiment 1, N = 42; Experiment 2, N = 21) performed with a system that was designed for teaching a difficult multijoint movement in a table tennis environment. The system was a fairly realistic computer animation of the environment and included paddles for the teacher and subject, as well as a virtual ball. Each subject attempted to learn a difficult shot by matching the pattern of movements of the expert teacher. Augmented feedback focused the attention of the subject on a minimum set of movement details that were most relevant to the task; feedback was presented in a form that required the least perceptual processing. Effectiveness of training was determined by measuring their performance in the real task. Subjects who received the virtual environment training performed significantly better than subjects who received a comparable amount of real-task practice or coaching. Kinematic analysis indicated that practice with the expert's trajectory served as a basis for performance on the real-world task and that the movements executed after training were subject-specific modifications of the expert's trajectory. Practice with this trajectory alone was not sufficient for transfer to the real task, however: When a critical component of the virtual environment was removed, subjects showed no transfer to the real task.     

The Use of Prescanning in the Parameterization of Sequential Pointing and Reaching Movements

The accuracy of reaching movements improves when active gaze can be used to fixate on targets. The advantage of free gaze has been attributed to the use of ocular proprioception or efference signals for online control. The time course of this process, however, is not established, and it is unclear how far in advance gaze can move and still be used to parameterize subsequent movements. In this experiment, the authors considered the advantage of prescanning targets for both pointing and reaching movements. The authors manipulated the visual information and examined the extent to which prescanning of targets could compensate for a reduction in online visual feedback. In comparison with a conventional reaching/pointing condition, the error in pointing was reduced, the eye-hand lead decreased, and both the hand-closure time and the size of the maximum grip aperture in reaching were modulated when prescanning was allowed. These results indicate that briefly prescanning multiple targets just prior to the movement allows the refinement of subsequent hand movements that yields an improvement in accuracy. This study therefore provides additional evidence that the coordinate information arising from efference or ocular-proprioceptive signals can, for a limited period, be buffered and later used to generate a sequence of movements.     

The use of prescanning in the parameterization of sequential pointing and reaching movements

The accuracy of reaching movements improves when active gaze can be used to fixate on targets. The advantage of free gaze has been attributed to the use of ocular proprioception or efference signals for online control. The time course of this process, however, is not established, and it is unclear how far in advance gaze can move and still be used to parameterize subsequent movements. In this experiment, the authors considered the advantage of prescanning targets for both pointing and reaching movements. The authors manipulated the visual information and examined the extent to which prescanning of targets could compensate for a reduction in online visual feedback. In comparison with a conventional reaching/pointing condition, the error in pointing was reduced, the eye-hand lead decreased, and both the hand-closure time and the size of the maximum grip aperture in reaching were modulated when prescanning was allowed. These results indicate that briefly prescanning multiple targets just prior to the movement allows the refinement of subsequent hand movements that yields an improvement in accuracy. This study therefore provides additional evidence that the coordinate information arising from efference or ocular-proprioceptive signals can, for a limited period, be buffered and later used to generate a sequence of movements.     

Experiencing the Cross-Sensory Error Signal During Movement Leads to Proprioceptive Recalibration

Abstract

Reaching to targets in a virtual reality environment with misaligned visual feedback of the hand results in changes in movements (visuomotor adaptation) and sense of felt hand position (proprioceptive recalibration). We asked if proprioceptive recalibration arises even when the misalignment between visual and proprioceptive estimates of hand position is only experienced during movement. Participants performed a “shooting task” through the targets with a cursor that was rotated 30° clockwise relative to hand motion. Results revealed that, following training on the shooting task, participants adapted their reaches to all targets by approximately 16° and recalibrated their sense of felt hand position by 8°. Thus, experiencing a sensory misalignment between visual and proprioceptive estimates of hand position during movement leads to proprioceptive recalibration.     

Reaching measures of monocular distance perception: forward versus side-to-side head movements and haptic feedback

We investigated whether forward or side-to-side head movements yielded more accurate and precise monocular egocentric distance information, as shown by performance in a reaching task. Observers wore a head-mounted camera and display to isolate the optic flow generated by their head movements and had to reach to align a stylus directly under a target surface. Performance in the two head movement conditions was also tested with normal monocular vision. We tested performance in the two head movement conditions when the observers were given haptic feedback and compared performance when haptic feedback was removed. Performance was both more accurate and more precise in the forward head movement condition than in the side-to-side head movement condition. Performance in the side-to-side condition also deteriorated more after the removal of haptic feedback than did performance in the forward head movement condition. In the normal monocular condition, performance was comparable for the two head movement conditions. The implications for enucleated patients are discussed.     

Consistent haptic feedback is required but it is not enough for natural reaching to virtual cylinders

In virtual reality it is easy to control the visual cues that tell us about an object's shape. However, it is much harder to provide realistic virtual haptic feedback when grasping virtual objects. In this study we examined the role of haptic feedback when grasping (virtual) cylinders with an elliptical circumference. In Experiment 1 we placed the same circular cylinder at the simulated location of virtual elliptical cylinders of varying shape, so that the haptic feedback did not change when the visually specified shape changed. We found that the scaling of maximum grip aperture with the diameter of the nearest principal axis (.14+/-.04) was much weaker than when grasping real cylinders (.54+/-.04, Cuijpers, Brenner, & Smeets, 2006 Grasping reveals visual misjudgements of shape. Experimental Brain Research, 175, 32-44). For the scaling of grip orientation with the orientation of the cylinder we found large individual differences: the range is .07-.82 (average .42+/-.07) as compared to .55-.79 (average .67+/-.03) for grasping real cylinders. In Experiment 2 we provided consistent haptic feedback by placing real cylinders that matched the location, shape and orientation of the virtual cylinders. The scaling gains of both maximum grip aperture (.39+/-.04) and grip orientation (.56+/-.08) were substantially higher than in Experiment 1, but still lower than for grasps to real cylinders. The variability between participants for the scaling of grip orientation was also much reduced. These results showed that although haptic feedback must be consistent with visual information, it is not sufficient for natural prehension. We discuss the implications of these findings in terms of the integration of visual information with haptic feedback.     

Visual and haptic feedback contribute to tuning and online control during object manipulation

The authors employed a virtual environment to investigate how humans use haptic and visual feedback in a simple, rhythmic object-manipulation task. The authors hypothesized that feedback would help participants identify the appropriate resonant frequency and perform online control adjustments. The 1st test was whether sensory feedback is needed at all; the 2nd was whether the motor system combines visual and haptic feedback to improve performance. Task performance was quantified in terms of work performed on the virtual inertia, ability to identify the correct rhythm, and variability of movement. Strict feedforward control was found to be ineffective for this task, even when participants had previous knowledge of the rhythm. Participants (N = 11) performed far better when feedback was available (11 times more work, 2.2 times more precise frequency, 30% less variability; p < .05 for all 3 performance measures). Using sensory feedback, participants were able to rapidly identify 4 different spring-inertia systems without foreknowledge of the corresponding resonant frequencies. They performed over 20% more work with 24% less variability when provided with both visual and haptic feedback than they did with either feedback channel alone (p < .05), providing evidence that they integrated online sensory channels. Whereas feedforward control alone led to poor performance, feedback control led to fast tuning or calibration of control according to the resonant frequency of the object, and to better control of the rhythmic movement itself.     

Performance in haptic geometrical matching tasks depends on movement and position of the arms

Previous research on the properties of haptic space has shown systematic deviations from Euclidean parallelity in haptic parallelity tasks. The mainstream explanation for these deviations is that, in order to perform the task, participants generate a spatial representation with a frame of reference that integrates egocentric and allocentric components. Several studies have shown that the amount and type of deviations are affected by the configurations with regard to the arms and the rods to be matched. The present study reports 4 experiments that further address the effects of task configurations and body movements. Experiments 1 and 2 replicate and extend previous results concerning haptic matching task and acoustic pointing tasks. The third experiment includes acoustic cues aligned differentially to the reference and test bars. The fourth experiment concerns a geometrical matching task performed in the rear peripersonal space. Results show that haptic deviations from the Euclidean space are modulated by the available cues and by the body configurations. This indicates the need for further analysis on the role of body, arm and shoulder positions, and movement effects in haptic space perception.     

Haptic interfaces for virtual environments: perceived instability and force constancy in haptic sensing of virtual surfaces.

Haptic interfaces are becoming more commonplace in virtual environment and teleoperation systems. There is a growing need to not only continue to improve hardware platforms and rendering algorithms, but evaluate human performance with haptic interfaces. This review summarizes two recent studies inspired by perception problems in using haptic interfaces to interact with virtual environments. The first study evaluated perceived quality of virtual haptic textures and discovered several types of perceived instability and their sources. We found that the buzzing type of perceived instability was most likely due to the mechanical resonance of the haptic interface hardware, and the aliveness type of perceived instability due to our inability to sense the slight movements of our hands in free space. The second study focused on the motor strategy employed during interaction with a virtual surface via a force-feedback haptic interface. We found that users tended to maintain a constant penetration force into a virtual surface when interacting with the surface. This can result in a reversal in perceived relative surface heights if the taller surface is rendered with a lower stiffness, thereby resulting in an erroneous perception of the virtual environment being rendered. For both studies, possible solutions to improving human perception of virtual and remote objects via hardware and/or software are discussed.     

Adaptation to prisms: Do proprioceptive changes mediate adapted behaviour with ballistic arm movements?

In Experiment I subjects pointed repeatedly at a target viewed through laterally displacing prisms and received terminal visual feedback. In one task the pointing movements were slow (proprioceptively controlled), and in the other they were fast (pre-programmed). In both tasks adaptation proceeded at the same rate and to the same level of performance. Following fast pointing with prisms a large amount of arm-body adaptation was found with slow and fast test movements, while following slow pointing with prisms a large amount of arm-body adaptation was found with slow test movements, but only a small amount with fast test movements. The result suggests that adapted behaviour with preprogrammed movements is not mediated by a proprioceptive change. In Experiment II pointing movements were passive. No arm-body adaptation was found with fast test movements, and, contrary to expectation, only a small amount with slow test movements.     

Trunk Compensation During Bimanual Reaching at Different Heights by Healthy and Hemiparetic Adults

The authors explored how trunk compensation and hand symmetry in stroke survivors and healthy controls were affected by the distance and height of virtual targets during a bimanual reaching task. Participants were asked to reach to 4 different virtual targets set at: 90% of their arm length at shoulder, xiphoid process, and knee height, and 50% of their arm length at xiphoid process height. For the stroke group, for all targets, the hands’ movements were more asymmetrical than those of the healthy group, with more asymmetry observed in the direction of gravity, and trunk forward displacement values were larger and more variable. The knee targets had the largest trunk displacement values; index of curvature and trunk displacement were strongly correlated with participants’ impairment scores. A strong correlation was found between the hands’ asymmetry in the anterior or posterior direction for the shoulder targets, and the impairment scores. The results suggest that target height influences the degree of trunk compensation and hand symmetry during bimanual reaching by hemiparetic participants.     

Role of proprioceptive information in movement programming and control in 5 to 11-year old children

The role of proprioceptive inputs in the control of goal-directed movements was examined, by means of the tendon vibration technique, in 5 to 11-year old children performing a serial pointing task. Children pointed, with movements of various amplitudes and at various positions, by alternating wrist flexions and extensions. Tendon vibration was applied to both agonist and antagonist muscles to perturb relevant muscular proprioceptive inputs during the static or dynamic phase of the task, i.e., during stops on targets or during movement execution. Constant and variable amplitude errors as well as constant position error were evaluated. Vibratory perturbation applied during movement execution resulted in a similar reduction in movement amplitude, yielding an increased constant error in all age groups and a systematic position error in the direction of the movement starting point. Perturbing proprioception during static phases preceding movement resulted in an age-related increase in the variable amplitude error, which was maximal in 5-year old children performing extension movements. The results were interpreted in terms of the use of proprioceptive information in the feedforward and feedback based components of movement control in children. In particular, the results indicated (1) developmental changes in the relative weighting of each component, (2) an increased capacity to move from one strategy to the other, depending on the availability of information, and (3) developmental changes from an alternated to an integrated control of amplitude and position in serial pointing.     

Auditory cueing facilitates temporospatial accuracy of sequential movements

Effectively executing goal-directed behaviours requires both temporal and spatial accuracy. Previous work has shown that providing auditory cues enhances the timing of upper-limb movements. Interestingly, alternate work has shown beneficial effects of multisensory cueing (i.e., combined audiovisual) on temporospatial motor control. As a result, it is not clear whether adding visual to auditory cues can enhance the temporospatial control of sequential upper-limb movements specifically. The present study utilized a sequential pointing task to investigate the effects of auditory, visual, and audiovisual cueing on temporospatial errors. Eighteen participants performed pointing movements to five targets representing short, intermediate, and large movement amplitudes. Five isochronous auditory, visual, or audiovisual priming cues were provided to specify an equal movement duration for all amplitudes prior to movement onset. Movement time errors were then computed as the difference between actual and predicted movement times specified by the sensory cues, yielding delta movement time errors (ΔMTE). It was hypothesized that auditory-based (i.e., auditory and audiovisual) cueing would yield lower movement time errors compared to visual cueing. The results showed that providing auditory relative to visual priming cues alone reduced ΔMTE particularly for intermediate amplitude movements. The results further highlighted the beneficial impact of unimodal auditory cueing for improving visuomotor control in the absence of significant effects for the multisensory audiovisual condition.     

Higher visuo-Attentional Demands of Multiple Object Tracking (MOT) Lead to A Lower Precision in Pointing Movements

Multiple object tracking (MOT) requires visually attending to dynamically moving targets and distractors. This cognitive ability is based on perceptual-attentional processes that are also involved in goal-directed movements. This study aimed to test the hypothesis that MOT affects the motor performance of aiming movements. Therefore, the participants performed pointing movements using their fingers or a computer mouse that controlled the movements of a cursor directed at the targets in the MOT task. The precision of the pointing movements was measured, and it was predicted that a higher number of targets and distractors in the MOT task would result in a lower pointing precision. The results confirmed this hypothesis, indicating that MOT might influence the performance of motor actions. The potential factors underlying this influence are discussed.     

Calibration and Alignment are Separable: Evidence From Prism Adaptation

In 2 prism adaptation experiments, the authors investigated the effects of limb starting position visibility (visible or not visible) and visual feedback availability (early or late in target pointing movements). Thirty-two students participated in Experiment 1 and 24 students participated in Experiment 2. Independent of visual feedback availability, constant error was larger and variable error was smaller for target pointing when limb starting position was visible during prism exposure. Independent of limb starting position visibility, aftereffects of prism exposure were determined by visual feedback availability. Those results support the hypothesis that calibration is determined by limb starting position visibility, whereas alignment is determined separately by visual feedback availability.     

Visual attention amplifies response priming of pointing movements to color targets

We studied the influence of spatial visual attention on the time course of primed pointing movements. We measured pointing responses to color targets preceded by color stimuli priming either the same response as the target or the opposite response. The effects of visual attention at the prime and target locations were studied by varying both the cue-prime and prime-target intervals when presenting either exogenous attentional cues or, in a separate experiment, endogenous cues whose processing was a precondition for performing the task. Pointing trajectories revealed large priming effects in which pointing responses were first controlled by prime signals and then captured in midflight by target signals. Priming effects were strongly amplified when the relevant prime locations were visually attended at optimal cue-prime SOAs, with attention modulating the entire time course of the primed pointing movements. We propose that visual attention amplifies the earliest waves of visuomotor feedforward information, elicited in turn by primes and by targets.