Coordinated control of eye and hand movements in dynamic reaching

In the present study, we integrated two recent, at first sight contradictory findings regarding the question whether saccadic eye movements can be generated to a newly presented target during an ongoing hand movement. Saccades were measured during so-called adaptive and sustained pointing conditions. In the adapted pointing condition, subjects had to direct both their gaze and arm movements to a displaced target location. The results showed that the eyes could fixate the new target during pointing. In addition, a temporal coupling of these corrective saccades was found with changes in arm movement trajectories when reaching to the new target. In the sustained pointing condition, however, the same subjects had to point to the initial target, while trying to deviate their gaze to a new target that appeared during pointing. It was found that the eyes could not fixate the new target before the hand reached the initial target location. Together, the results indicate that ocular gaze is always forced to follow the target intended by a manual arm movement. A neural mechanism is proposed that couples ocular gaze to the target of an arm movement. Specifically, the mechanism includes a reach neuron layer besides the well-known saccadic layer in the primate superior colliculus. Such a tight, sub-cortical coupling of ocular gaze to the target of a reaching movement can explain the contrasting behavior of the eyes in dependency of whether the eye and hand share the same target position or attempt to move to different locations.     

Visual stability with goal-directed eye and arm movements toward a target displaced during saccadic suppression

This experiment tested whether the perceived stability of the environment is altered when there is a combination of eye and visually open-loop hand movements toward a target displaced during the eye movements, i.e., during saccadic suppression. Visual-target eccentricity randomly decreased or increased during eye movements and subjects reported whether they perceived a target displacement or not, and if so, the direction of the displacement. Three experimental conditions, involving different combinations of eye and arm movements, were tested: (a) eye movements only; (b) simultaneous eye and rapid arm movements toward the target; and (c) simultaneous eye and arm movements with a restraint blocking the arm as soon as the hand left the starting position. The perceptual threshold of target displacements resulting in an increased target eccentricity was greater when subjects combined eye and arm movements toward the target object, specially for the no-restraint condition. Subjects corrected most of their arm trajectory toward the displaced target despite the short movement times (average MT = 189 ms). After the movements, the null error feedback of the hand's final position presumably overlapped the retino-oculomotor signal error and could be responsible for the deficient perception of target displacements. Thus, subjects interpreted the terminal hand positions as being within the range of the endpoint variability associated with the production of rapid arm movements rather than as a change of the environment. These results suggest that a natural strategy adopted for processing spatial information, especially in a competing situation, could favour a constancy tendency, avoiding systematic perception of a change of environment for any noise or variability at the central or peripheral levels.     

Top-down influences make saccades deviate away: the case of endogenous cues

We tested a recent hypothesis suggesting that the eye deviates away from a location when top-down preparation can influence target selection. Participants had to make an eye movement to a peripheral target. Before the upcoming target, a central cue indicated the likely target location. Results show that when the target was presented at a location different from that indicated by the cue, eye movements to the target deviated away from the cued location. Because central cues are under top-down control, the present results are in line with a determining role of top-down preparation on saccade direction. These results contrast with the findings reported in a similar paradigm executed with hand movements, in which the movements were mostly initiated in the direction of the cued location. Therefore, we conclude that inhibitory effects typically observed when executing eye movements may not be observed when executing hand movements in similar conditions.     

Interactions between goal-directed eye and arm movements: arguments for an interdependent motor control

The accuracy of movements of the arm directed toward a point in space was investigated in healthy human subjects. To study the influence of the eye movement itself, on the guidance of the arm in the absence of any visual context, subjects performed the goal-directed arm movements without visual feedback about the arm displacement and the target position. The subjects were asked either to keep their eyes centered or oriented toward a previously flashed target. The analysis of the distribution of the errors in arm final position in the two conditions suggests that the eye movement influences the final position adopted by the arm. It is postulated that an interaction exists between the eye and arm systems during the motor program elaboration phase.     

Oculocentric coding of inhibited eye movements to recently attended locations

Results are reported for experiments that examined eye movements directed toward recently cued objects. In 1 experiment participants were slower to initiate saccades toward the earlier location of an object that had been cued, even though the cued object had subsequently moved away from that location. Other experiments involved exploring the reference frame within which the inhibited eye movements are encoded. These experiments revealed that the eye movement that is inhibited is encoded in an oculocentric-rather than an environmental-reference frame. However, simple detection as indexed by manual keypress responses is encoded in an environmental reference frame. The results have implications for inhibition of return, for the link between eye movements and attention, and for the nature of the spatial reference frames in which both covert and overt movements of attention are encoded.     

Grasping of extrafoveal targets: A robotic model

We present a computational model of grasping of non-fixated (extrafoveal) target objects which is implemented on a robot setup, consisting of a robot arm with cameras and gripper. This model is based on the premotor theory of attention (Rizzolatti et al., 1994) which states that spatial attention is a consequence of the preparation of goal-directed, spatially coded movements (especially saccadic eye movements). In our model, we add the hypothesis that saccade planning is accompanied by the prediction of the retinal images after the saccade. The foveal region of these predicted images can be used to determine the orientation and shape of objects at the target location of the attention shift. This information is necessary for precise grasping. Our model consists of a saccade controller for target fixation, a visual forward model for the prediction of retinal images, and an arm controller which generates arm postures for grasping. We compare the precision of the robotic model in different task conditions, among them grasping (1) towards fixated target objects using the actual retinal images, (2) towards non-fixated target objects using visual prediction, and (3) towards non-fixated target objects without visual prediction. The first and second setting result in good grasping performance, while the third setting causes considerable errors of the gripper orientation, demonstrating that visual prediction might be an important component of eye–hand coordination. Finally, based on the present study we argue that the use of robots is a valuable research methodology within psychology.     

Eye-hand coordination: spatial localization after saccadic and pursuit eye movements

Subjects produced speeded and unspeeded hand movements to a target location after either saccadic or pursuit eye movements to the target. Hand movements began either aligned with the initial position of gaze or from some other location. Subjects generally underestimated the extent of the pursuit eye movements relative to estimates made after saccades. With speeded hand movements, however, the underestimation was reduced considerably if the hand movements began aligned with a location other than the initial position of gaze. The results reveal details of the mechanisms underlying eye-hand coordination and show that important differences exist in the information used for localization for slow and rapid limb movements.     

The role of visual attention in saccadic eye movements

The relationship between saccadic eye movements and covert orienting of visual spatial attention was investigated in two experiments. In the first experiment, subjects were required to make a saccade to a specified location while also detecting a visual target presented just prior to the eye movement. Detection accuracy was highest when the location of the target coincided with the location of the saccade, suggesting that subjects use spatial attention in the programming and/or execution of saccadic eye movements. In the second experiment, subjects were explicitly directed to attend to a particular location and to make a saccade to the same location or to a different one. Superior target detection occurred at the saccade location regardless of attention instructions. This finding shows that subjects cannot move their eyes to one location and attend to a different one. The results of these experiments suggest that visuospatial attention is an important mechanism in generating voluntary saccadic eye movements.     

Independent allocation of attention to eye and hand targets in coordinated eye-hand movements

When reaching for objects, people frequently look where they reach. This raises the question of whether the targets for the eye and hand in concurrent eye and hand movements are selected by a unitary attentional system or by independent mechanisms. We used the deployment of visual attention as an index of the selection of movement targets and asked observers to reach and look to either the same location or separate locations. Results show that during the preparation of coordinated movements, attention is allocated in parallel to the targets of a saccade and a reaching movement. Attentional allocations for the two movements interact synergistically when both are directed to a common goal. Delaying the eye movement delays the attentional shift to the saccade target while leaving attentional deployment to the reach target unaffected. Our findings demonstrate that attentional resources are allocated independently to the targets of eye and hand movements and suggest that the goals for these effectors are selected by separate attentional mechanisms.     

Visual tracking of active and passive movements of the hand

Two experiments were performed to evaluate the influence of movement frequency and predictability on visual tracking of the actively and the passively moved hand. Four measures of tracking precision were employed: (a) saccades/cycle, (b) percent of pursuit movement, (c) eye amplitude/arm amplitude, (d) asynchrony of eye and hand at reversal. Active and passive limb movements were tracked with nearly identical accuracy and were always vastly superior to tracking an external visual target undergoing comparable motion. Proprioceptive information about target position appears to provide velocity and position information about target location. Its presence permits the development of central eye-movement programmes that move the eyes in patterns that approximate but do not exactly match, temporally or spatially, the motion of the hand.     

Influence of target uncertainty on reaching movements while standing in stroke

Stroke individuals frequently have balance problems and impaired arm movements that affect their daily activities. We investigated the influence of target uncertainty and the side of the brain lesion on the performance of arm movements and postural adjustments during reaching in a standing position by stroke individuals. Participants stood on force plates and reached a target displayed on the center of a monitor screen under conditions differentiated by the prior knowledge of the target location at the beginning of the movement. Individuals who had a stroke in the right side of the brain performed the tasks with the ipsilesional, right upper limb while the individuals with a left stroke performed with the ipsilesional, left upper limb. Healthy individuals performed with right and left limbs, which data were later averaged for statistical analysis. Kinematic analysis of the arm and lower limb joints and displacements of the center of pressure of each lower limb were compared between target conditions and groups. Stroke individuals showed larger center of pressure displacements of the contralesional compared to the ipsilesional limb while these displacements were symmetrical between lower limbs for the healthy individuals, regardless of the target condition. The target uncertainty affected both the characteristics of the arm movements and postural adjustments before movement onset. Right stroke individuals used more ankle joint movements under the uncertain compared to the certain condition. The uncertainty in target location affects the arm reaching in upright standing, but the effects depend on the side of the brain lesion.     

Inhibition of return is composed of attentional and oculomotor processes

Response time can be delayed if a target stimulus appears at a location or object that was previously cued. This inhibition of return (IOR) phenomenon has been attributed to a delay in activating attentional or motor processes to a previously cued stimulus. Two experiments required subjects to localize or identify a target stimulus. In Experiment 1, the subjects’ eyes were not monitored. In Experiment 2, the subjects’ eyes were monitored, and the subjects were instructed to either execute or withhold an eye movement to a target stimulus. The results indicated that IOR was always present for location and identification responses, supporting an attentional account of IOR. However, IOR was larger when eye movements were executed, indicating that a motor component can contribute to IOR. Finally, when eye movements were withheld, IOR was larger when a target was presented alone than when it was presented with a distractor, suggesting that IOR is larger for exogenous than for endogenous covert orienting. Together, the data indicate that IOR is composed of both an oculomotor component and an attentional component.     

Inhibition of return is composed of attentional and oculomotor processes.

Response time can be delayed if a target stimulus appears at a location or object that was previously cued. This inhibition of return (IOR) phenomenon has been attributed to a delay in activating attentional or motor processes to a previously cued stimulus. Two experiments required subjects to localize or identify a target stimulus. In Experiment 1, the subjects' eyes were not monitored. In Experiment 2, the subjects' eyes were monitored, and the subjects were instructed to either execute or withhold an eye movement to a target stimulus. The results indicated that IOR was always present for location and identification responses, supporting an attentional account of IOR. However, IOR was larger when eye movements were executed, indicating that a motor component can contribute to IOR. Finally, when eye movements were withheld, IOR was larger when a target was presented alone than when it was presented with a distractor, suggesting that IOR is larger for exogenous than for endogenous covert orienting. Together, the data indicate that IOR is composed of both an oculomotor component and an attentional component.     

Amplitude Scaling Compensates for Serial Delays in Correcting Eye and Arm Movements

Spatial and metrical parameters of the eye and arm movements made by human subjects (N = 7) in response to visual targets that were stepped unexpectedly either once (single step) or twice (double step) were studied. For the double-step, the displacement of a visual target was decreased or increased in amplitude at intervals before and during a movement. Provided the second target step occurred more than 100 ms before the onset of movement, the amplitude of the subjects' first response was altered in the direction of the new target location. But this amplitude scaling was not always sufficient to reach the new target location, and a second corrective response was required. The latency in producing this second response was greatly increased above reaction time latencies of movements to single-step targets, especially when the target change occurred 100 ms or more before movement onset. These findings suggest that even though serial processing limitations delay the production of a second corrective response, continuous parallel processing of visual information enables the amplitude of the first response to be altered with minimal delay. This enables some degree of real-time continuous control by the visuomotor control system.     

Exploring attention-based explanations for some violations of Hick’s law for aimed movements

Choice reaction time generally increases linearly with the logarithm of the number of potential stimulus-response alternatives, a regularity known as Hick's law. Two apparent violations of this generalization, which have been reported for aimed eye movements (Kveraga, Boucher, & Hughes, Experimental Brain Research, 146, 307-314, 2002), and arm movements (Wright, Marino, Belovsky, & Chubb, Experimental Brain Research, 179, 475-496, 2007), occurred when the indicator stimulus was an abrupt change at the location that was the target of the to-be-made movement. We report two experiments that examined and rejected the hypothesis that these abrupt-onset indicator stimuli triggered a shift in exogenous attention and that this led to unusually small uncertainty effects. Each experiment compared this indicator stimulus with a single alternative: Experiment 1 tested an indicator stimulus at all locations other than the target; Experiment 2 tested a central pointer to the target. Neither alternative led to an uncertainty effect for pointing responses that was of the size typically observed for other responses using the same stimuli.     

Eye movements and illusions of alignment

Extraction of the location of a target in the visual periphery is a fundamental visual process which may be manifested both in conscious judgments of location, such as decisions about alignment, and in the processing required to execute an eye movement to a peripheral target. In both cases, location judgement is affected by the neighbouring visual configuration. An experiment is reported in which the amplitude of saccadic eye movements has been measured to study the effect of a configuration consisting of a single 'induction line'. The results demonstrate systematic effects of this line on the size of the saccade which are closely similar to those previously found in an alignment task.     

Amplitude Scaling Compensates for Serial Delays in Correcting Eye and Arm Movements

Spatial and metrical parameters of the eye and arm movements made by human subjects (N = 7) in response to visual targets that were stepped unexpectedly either once (single step) or twice (double step) were studied. For the double-step, the displacement of a visual target was decreased or increased in amplitude at intervals before and during a movement. Provided the second target step occurred more than 100 ms before the onset of movement, the amplitude of the subjects' first response was altered in the direction of the new target location. But this amplitude scaling was not always sufficient to reach the new target location, and a second corrective response was required. The latency in producing this second response was greatly increased above reaction time latencies of movements to single-step targets, especially when the target change occurred 100 ms or more before movement onset. These findings suggest that even though serial processing limitations delay the production of a second corrective response, continuous parallel processing of visual information enables the amplitude of the first response to be altered with minimal delay. This enables some degree of real-time continuous control by the visuomotor control system.     

Vector coding in slow goal-directed arm movements

It has been found that the estimate of relative target direction is consistently biased. Relative target direction refers to the direction in which a target is located relative to another location in space (e.g., a starting position in the case of goal-directed movements). In this study, we have tested two models that could underlie this biased estimate. The first proposed model is based on a distorted internal representation of locations (i.e., we perceive a target at the “wrong” location). We call this thedistorted location model. The second model is based on the idea that the derivation of target direction from spatial information about starting and target position is biased. We call this thebiased direction model. These two models lead to different predictions of the deviations that occur when the distance between the starting position and the target position is increased. Since we know from previous studies that the initial direction of slow arm movements reflects the target direction estimate, we tested the two models by analyzing the initial direction of slow arm movements. The results show that the biased direction model can account for the biases we find in the target direction estimate for various target distances, whereas the distorted location model cannot. In two additional experiments, we explored this model further. The results show that the biases depend only on the orientation of the line through starting position and target position relative to the plane through longitudinal head or body axis and starting position. We conclude that the initial part of (slow) goal-directed arm movements is planned on the basis of a (biased) target direction estimate and not on the basis of a wrong internal representation of target location. This supports the hypothesis that we code displacements of our limbs in space as a vector.     

Visuospatial perceptual sequence learning and eye movements

We examined perceptual sequence learning by means of an adapted serial reaction time task in which eye movements were unnecessary for performing the sequence learning task. Participants had to respond to the identity of a target letter pair ("OX" or "XO") appearing in one of four locations. On the other locations, similar distractor letter pairs ("QY" or "YQ") were shown. While target identity changed randomly, target location was structured according to a deterministic sequence. To render eye movements superfluous, (1) stimulus letter pairs appeared around a fixation cross with a visual angle of 0.63°, which means that they appeared within the foveal visual area and (2) the letter pairs were presented for only 100 ms, a period too short to allow proper eye movements. Reliable sequence knowledge was acquired under these conditions, as responses were both slower and less accurate when the trained sequence was replaced by an untrained sequence. These results support the notion that perceptual sequence learning can be based on shifts of attention without overt oculomotor movements.     

Simultaneous processing of visual information and planning of hand movements in a visuo-manual search task

When searching for a target with eye movements, saccades are planned and initiated while the visual information is still being processed. If hand movements are needed to perform a search task, can they too be planned while visual information from the current position is still being processed? To find out we studied a visual search task in which participants had to move their hand to shift a window through which they could see the items. The task was to find an O in a circle of Cs. The size of the window and the sizes of the gaps in the Cs were varied. Participants made fast, smooth arm movements between items and adjusted their movements, when on the items, to the window size. On many trials the window passed the target and returned, indicating that the next movement had been planned before identifying the item that was in view.