Physical imagery: kinematic versus dynamic models.

Physical imagery occurs when people imagine one object causing a change to a second object. To make inferences through physical imagery, people must represent information that coordinates the interactions among the imagined objects. The current research contrasts two proposals for how this coordinating information is realized in physical imagery. In the traditional kinematic formulation, imagery transformations are coordinated by geometric information in analog spatial representations. In the dynamic formulation, transformations may also be regulated by analog representations of force and resistance. Four experiments support the dynamic formulation. They show, for example, that without making changes to the spatial properties of a problem, dynamic perceptual information (e.g., torque) and beliefs about physical properties (e. g., viscosity) affect the inferences that people draw through imagery. The studies suggest that physical imagery is not so much an analog of visual perception as it is an analog of physical action. A simple model that represents force as a rate helps explain why inferences can emerge through imagined actions even though people may not know the answer explicitly. It also explains how and when perception, beliefs, and learning can influence physical imagery.     

Cognitive processes involved in smooth pursuit eye movements

Ocular pursuit movements allow moving objects to be tracked with a combination of smooth movements and saccades. The principal objective is to maintain smooth eye velocity close to object velocity, thus minimising retinal image motion and maintaining acuity. Saccadic movements serve to realign the image if it falls outside the fovea, the area of highest acuity. Pursuit movements are often portrayed as voluntary but their basis lies in processes that sense retinal motion and can induce eye movements without active participation. The factor distinguishing pursuit from such reflexive movements is the ability to select and track a single object when presented with multiple stimuli. The selective process requires attention, which appears to raise the gain for the selected object and/or suppress that associated with other stimuli, the resulting competition often reducing pursuit velocity. Although pursuit is essentially a feedback process, delays in motion processing create problems of stability and speed of response. This is countered by predictive processes, probably operating through internal efference copy (extra-retinal) mechanisms using short-term memory to store velocity and timing information from prior stimulation. In response to constant velocity motion, the initial response is visually driven, but extra-retinal mechanisms rapidly take over and sustain pursuit. The same extra-retinal mechanisms may also be responsible for generating anticipatory smooth pursuit movements when past experience creates expectancy of impending object motion. Similar, but more complex, processes appear to operate during periodic pursuit, where partial trajectory information is stored and released in anticipation of expected future motion, thus minimising phase errors associated with motion processing delays.     

Cognitive processes involved in smooth pursuit eye movements

Ocular pursuit movements allow moving objects to be tracked with a combination of smooth movements and saccades. The principal objective is to maintain smooth eye velocity close to object velocity, thus minimising retinal image motion and maintaining acuity. Saccadic movements serve to realign the image if it falls outside the fovea, the area of highest acuity. Pursuit movements are often portrayed as voluntary but their basis lies in processes that sense retinal motion and can induce eye movements without active participation. The factor distinguishing pursuit from such reflexive movements is the ability to select and track a single object when presented with multiple stimuli. The selective process requires attention, which appears to raise the gain for the selected object and/or suppress that associated with other stimuli, the resulting competition often reducing pursuit velocity. Although pursuit is essentially a feedback process, delays in motion processing create problems of stability and speed of response. This is countered by predictive processes, probably operating through internal efference copy (extra-retinal) mechanisms using short-term memory to store velocity and timing information from prior stimulation. In response to constant velocity motion, the initial response is visually driven, but extra-retinal mechanisms rapidly take over and sustain pursuit. The same extra-retinal mechanisms may also be responsible for generating anticipatory smooth pursuit movements when past experience creates expectancy of impending object motion. Similar, but more complex, processes appear to operate during periodic pursuit, where partial trajectory information is stored and released in anticipation of expected future motion, thus minimising phase errors associated with motion processing delays.     

Eye-hand coordination in goal-directed aiming.

In a number of studies, we have demonstrated that the spatial-temporal coupling of eye and hand movements is optimal for the pickup of visual information about the position of the hand and the target late in the hand's trajectory. Several experiments designed to examine temporal coupling have shown that the eyes arrive at the target area concurrently with the hand achieving peak acceleration. Between the time the hand reached peak velocity and the end of the movement, increased variability in the position of the shoulder and the elbow was accompanied by a decreased spatial variability in the hand. Presumably, this reduction in variability was due to the use of retinal and extra-retinal information about the relative positions of the eye, hand and target. However, the hand does not appear to be a slave to the eye. For example, we have been able to decouple eye movements and hand movements using Müller-Lyer configurations as targets. Predictable bias, found in primary and corrective saccadic eye movements, was not found for hand movements, if on-line visual information about the target was available during aiming. That is, the hand remained accurate even when the eye had a tendency to undershoot or overshoot the target position. However, biases of the hand were evident, at least in the initial portion of an aiming movement, when vision of the target was removed and vision of the hand remained. These findings accent the versatility of human motor control and have implications for current models of visual processing and limb control.     

Pursuit compensation during self-motion

The pattern of motion in the retinal image during self-motion contains information about the person's movement. Pursuit eye movements perturb the pattern of retinal-image motion, complicating the problem of self-motion perception. A question of considerable current interest is the relative importance of retinal and extra-retinal signals in compensating for these effects of pursuit on the retinal image. We addressed this question by examining the effect of prior motion stimuli on self-motion judgments during pursuit. Observers viewed 300 ms random-dot displays simulating forward self-motion during pursuit to the right or to the left; at the end of each display a probe appeared and observers judged whether they would pass left or right of it. The display was preceded by a 300 ms dot pattern that was either stationary or moved in the same direction as, or opposite to, the eye movement. This prior motion stimulus had a large effect on self-motion judgments when the simulated scene was a frontoparallel wall (experiment 1), but not when it was a three-dimensional (3-D) scene (experiment 2). Corresponding simulated-pursuit conditions controlled for purely retinal motion aftereffects, implying that the effect in experiment 1 is mediated by an interaction between retinal and extra-retinal signals. In experiment 3, we examined self-motion judgments with respect to a 3-D scene with mixtures of real and simulated pursuit. When real and simulated pursuits were in opposite directions, performance was determined by the total amount of pursuit-related retinal motion, consistent with an extra-retinal 'trigger' signal that facilitates the action of a retinally based pursuit-compensation mechanism. However, results of experiment 1 without a prior motion stimulus imply that extra-retinal signals are more informative when retinal information is lacking. We conclude that the relative importance of retinal and extra-retinal signals for pursuit compensation varies with the informativeness of the retinal motion pattern, at least for short durations. Our results provide partial explanations for a number of findings in the literature on perception of self-motion and motion in the frontal plane.     

I see what you're saying: the integration of complex speech and scenes during language comprehension

The effect of language-driven eye movements in a visual scene with concurrent speech was examined using complex linguistic stimuli and complex scenes. The processing demands were manipulated using speech rate and the temporal distance between mentioned objects. This experiment differs from previous research by using complex photographic scenes, three-sentence utterances and mentioning four target objects. The main finding was that objects that are more slowly mentioned, more evenly placed and isolated in the speech stream are more likely to be fixated after having been mentioned and are fixated faster. Surprisingly, even objects mentioned in the most demanding conditions still show an effect of language-driven eye-movements. This supports research using concurrent speech and visual scenes, and shows that the behavior of matching visual and linguistic information is likely to generalize to language situations of high information load.     

The man who executed "imagined" movements: evidence for dissociable components of the body schema

We examined the nature of representations underlying motor imagery and execution in a patient (CW) with bilateral parietal lesions. When imagining hand movements, CW executed the imagined motor act but was unaware of the movements. These movements were significantly more accurate than volitional movements for the left but not right hand. CW also exhibited preserved motor imagery for the left but not right hand. Consistent with previous accounts, these findings suggest that motor imagery may normally involve the inhibition of movements. CW's unawareness of movements during motor imagery may reflect inattention or misattribution of the unexpected sensory feedback. Furthermore, in line with current models of motor control, motor imagery may depend on the integrity of a "forward model" derived from motor outflow information to generate a prediction of the consequences of a motor command. Such predictions appear to be preserved for imagery of left but not right hand movements in CW. Action may additionally depend on precise updating of effector position derived from the comparison of predicted and actual sensory information. We propose that CW's impaired volitional movements may be attributable to the degradation of such an updating mechanism.     

Imagined haptic exploration in judgments of object properties

In Experiment 1, each subject rated a single, named object for its roughness, hardness, temperature, weight, size, or shape. In Experiment 2, each subject compared one pair of objects along the same dimensions. In both studies, a substantial proportion of subjects who judged the first four dimensions imagined a hand making exploratory movements appropriate for the designated information. The proportion of hand-exploration images decreased substantially when judging size or shape, or when judgments could be made readily through general semantic knowledge. The results suggest that the incorporation of haptic exploration into visual imagery provides access to information about haptically accessible object properties.     

A Note on State Individual Differences in Accuracy of Response Imagery: Aftereffects and Reminiscence

In two sessions, separated by 7 days, subjects imagined themselves performing a tracking task under a massed practice schedule. After conditions of interpolated rest and no-rest, which were counterbalanced across sessions, subjects actually performed the tracking task. During imagery practice, subjects verbally reported the temporal component of the task. The temporal accuracy of verbal reports varied widely across subjects, but not within subjects. Furthermore, a performance gain was demonstrated as a function of interpolated rest versus no-rest (reminiscence effect). Finally, the accuracy of verbal reports predicted imagery aftereffects, but not reminiscence effects.     

Imagined movements that leak out

In a case study that fundamentally alters our understanding of motor imagery, Schwoebel et al. report a patient who unintentionally carries out imagined movements. Furthermore, his 'imagery' movements are more accurate than his intended movements, which suggests that the inhibitory signal that normally prevents us from acting out our motor imagery can be selectively blocked. Removing this inhibition allows us to observe motor imagery 'in action', and reveals that motor imagery and motor planning for execution are not identical.     

Dynamic locomotor imagery in athletes with severe visual impairments

Motor imagery is a mental process not accompanied by movement and widely studied in healthy subjects, related to hand movements in terms of timing. This study compared static and dynamic motor imagery analyzing temporal and spatial features in different locomotor conditions in three different groups of subjects: high-skilled athletes with visual impairments, a group of sighted unprofessional athletes and a control group of sighted subjects. We found that dynamic motor imagery resulted in timely closer to real performance than static motor imagery. The discrepancies between dynamic motor imagery and real condition, in fact, resulted limited to uncommon locomotion, such as lateral walking. Motor imagery resulted closer to real performance in terms of timing than in terms of step length, with the exception of athletes with visual impairments that, differently from the other groups, did not show any significant differences between the numbers of imagined and performed steps. It opens a new question about the relationship between temporal and spatial imagination of locomotion.     

The neural basis of smooth pursuit eye movements in the rhesus monkey brain

Smooth pursuit eye movements are performed in order to prevent retinal image blur of a moving object. Rhesus monkeys are able to perform smooth pursuit eye movements quite similar as humans, even if the pursuit target does not consist in a simple moving dot. Therefore, the study of the neuronal responses as well as the consequences of micro-stimulation and lesions in trained monkeys performing smooth pursuit is a powerful approach to understand the human pursuit system. The processing of visual motion is achieved in the primary visual cortex and the middle temporal area. Further processing including the combination of retinal image motion signals with extra-retinal signals such as the ongoing eye and head movement occurs in subsequent cortical areas as the medial superior temporal area, the ventral intraparietal area and the frontal and supplementary eye field. The frontal eye field especially contributes anticipatory signals which have a substantial influence on the execution of smooth pursuit. All these cortical areas send information to the pontine nuclei, which in turn provide the input to the cerebellum. The cerebellum contains two pursuit representations: in the paraflocculus/flocculus region and in the posterior vermis. While the first representation is most likely involved in the coordination of pursuit and the vestibular-ocular reflex, the latter is involved in the precise adjustments of the eye movements such as adaptation of pursuit initiation. The output of the cerebellum is directed to the moto-neurons of the extra-ocular muscles in the brainstem.     

When music tempo affects the temporal congruence between physical practice and motor imagery

When people listen to music, they hear beat and a metrical structure in the rhythm; these perceived patterns enable coordination with the music. A clear correspondence between the tempo of actual movement (e.g., walking) and that of music has been demonstrated, but whether similar coordination occurs during motor imagery is unknown. Twenty participants walked naturally for 8 m, either physically or mentally, while listening to slow and fast music, or not listening to anything at all (control condition). Executed and imagined walking times were recorded to assess the temporal congruence between physical practice (PP) and motor imagery (MI). Results showed a difference when comparing slow and fast time conditions, but each of these durations did not differ from soundless condition times, hence showing that body movement may not necessarily change in order to synchronize with music. However, the main finding revealed that the ability to achieve temporal congruence between PP and MI times was altered when listening to either slow or fast music. These data suggest that when physical movement is modulated with respect to the musical tempo, the MI efficacy of the corresponding movement may be affected by the rhythm of the music. Practical applications in sport are discussed as athletes frequently listen to music before competing while they mentally practice their movements to be performed.     

Did you see that? Dissociating advanced visual information and ball flight constrains perception and action processes during one-handed catching

The integration of separate, yet complimentary, cortical pathways appears to play a role in visual perception and action when intercepting objects. The ventral system is responsible for object recognition and identification, while the dorsal system facilitates continuous regulation of action. This dual-system model implies that empirically manipulating different visual information sources during performance of an interceptive action might lead to the emergence of distinct gaze and movement pattern profiles. To test this idea, we recorded hand kinematics and eye movements of participants as they attempted to catch balls projected from a novel apparatus that synchronised or de-synchronised accompanying video images of a throwing action and ball trajectory. Results revealed that ball catching performance was less successful when patterns of hand movements and gaze behaviours were constrained by the absence of advanced perceptual information from the thrower's actions. Under these task constraints, participants began tracking the ball later, followed less of its trajectory, and adapted their actions by initiating movements later and moving the hand faster. There were no performance differences when the throwing action image and ball speed were synchronised or de-synchronised since hand movements were closely linked to information from ball trajectory. Results are interpreted relative to the two-visual system hypothesis, demonstrating that accurate interception requires integration of advanced visual information from kinematics of the throwing action and from ball flight trajectory.     

Effects of action on children’s and adults’ mental imagery

The aim of this study was to investigate whether and which aspects of a concurrent motor activity can facilitate children’s and adults’ performance in a dynamic imagery task. Children (5-, 7-, and 9-year-olds) and adults were asked to tilt empty glasses, filled with varied amounts of imaginary water, so that the imagined water would reach the rim. Results showed that in a manual tilting task where glasses could be tilted actively with visual feedback, even 5-year-olds performed well. However, in a blind tilting task and in a static judgment task, all age groups showed markedly lower performance. This implies that visual movement information facilitates imagery. In a task where the tilting movement was visible but regulated by means of an on-and-off remote control, a clear age trend was found, indicating that active motor control and motor feedback are particularly important in imagery performance of younger children.     

Passive tactile feedback facilitates mental rotation of handheld objects

Mental rotation of objects improves when passive tactile information for the rotating object accompanies the imagined rotation (Wraga, Creem, & Proffitt, 2000). We examined this phenomenon further using a within-subjects paradigm involving handheld objects. In Experiment 1, participants imagined rotating an unseen object placed on their upturned palms. The participants were faster at mental rotation when the object was rotated on their palm than when the object remained stationary. Experiment 2 tested whether the performance advantage would endure when the participants received tactile information for only the start- and endpoints of the rotation event. This manipulation did not improve performance, relative to a stationary control. Experiment 3 revealed that ambiguous tactile information, continuous with the rotation event but independent of object shape, actually degraded performance, relative to a stationary control. In Experiment 4, we found that continuous tactile rotation discrepant from imagined object movement also hindered performance, as compared with continuous tactile information aligned with imagined object movement. The findings suggest a tight coupling between tactile information specifying continuous object rotation and the corresponding internal representation of the rotating object.     

Mental imagery and the third dimension

What sort of medium underlies imagery for three-dimensional scenes? In the present investigation, the time subjects took to scan between objects in a mental image was used to infer the sorts of geometric information that images preserve. Subjects studied an open box in which five objects were suspended, and learned to imagine this display with their eyes closed. In the first experiment, subjects scanned by tracking an imaginary point moving in a straight line between the imagined objects. Scanning times increased linearly with increasing distance between objects in three dimensions. Therefore metric 3-D information must be preserved in images, and images cannot simply be 2-D "snapshots." In a second experiment, subjects scanned across the image by "sighting" objects through an imaginary rifle sight. Here scanning times were found to increase linearly with the two-dimensional separations between objects as they appeared from the original viewing angle. Therefore metric 2-D distance information in the original perspective view must be preserved in images, and images cannot simply be 3-D "scale-models" that are assessed from any and all directions at once. In a third experiment, subjects mentally rotated the display 90 degrees and scanned between objects as they appeared in this new perspective view by tracking an imaginary rifle signt, as before. Scanning times increased linearly with the two-dimensional separations between objects as they would appear from the new relative viewing perspective. Therefore images can display metric 2-D distance information in a perspective view never actually experiences, so mental images cannot simply be "snapshot plus scale model" pairs. These results can be explained by a model in which the three-dimensional structure of objects is encoded in long-term memory in 3-D object-centered coordinate systems. When these objects are imagined, this information is then mapped onto a single 2-D "surface display" in which the perspective properties specific to a given viewing angle can be depicted. In a set of perceptual control experiments, subjects scanned a visible display by (a) simply moving their eyes from one object to another, (b) sweeping an imaginary rifle sight over the display, or (c) tracking an imaginary point moving from one object to another. Eye-movement times varied linearly with 2-D interobject distance, as did time to scan with an imaginary rifle sight; time to tract a point varied independently with the 3-D and 2-D interobject distances. These results are compared with the analogous image scanning results to argue that imagery and perception share some representational structures but that mental image scanning is a process distinct from eye movements or eye-movement commands.     

Response Imagery: Aftereffects and Reminiscence

In two experiments, subjects imagined themselves performing a tracking task under a massed practice schedule. After interpolated rest or no rest, subjects actually performed the criterion task. Some subjects' imagery was augmented with sounds that matched the temporal characteristics of the criterion task. These subjects produced greater aftereffects than subjects who imaged without augmentation or subjects provided with imagery augmentation matching a variation of the criterion. Reminiscence (performance gain attributed to interpolated rest) was demonstrated with imagery, except when the accompanying augmentation was faster than the criterion.