Asymmetries in visuomotor recalibration of time perception: Does causal binding distort the window of integration?

The recalibration of perceived visuomotor simultaneity to vision-lead and movement-lead temporal discrepancies is marked by an underlying causal asymmetry, if the movement (button press) is voluntary and self-initiated; a visual stimulus lagging the button press may be interpreted as causally linked sensory feedback (intentional or causal binding), a leading visual stimulus not. Here, we test whether this underlying causal asymmetry leads to directional asymmetries in the temporal recalibration of visuomotor time perception, using an interval estimation paradigm. Participants were trained to the presence of one of three temporal discrepancies between a motor action (button press) and a visual stimulus (flashed disk): 100 ms vision-lead, simultaneity, and 100 ms movement-lead. By adjusting a point on a visual scale, participants then estimated the interval between the visual stimulus and the button press over a range of discrepancies. Comparing the results across conditions, we found that temporal recalibration appears to be implemented nearly exclusively on the movement-lead side of the range of discrepancies by a uni-lateral lengthening or shortening of the window of temporal integration. Interestingly, this marked asymmetry does not lead to a significantly asymmetrical recalibration of the point of subjective simultaneity or to significant differences in discriminability. This seeming contradiction (symmetrical recalibration of subjective simultaneity and asymmetrical recalibration of interval estimation) poses a challenge to common models of temporal order perception that assume an underlying time measurement process with Gaussian noise. Using a two-criterion model of the window of temporal integration, we illustrate that a compressive bias around perceived simultaneity (temporal integration) even prior to perceptual decisions about temporal order would be very hard to detect given the sensitivity of the psychophysical procedures commonly used.     

Auditory‐motor adaptation is reduced in adults who stutter but not in children who stutter

Previous studies have shown that adults who stutter produce smaller corrective motor responses to compensate for unexpected auditory perturbations in comparison to adults who do not stutter, suggesting that stuttering may be associated with deficits in integration of auditory feedback for online speech monitoring. In this study, we examined whether stuttering is also associated with deficiencies in integrating and using discrepancies between expected and received auditory feedback to adaptively update motor programs for accurate speech production. Using a sensorimotor adaptation paradigm, we measured adaptive speech responses to auditory formant frequency perturbations in adults and children who stutter and their matched nonstuttering controls. We found that the magnitude of the speech adaptive response for children who stutter did not differ from that of fluent children. However, the adaptation magnitude of adults who stutter in response to auditory perturbation was significantly smaller than the adaptation magnitude of adults who do not stutter. Together these results indicate that stuttering is associated with deficits in integrating discrepancies between predicted and received auditory feedback to calibrate the speech production system in adults but not children. This auditory‐motor integration deficit thus appears to be a compensatory effect that develops over years of stuttering.     

Action,prediction, and temporal awareness

The brain needs to track changes in the relation between action and effect. In two experiments, participants made voluntary keypress actions. In an adaptation phase, these actions were followed after a fixed interval by a tone. During a subsequent test phase, the duration of the interval was unexpectedly changed. We used time perception as an implicit marker of the experience of participants' control over the effect, and confirmed a temporal binding between actions and effects. On test trials, participants perceived tones to occur as shifted towards their time of occurrence in the preceding adaptation phase. Therefore, the perceived time of a tone was partly based on learning of an internal prediction, rather than on the time of actual sensory input. This predictive model is rapidly updated over a few trials (Experiment 1), and requires attention to the tones (Experiment 2). The brain learns action–effect relations. This predictive learning influences the perception of effects, and underlies some temporal illusions associated with action.     

The role of auditory context in action-effect-related motor adaptation

Previous research indicates that adding auditory effects to a (silent) action can lead to substantial efficiency gains in the performance of the action, while compromising the connection between the motor and the auditory event (e.g., by removing, or by delaying the auditory effects), leads the agent to compensate for the loss of auditory feedback by executing actions in a way which increases the probability of success or enhances feedback in other modalities, thus departing from the optimal action performance. The current study explored how this motor adaptation was affected when the quality of auditory feedback was reduced by contextual factors, while keeping the physical link between the action and auditory effect intact. In two experiments, participants elicited pure tones by pinching a force sensitive resistor (FSR). In some of the conditions action-effect contingency was reduced by intermixing externally initiated tones with the self-induced ones. Pinch-force measurements indicated that action optimization was affected by contextual factors. The influence of auditory context was the most pronounced when the discrimination of self-induced and external tones was made difficult by the similarity and temporal proximity of the self-induced and external tones. In these conditions, tone eliciting actions were more forceful in comparison to conditions in which no external tones were presented, and in comparison to conditions in which the external tones were easily distinguishable from self-induced ones. This suggests that contextual factors can induce similar motor adjustments as manipulating the physical connection between the action and its sensory consequences.     

Action enhances auditory but not visual temporal sensitivity

People naturally dance to music, and research has shown that rhythmic auditory stimuli facilitate production of precisely timed body movements. If motor mechanisms are closely linked to auditory temporal processing, just as auditory temporal processing facilitates movement production, producing action might reciprocally enhance auditory temporal sensitivity. We tested this novel hypothesis with a standard temporal-bisection paradigm, in which the slope of the temporal-bisection function provides a measure of temporal sensitivity. The bisection slope for auditory time perception was steeper when participants initiated each auditory stimulus sequence via a keypress than when they passively heard each sequence, demonstrating that initiating action enhances auditory temporal sensitivity. This enhancement is specific to the auditory modality, because voluntarily initiating each sequence did not enhance visual temporal sensitivity. A control experiment ruled out the possibility that tactile sensation associated with a keypress increased auditory temporal sensitivity. Taken together, these results demonstrate a unique reciprocal relationship between auditory time perception and motor mechanisms. As auditory perception facilitates precisely timed movements, generating action enhances auditory temporal sensitivity.     

Sensation of agency and perception of temporal order

After adaptation to a fixed temporal delay between actions and their sensory consequences, stimuli delivered during the delay are perceived to occur prior to actions. Temporal judgments are also influenced by the sensation of agency (experience of causing our own actions and their sensory consequences). Sensory consequences of voluntary actions are perceived to occur earlier in time than those of involuntary actions. However, it is unclear whether temporal order illusions influence the sensation of agency. Thus, we tested how the illusionary reversal of motor actions and sound events affect the sensation of agency. We observed an absence of the sensation of agency in the auditory modality in a condition in which sounds were falsely perceived as preceding motor acts relative to the perceived temporal order in the control condition. This finding suggests a strong association between the sensation of agency and the temporal order perception of actions and their consequences.     

Spatial grouping resolves ambiguity to drive temporal recalibration

Cross-modal temporal recalibration describes a shift in the point of subjective simultaneity (PSS) between 2 events following repeated exposure to asynchronous cross-modal inputs--the adaptors. Previous research suggested that audiovisual recalibration is insensitive to the spatial relationship between the adaptors. Here we show that audiovisual recalibration can be driven by cross-modal spatial grouping. Twelve participants adapted to alternating trains of lights and tones. Spatial position was manipulated, with alternating sequences of a light then a tone, or a tone then a light, presented on either side of fixation (e.g., left tone--left light--right tone--right light, etc.). As the events were evenly spaced in time, in the absence of spatial-based grouping it would be unclear if tones were leading or lagging lights. However, any grouping of spatially colocalized cross-modal events would result in an unambiguous sense of temporal order. We found that adapting to these stimuli caused the PSS between subsequent lights and tones to shift toward the temporal relationship implied by spatial-based grouping. These data therefore show that temporal recalibration is facilitated by spatial grouping.     

The effects of voluntary movements on auditory-haptic and haptic-haptic temporal order judgments

In two experiments we investigated the effects of voluntary movements on temporal haptic perception. Measures of sensitivity (JND) and temporal alignment (PSS) were obtained from temporal order judgments made on intermodal auditory-haptic (Experiment 1) or intramodal haptic (Experiment 2) stimulus pairs under three movement conditions. In the baseline, static condition, the arm of the participants remained stationary. In the passive condition, the arm was displaced by a servo-controlled motorized device. In the active condition, the participants moved voluntarily. The auditory stimulus was a short, 500Hz tone presented over headphones and the haptic stimulus was a brief suprathreshold force pulse applied to the tip of the index finger orthogonally to the finger movement. Active movement did not significantly affect discrimination sensitivity on the auditory-haptic stimulus pairs, whereas it significantly improved sensitivity in the case of the haptic stimulus pair, demonstrating a key role for motor command information in temporal sensitivity in the haptic system. Points of subjective simultaneity were by-and-large coincident with physical simultaneity, with one striking exception in the passive condition with the auditory-haptic stimulus pair. In the latter case, the haptic stimulus had to be presented 45ms before the auditory stimulus in order to obtain subjective simultaneity. A model is proposed to explain the discrimination performance.     

Duration of auditory sensory memory in parents of children with SLI: a mismatch negativity study

In a previous behavioral study, we showed that parents of children with SLI had a subclinical deficit in phonological short-term memory. Here, we tested the hypothesis that they also have a deficit in nonverbal auditory sensory memory. We measured auditory sensory memory using a paradigm involving an electrophysiological component called the mismatch negativity (MMN). The MMN is a measure of the brain's ability to detect a difference between a frequent standard stimulus (1000 Hz tone) and a rare deviant one (1200 Hz tone). Memory effects were assessed by varying the inter-stimulus interval (ISI) between the standard and deviant. We predicted that parents of children with SLI would have a smaller MMN than parents of typically developing children at a long ISI (3000 ms), but not at a short one (800 ms). This was broadly confirmed. However, individual differences in MMN amplitude did not correlate with measures of phonological short-term memory. Attenuation of MMN amplitude at the longer ISI thus did not provide unambiguous support for the hypothesis of a reduced auditory sensory memory in parents of affected children. We conclude by reviewing possible explanations for the observed group effects.     

Individual differences in vulnerability to subjective time distortion1

Time duration is perceived to be longer when accompanied by dynamic sensory stimulation than when accompanied by static stimulation. This distortion of time perception is thought to be due to the acceleration of an internal pacemaker that has been assumed to be the main component of temporal judgments. In order to investigate whether the function of the internal pacemaker is modality dependent or independent, we examined the correlation of visual flicker and auditory flutter effects on a temporal production task. While seeing a 10‐Hz visual flicker or hearing a 10‐Hz auditory flutter, participants estimated a duration of 2500 ms as accurately as possible by pressing a button. The results showed a significant within‐individual correlation between the time distortion due to visual flicker and that due to auditory flutter. Additionally, we found that time distortion due to auditory flutter tended to be larger in female participants than in male participants. These results suggest that the mechanisms underlying subjective time dilation are similar between vision and audition within individuals, but that they vary across individuals.     

Audiotactile interactions in temporal perception

In the present review, we focus on how commonalities in the ontogenetic development of the auditory and tactile sensory systems may inform the interplay between these signals in the temporal domain. In particular, we describe the results of behavioral studies that have investigated temporal resolution (in temporal order, synchrony/asynchrony, and simultaneity judgment tasks), as well as temporal numerosity perception, and similarities in the perception of frequency across touch and hearing. The evidence reviewed here highlights features of audiotactile temporal perception that are distinctive from those seen for other pairings of sensory modalities. For instance, audiotactile interactions are characterized in certain tasks (e.g., temporal numerosity judgments) by a more balanced reciprocal influence than are other modality pairings. Moreover, relative spatial position plays a different role in the temporal order and temporal recalibration processes for audiotactile stimulus pairings than for other modality pairings. The effect exerted by both the spatial arrangement of stimuli and attention on temporal order judgments is described. Moreover, a number of audiotactile interactions occurring during sensory-motor synchronization are highlighted. We also look at the audiotactile perception of rhythm and how it may be affected by musical training. The differences emerging from this body of research highlight the need for more extensive investigation into audiotactile temporal interactions. We conclude with a brief overview of some of the key issues deserving of further research in this area.     

Subliminal action priming modulates the perceived intensity of sensory action consequences

The sense of control over the consequences of one’s actions depends on predictions about these consequences. According to an influential computational model, consistency between predicted and observed action consequences attenuates perceived stimulus intensity, which might provide a marker of agentic control. An important assumption of this model is that these predictions are generated within the motor system. However, previous studies of sensory attenuation have typically confounded motor-specific perceptual modulation with perceptual effects of stimulus predictability that are not specific to motor action. As a result, these studies cannot unambiguously attribute sensory attenuation to a motor locus. We present a psychophysical experiment on auditory attenuation that avoids this pitfall. Subliminal masked priming of motor actions with compatible prime–target pairs has previously been shown to modulate both reaction times and the explicit feeling of control over action consequences. Here, we demonstrate reduced perceived loudness of tones caused by compatibly primed actions. Importantly, this modulation results from a manipulation of motor processing and is not confounded by stimulus predictability. We discuss our results with respect to theoretical models of the mechanisms underlying sensory attenuation and subliminal motor priming.     

Multi-limb coordination and rhythmic variability under varying sensory availability conditions in children with DCD

Children with Developmental Coordination Disorder (DCD) have sensory processing deficits; how do these influence the interface between sensory input and motor performance? Previously, we found that children with DCD were less able to organize and maintain a gross motor coordination task in time to an auditory cue, particularly at higher frequencies [Whitall, J., Getchell, N., McMenamin, S., Horn, C., Wilms-Floet, A., & Clark, J. (2006). Perception-action coupling in children with and without DCD: Frequency locking between task relevant auditory signals and motor responses in a dual motor task. Child: Care, Health, and Development, 32, 679-692]. In the present study, we examine the same task (clapping in-phase to marching on a platform) under conditions involving the removal of vision and hearing. Eleven children with DCD (mean=7.21, SD=0.52 years), 7 typically developing (TD) children (mean=6.95+/-0.72 years), and 10 adults performed continuous clapping while marching under four conditions: with vision and hearing, without vision, without hearing, and without both. Results showed no significant condition effects for any measure taken. The DCD group was more variable in phasing their claps and footfalls than both the adult group and the TD group. There were also significant group effects for inter-clap interval coefficient of variation and inter-footfall interval coefficient of variation, with the DCD group being the most variable for both measures. Coherence analysis between limb combinations (e.g., left arm-right arm, right arm-left leg) revealed that the adults exhibited significantly greater coherence for each combination than both of the children's groups. The TD group showed significantly greater coherence than the DCD group for every limb combination except foot-foot and left hand-right foot. Measures of approximate entropy indicated that adults differed from children both with and without DCD in the structure of the variability across a trial with adults showing more complexity. Children with DCD are able to accomplish a self-initiated gross-motor coordination task but with increased variability for most but not all measures compared to typically developing children. The availability of visual and/or auditory information does not play a significant role in stabilizing temporal coordination of this task, suggesting that these are not salient sources of information for this particular task.     

The structure of sensory events and the accuracy of time judgments

We investigated how does the structure of empty time intervals influence temporal processing. In experiment 1, the intervals to be discriminated were the silent durations marked by two sensory signals, both lasting 10 or 500 ms; these signals were two identical flashes (intramodal: VV), or one visual flash (V) followed by an auditory tone (A) (intermodal: VA). For the range of duration under investigation (standards = 0.2, 0.6, 1, or 1.4 s), the results indicated that both the marker length and sensory mode influenced discrimination, but no interaction between these variables or between one of these variables and standard duration was significant. In experiment 2, we compared, for each of four marker-type conditions (VV, AA, VA, AV; and standard = 1 s), intervals marked by two 10 ms signals with intervals marked by unequal signal length (markers 1 and 2 lasting 10 and 500 ms, or 500 and 10 ms). As in experiment 1, the results revealed significant marker-mode and marker-length effects, but no significant interaction between these variables. Experiment 3 showed that, for the same conditions as in experiment 2, perceived duration is not influenced by marker length and that the variability of interval reproductions does not depend on the perceived duration of intervals. The results are discussed in the light of a single-clock hypothesis: marker-length and marker-mode effects are presented as being non-temporal sources of variability associated mainly with sensory and memory processes.     

Auditory-visual conflicts in the perceived duration of lights, tones and gaps

The perceived duration of a short tone (1,000 or 1,500 msec) was longer than that of a separately presented light of equal length. Thus, when light and tone were presented simultaneously, there was a conflict in perceived duration. In that case, the perceived duration of an interval filled with both light and tone was close to that of an interval filled with tone alone. A silent gap in otherwise continuous tone was perceived as longer than a gap in otherwise continuous light, and the perceived duration of a gap occurring simultaneously in both light and tone was close to that of a gap in tone alone. Thus, auditory dominance occurred under the preceding conditions-that is, auditory-visual conflicts in perceived duration, whether occurring between filled intervals or gaps, were resolved in favor of the auditory modality. Visual dominance occurred only under one condition, in which the intensity of tone was reduced, and in which the perceived duration of a 500-msec light was longer than that of a 500-msec tone. The finding of auditory dominance in the perception of time runs counter to the results of studies of sensory conflicts in spatial perception, where vision typically dominates audition and touch.     

Simple and contingent aftereffects of perceived duration in vision and audition

After repeated presentations of a long inspection tone (800 or 1,000 msec), a test tone of intermediate duration (600 msec) appeared shorter than it would otherwise appear. A short inspection tone (200 or 400 msec) tended to increase the apparent length of the intermediate test tone. Thus, a negative aftereffect of perceived auditory duration occurred, and a similar aftereffect occurred in the visual modality. These aftereffects, each involving a single sensory dimension, aresimple aftereffects. The following procedures producedcontingent aftereffects of perceived duration. A pair of lights, the first short and the second long, was presented repeatedly during an inspection period. When a pair of test lights of intermediate duration was then presented, the first member of the pair appeared longer in relation to the second. A similar aftereffect occurred in the auditory modality. In these latter aftereffects, the perceived duration of a test light or tone is contingent—dependent—on its temporal order, first or second, within a pair of test stimuli. An experiment designed to test the possibility of cross-modal transfer of contingent aftereffects between audition and vision found no significant cross-modal aftereffects.     

Electrophysiological evidence for age effects on sensory memory processing of tonal patterns

In older adults, difficulties processing complex auditory scenes, such as speech comprehension in noisy environments, might be due to a specific impairment of temporal processing at early, automatic processing stages involving auditory sensory memory (ASM). Even though age effects on auditory temporal processing have been well-documented, there is a paucity of research on how ASM processing of more complex tone-patterns is altered by age. In the current study, age effects on ASM processing of temporal and frequency aspects of two-tone patterns were investigated using a passive listening protocol. The P1 component, the mismatch negativity (MMN) and the P3a component of event-related brain potentials (ERPs) to tone frequency and temporal pattern deviants were recorded in younger and older adults as a measure of auditory event detection, ASM processing, and attention switching, respectively. MMN was elicited with smaller amplitude to both frequency and temporal deviants in older adults. Furthermore, P3a was elicited only in the younger adults. In conclusion, the smaller MMN amplitude indicates that automatic processing of both frequency and temporal aspects of two-tone patterns is impaired in older adults. The failure to initiate an attention switch, suggested by the absence of P3a, indicates that impaired ASM processing of patterns may lead to less distractibility in older adults. Our results suggest age-related changes in ASM processing of patterns that cannot be explained by an inhibitory deficit.     

Auditory illusion analogous to the Baldwin figure in vision

An auditory illusion was demonstrated in which there was perceptual shortening of the temporal interval contiguous to a low frequency (large) tone and/or perceptual lengthening of the temporal interval contiguous to a high frequency (small) tone. This auditory analogy to the Baldwin figure was demonstrated after first determining that there was a symmetrical expansion of perceived time during the auditory filled-duration illusion.     

Properties of intermodal transfer after dual visuo- and auditory-motor adaptation

Previous work documented that sensorimotor adaptation transfers between sensory modalities: When subjects adapt with one arm to a visuomotor distortion while responding to visual targets, they also appear to be adapted when they are subsequently tested with auditory targets. Vice versa, when they adapt to an auditory-motor distortion while pointing to auditory targets, they appear to be adapted when they are subsequently tested with visual targets. Therefore, it was concluded that visuomotor as well as auditory-motor adaptation use the same adaptation mechanism. Furthermore, it has been proposed that sensory information from the trained modality is weighted larger than sensory information from an untrained one, because transfer between sensory modalities is incomplete. The present study tested these hypotheses for dual arm adaptation. One arm adapted to an auditory-motor distortion and the other either to an opposite directed auditory-motor or visuomotor distortion. We found that both arms adapted significantly. However, compared to reference data on single arm adaptation, adaptation in the dominant arm was reduced indicating interference from the non-dominant to the dominant arm. We further found that arm-specific aftereffects of adaptation, which reflect recalibration of sensorimotor transformation rules, were stronger or equally strong when targets were presented in the previously adapted compared to the non-adapted sensory modality, even when one arm adapted visually and the other auditorily. The findings are discussed with respect to a recently published schematic model on sensorimotor adaptation.     

Working memory resources are shared across sensory modalities

A common assumption in the working memory literature is that the visual and auditory modalities have separate and independent memory stores. Recent evidence on visual working memory has suggested that resources are shared between representations, and that the precision of representations sets the limit for memory performance. We tested whether memory resources are also shared across sensory modalities. Memory precision for two visual (spatial frequency and orientation) and two auditory (pitch and tone duration) features was measured separately for each feature and for all possible feature combinations. Thus, only the memory load was varied, from one to four features, while keeping the stimuli similar. In Experiment 1, two gratings and two tones—both containing two varying features—were presented simultaneously. In Experiment 2, two gratings and two tones—each containing only one varying feature—were presented sequentially. The memory precision (delayed discrimination threshold) for a single feature was close to the perceptual threshold. However, as the number of features to be remembered was increased, the discrimination thresholds increased more than twofold. Importantly, the decrease in memory precision did not depend on the modality of the other feature(s), or on whether the features were in the same or in separate objects. Hence, simultaneously storing one visual and one auditory feature had an effect on memory precision equal to those of simultaneously storing two visual or two auditory features. The results show that working memory is limited by the precision of the stored representations, and that working memory can be described as a resource pool that is shared across modalities.