When hearing turns into playing: Movement induction by auditory stimuli in pianists

In this study, pianists were tested for learned associations between actions (movements on the piano) and their perceivable sensory effects (piano tones). Actions were examined that required the playing of two-tone sequences (intervals) in a four-choice paradigm. In Experiment 1, the intervals to be played were denoted by visual note stimuli. Concurrently with these imperative stimuli, task-irrelevant auditory distractor intervals were presented (“potential” action effects, congruent or incongruent). In Experiment 2, imperative stimuli were coloured squares, in order to exclude possible influences of spatial relationships of notes, responses, and auditory stimuli. In both experiments responses in the incongruent conditions were slower than those in the congruent conditions. Also, heard intervals actually “induced” false responses. The reaction time effects were more pronounced in Experiment 2. In nonmusicians (Experiment 3), no evidence for interference could be observed. Thus, our results show that in expert pianists potential action effects are able to induce corresponding actions, which demonstrates the existence of acquired action-effect associations in pianists.     

Motion detection and motion verbs: language affects low-level visual perception

Recent theories propose that semantic representation and sensorimotor processing have a common substrate via simulation. We tested the prediction that comprehension interacts with perception, using a standard psychophysics methodology. While passively listening to verbs that referred to upward or downward motion, and to control verbs that did not refer to motion, 20 subjects performed a motion-detection task, indicating whether or not they saw motion in visual stimuli containing threshold levels of coherent vertical motion. A signal detection analysis revealed that when verbs were directionally incongruent with the motion signal, perceptual sensitivity was impaired. Word comprehension also affected decision criteria and reaction times, but in different ways. The results are discussed with reference to existing explanations of embodied processing and the potential of psychophysical methods for assessing interactions between language and perception.     

Instrument specificity in experienced musicians

Previous studies have shown that experienced pianists have acquired integrated action-effect (A-E) associations. In the present study, we were interested in how specific these associations are for the own instrument by investigating pianists and guitarists. A-E associations were examined by testing whether the perception of a “potential” action-effect has an influence on actions. Participants played chords on their instrument in response to visual stimuli, while they were presented task-irrelevant auditory distractors (congruent or incongruent) in varying instrument timbre. In Experiment 1, pianists exhibited an interference effect with timbres of their own instrument category (keyboard instruments: piano and organ). In Experiment 2 guitarists showed an interference effect only with guitar timbre. Thus, integrated A-E associations primarily seem to consist of a specific component on a sensory-motor level involving the own instrument. Additionally, categorical knowledge about how an instrument is played seems to be involved.     

Coloring an action: Intending to produce color events eliminates the Stroop effect

The implications of an ideomotor approach to action control were investigated. In Experiment 1, participants made manual responses to letter stimuli and they were presented with response-contingent color patches, i.e., colored action effects. This rendered stimuli of the same color as an action's effect effective primes of that action, suggesting that bilateral associations were created between actions and the effects they produced. Experiment 2 combined this set-up with a manual Stroop task, i.e., participants responded to congruent, neutral, or incongruent color-word compounds. Standard Stroop effects were observed in a control group without action effects and in a group with target-incompatible action effects, but the reaction time Stroop effect was eliminated if actions produced target-compatible color effects (e.g., blue word --> left key --> blue patch). Experiment 3 did not replicate this interaction between target-effect compatibility and color-word congruency with color words as action effects, which rules out semantically based accounts. Theoretical implications for both action-effect acquisition and the Stroop effect are discussed. It is suggested that learning action effects, the features of which overlap with the target, allows and motivates people to recode their actions in ways that make them more stimulus-compatible. This provides a processing shortcut for translating the relevant stimulus into the correct response and, thus, shields processing from the impact of competing word distractors.     

Action Understanding in the Superior Temporal Sulcus Region

ABSTRACT— The posterior superior temporal sulcus (STS) region plays an important role in the perception of social acts, although its full role has not been completely clarified. This functional magnetic resonance imaging experiment examined activity in the STS region as participants viewed actions that were congruent or incongruent with intentions established by a previous emotional context. Participants viewed an actress express either a positive or a negative emotion toward one of two objects and then subsequently pick up one of them. If the object that was picked up had received positive regard, or if the object that was not picked up had received negative regard, the action was congruent; otherwise, the action was incongruent. Activity in the right posterior STS region was sensitive to the congruency between the action and the actress's emotional expression (i.e., STS activity was greater on incongruent than on congruent trials). These findings suggest that the posterior STS represents not only biological motion, but also how another person's motion is related to his or her intentions.     

Action-effect coupling in pianists

Recent theories have stressed the role of effect anticipation in action control. Such a mechanism requires the prior acquisition of integrated action-effect associations. The strength of such associations should directly depend on the amount of learning, and therefore be most pronounced in motor experts. Using an interference paradigm, we investigated whether evidence of such representations can be demonstrated in expert pianists. Participants were required to play chords on a keyboard in response to imperative visual stimuli. Concurrently, task-irrelevant auditory stimuli (potential action effects) were presented that were congruent or incongruent with the chords to be played. In Experiment 1 we found evidence that expert pianists, compared with non-musicians, have acquired such action-effects representations. Response times were slower when the auditory stimulus was incongruent with the required response. In order to ascertain the locus of interference, we varied imperative stimuli and responses in Experiments 2 and 3. The results indicate that, for the most part, interference occurs on the response level rather than on an abstract level. However, the perception of action effects also evokes processing of abstract features, like the concept of major–minor mode.     

The effect of sound on visual apparent movement

Twelve subjects found the longest possible interstimulus interval (ISI) at which they perceived continuous apparent motion of one light instead of partial motion or succession between two lights. In the visual condition, two lights only were presented. In the bimodal conditions, binaurally presented tones were presented synchronously with the lights, and the lights and tones were either spatially congruent (in phase) or incongruent (180 degrees out of phase). Bimodal presentations lowered the upper ISI threshold for the perception of visual apparent motion, and the reduction was greater when the tones and lights were spatially congruent. The threshold reduction may be caused by a perceptual inference about localization of the lights in space or by a change in visual persistence.     

Interactions between apparent motion rivalry in vision and touch

In multistable perception, the brain alternates between several perceptual explanations of ambiguous sensory signals. It is unknown whether multistable processes can interact across the senses. In the study reported here, we presented subjects with unisensory (visual or tactile), spatially congruent visuotactile, and spatially incongruent visuotactile apparent motion quartets. Congruent stimulation induced pronounced visuotactile interactions, as indicated by increased dominance times for both vision and touch, and an increased percentage bias for the percept already dominant under unisensory stimulation. Thus, the joint evidence from vision and touch stabilizes the more likely perceptual interpretation and thereby decelerates the rivalry dynamics. Yet the temporal dynamics depended also on subjects' attentional focus and was generally slower for tactile than for visual reports. Our results support Bayesian approaches to perceptual inference, in which the probability of a perceptual interpretation is determined by combining visual, tactile, or visuotactile evidence with modality-specific priors that depend on subjects' attentional focus. Critically, the specificity of visuotactile interactions for spatially congruent stimulation indicates multisensory rather than cognitive-bias mechanisms.     

The role of cue-response mapping in motorvisual impairment and facilitation: evidence for different roles of action planning and action control in motorvisual dual-task priming

Previous research has shown that actions impair the visual perception of categorically action-consistent stimuli. On the other hand, actions can also facilitate the perception of spatially action-consistent stimuli. We suggest that motorvisual impairment is due to action planning processes, while motorvisual facilitation is due to action control mechanisms. This implies that because action planning is sensitive to modulations by cue-response mapping so should motorvisual impairment, while motorvisual facilitation should be insensitive to manipulations of cue-response mapping as is action control. We tested this prediction in three dual-task experiments. The impact of performing left and right key presses on the perception of unrelated, categorically or spatially consistent, stimuli was studied. As expected, we found motorvisual impairment for categorically consistent stimuli and motorvisual facilitation for spatially consistent stimuli. In all experiments, we compared congruent with incongruent cue-key mappings. Mapping manipulations affected motorvisual impairment, but not motorvisual facilitation. The results support our suggestion that motorvisual impairment is due to action planning, and motorvisual facilitation to action control.     

Visual contribution to the multistable perception of speech

The multistable perception of speech, or verbal transformation effect, refers to perceptual changes experienced while listening to a speech form that is repeated rapidly and continuously. In order to test whether visual information from the speaker's articulatory gestures may modify the emergence and stability of verbal auditory percepts, subjects were instructed to report any perceptual changes during unimodal, audiovisual, and incongruent audiovisual presentations of distinct repeated syllables. In a first experiment, the perceptual stability of reported auditory percepts was significantly modulated by the modality of presentation. In a second experiment, when audiovisual stimuli consisting of a stable audio track dubbed with a video track that alternated between congruent and incongruent stimuli were presented, a strong correlation between the timing of perceptual transitions and the timing of video switches was found. Finally, a third experiment showed that the vocal tract opening onset event provided by the visual input could play the role of a bootstrap mechanism in the search for transformations. Altogether, these results demonstrate the capacity of visual information to control the multistable perception of speech in its phonetic content and temporal course. The verbal transformation effect thus provides a useful experimental paradigm to explore audiovisual interactions in speech perception.     

Coupling kinematics of memory and kinematics of movement: the conditions for a psychological relativity

The organization of spatial and temporal information relative to memory-movement interactions was studied in a recall task. Stimuli consisted of four different configurations of eight dots, presented sequentially. In two configurations, the stimuli were spatiotemporal congruent, with (constant or variable) inter-stimulus distances corresponding to (constant or variable) inter-stimulus time intervals. In the other two configurations, the stimuli were spatiotemporal incongruent, with (constant or variable) inter-stimulus distances not corresponding to (variable or constant) inter-stimulus time intervals. After a learning phase consisting of 20 presentations of the target configuration, participants performed a series of pointing movements to reproduce both spatial and temporal characteristics of the stimulus 60 times in succession without ever re-examining the target configuration. Classically, in incongruent spatiotemporal conditions, the reproduction of, respectively, constant distances or constant time intervals are strongly disturbed by the simultaneous perception of variable inter-stimulus time intervals (Tau effect) or variable distances (Kappa effect). We assume that these spatiotemporal dependencies, which occur when the response relates to one dimension only, crucially depend on the non-integration of motoric aspects in memory.     

Reading action word affects the visual perception of biological motion

In the present study, we investigate whether reading an action-word can influence subsequent visual perception of biological motion. The participant's task was to perceptually judge whether a human action identifiable in the biological motion of a point-light display embedded in a high density mask was present or not in the visual sequence, which lasted for 633 ms on average. Prior to the judgement task, participants were exposed to an abstract verb or an action verb for 500 ms, which was related to the human action according to a congruent or incongruent semantic relation. Data analysis showed that correct judgements were not affected by action verbs, whereas a facilitation effect on response time (49 ms on average) was observed when a congruent action verb primed the judgement of biological movements. In relation with the existing literature, this finding suggests that the perception, the planning and the linguistic coding of motor action are subtended by common motor representations.     

Observing repetitive finger movements modulates response times of auditorily cued finger movements

Our motor and perceptual representations of actions seem to be intimately linked and the human mirror neuron system (MNS) has been proposed as the mediator. In two experiments, we presented biological or non-biological movement stimuli that were either congruent or incongruent to a required response prompted by a tone. When the tone occurred with the onset of the last movement in a series, i.e., it was perceived during the movement presentation, congruent biological stimuli resulted in faster reaction times than congruent non-biological stimuli. The opposite was observed for incongruent stimuli. When the tone was presented after visual movement stimulation, however, no such interaction was present. This implies that biological movement stimuli only affect motor behaviour during visual processing but not thereafter. These data suggest that the MNS is an "online" system; longstanding repetitive visual stimulation (Experiment 1) has no benefit in comparison to only one or two repetitions (Experiment 2).     

Intention and attention in ideomotor learning

Human actions may be carried out in response to exogenous stimuli (stimulus based) or they may be selected endogenously on the basis of the agent's intentions (intention based). We studied the functional differences between these two types of action during action-effect (ideomotor) learning. Participants underwent an acquisition phase, in which each key-press (left/right) triggered a specific tone (low pitch/high pitch) either in a stimulus-based or in an intention-based action mode. Consistent with previous findings, we demonstrate that auditory action effects gain the ability to prime their associated responses in a later test phase only if the actions were selected endogenously during acquisition phase. Furthermore, we show that this difference in ideomotor learning is not due to different attentional demands for stimulus-based and intention-based actions. Our results suggest that ideomotor learning depends on whether or not the action is selected in the intention-based action mode, whereas the amount of attention devoted to the action-effect is less important.     

Neural processing of intentional biological motion in unaffected siblings of children with autism spectrum disorder: An fMRI study

Despite often showing behaviorally typical levels of social cognitive ability, unaffected siblings of children with autism spectrum disorder have been found to show similar functional and morphological deficits within brain regions associated with social processing. They have also been reported to show increased activation to biological motion in these same regions, such as the posterior superior temporal sulcus (pSTS), relative to both children with autism and control children. It has been suggested that this increased activation may represent a compensatory reorganization of these regions as a result of the highly heritable genetic influence of autism. However, the response patterns of unaffected siblings in the domain of action perception are unstudied, and the phenomenon of compensatory activation has not yet been replicated. The present study used functional magnetic resonance imaging to determine the neural responses to intentional biological actions in 22 siblings of children with autism and 22 matched controls. The presented actions were either congruent or incongruent with the actor’s emotional cue. Prior studies reported that typically developing children and adults, but not children with autism, show increased activation to incongruent actions (relative to congruent), within the pSTS and dorsolateral prefrontal cortex. We report that unaffected siblings did not show a compensatory response, or a preference for incongruent over congruent trials, in any brain region. Moreover, interaction analyses revealed a sub-region of the pSTS in which control children showed an incongruency preference to a significantly greater degree than siblings, which suggests a localized deficit in siblings. A sample of children with autism also did not show differential activation in the pSTS, providing further evidence that it is an area of selective disruption in children with autism and siblings. While reduced activation to both conditions was unique to the autism sample, lack of differentiation to incongruent and congruent intentional actions was common to both children with ASD and unaffected siblings.     

Using art for comparison and distraction: Effects on negative emotions and judgements of satisfaction

Abstract

Two studies examined whether emotional comparison and distraction with emotion congruent and incongruent art would improve the well-being of dysphoric undergraduates. In both studies, subjects: (1) imagined a sad event; (2) compared their mood to that expressed by incongruent art (upward comparison) or congruent art (downward comparison); or focused on technical features of incongruent art (incongruent distraction) or congruent art (congruent distraction); and (3) rated their emotions and life satisfaction. The incongruent distraction group reported feeling more positive and more satisfied, and the downward comparison group reported feeling more satisfied, than the upward comparison or congruent distraction groups. Thus, comparison and distraction can improve well-being when directed towards emotion congruent and incongruent art, respectively.     

Audiovisual influences on the perception of visual apparent motion: exploring the effect of a single sound

Previous research has shown that irrelevant sounds can facilitate the perception of visual apparent motion. Here the effectiveness of a single sound to facilitate motion perception was investigated in three experiments. Observers were presented with two discrete lights temporally separated by stimulus onset asynchronies from 0 to 350 ms. After each trial, observers classified their impression of the stimuli using a categorisation system. A short sound presented temporally (and spatially) midway between the lights facilitated the impression of motion relative to baseline (lights without sound), whereas a sound presented either before the first or after the second light or simultaneously with the lights did not affect motion impression. The facilitation effect also occurred with sound presented far from the visual display, as well as with continuous-sound that was started with the first light and terminated with the second light. No facilitation of visual motion perception occurred if the sound was part of a tone sequence that allowed for intramodal perceptual grouping of the auditory stimuli prior to the critical audiovisual stimuli. Taken together, the findings are consistent with a low-level audiovisual integration approach in which the perceptual system merges temporally proximate sound and light stimuli, thereby provoking the impression of a single multimodal moving object.     

Eighteen-month-olds integrate verbal cues into their action processing: Evidence from ERPs and mu power

Behavioral research has shown that infants use both behavioral cues and verbal cues when processing the goals of others’ actions. For instance, 18-month-olds selectively imitate an observed goal-directed action depending on its (in)congruence with a model’s previous verbal announcement of a desired action goal. This EEG-study analyzed the electrophysiological underpinnings of these behavioral findings on the two functional levels of conceptual action processing and motor activation. Mid-latency mean negative ERP amplitude and mu-frequency band power were analyzed while 18-month-olds (N = 38) watched videos of an adult who performed one out of two potential actions on a novel object. In a within-subjects design, the action demonstration was preceded by either a congruent or an incongruent verbally announced action goal (e.g., “up” or “down” and upward movement). Overall, ERP negativity did not differ between conditions, but a closer inspection revealed that in two subgroups, about half of the infants showed a broadly distributed increased mid-latency ERP negativity (indicating enhanced conceptual action processing) for either the congruent or the incongruent stimuli, respectively. As expected, mu power at sensorimotor sites was reduced (indicating enhanced motor activation) for congruent relative to incongruent stimuli in the entire sample. Both EEG correlates were related to infants’ language skills. Hence, 18-month-olds integrate action-goal-related verbal cues into their processing of others’ actions, at the functional levels of both conceptual processing and motor activation. Further, cue integration when inferring others’ action goals is related to infants’ language proficiency.     

Contiguity and contingency in action-effect learning

According to the two-stage model of voluntary action, the ability to perform voluntary action is acquired in two sequential steps. Firstly, associations are acquired between representations of movements and of the effects that frequently follow them. Secondly, the anticipation or perception of an acquired action effect primes the movement that has been learnt to produce this effect; the acquired action-effect associations thus mediate the selection of actions that are most appropriate to achieve an intended action goal. If action-effect learning has an associative basis, it should be influenced by factors that are known to affect instrumental learning, such as the temporal contiguity and the probabilistic contingency of movement and effect. In two experiments, the contiguity or the contingency between key presses and subsequent tones was manipulated in various ways. As expected, both factors affected the acquisition of action-effect relations as assessed by the potency of action effects to prime the corresponding action in a later behavioral test. In particular, evidence of action-effect associations was obtained only if the effect of the action was delayed for no more than 1 s, if the effect appeared more often in the presence than in the absence of the action, or if action and effect were entirely uncorrelated but the effect appeared very often. These findings support the assumption that the control of voluntary actions is based on action-effect representations that are acquired by associative learning mechanisms.     

Neural processing of intentional biological motion in unaffected siblings of children with autism spectrum disorder: An fMRI study

Despite often showing behaviorally typical levels of social cognitive ability, unaffected siblings of children with autism spectrum disorder have been found to show similar functional and morphological deficits within brain regions associated with social processing. They have also been reported to show increased activation to biological motion in these same regions, such as the posterior superior temporal sulcus (pSTS), relative to both children with autism and control children. It has been suggested that this increased activation may represent a compensatory reorganization of these regions as a result of the highly heritable genetic influence of autism. However, the response patterns of unaffected siblings in the domain of action perception are unstudied, and the phenomenon of compensatory activation has not yet been replicated. The present study used functional magnetic resonance imaging to determine the neural responses to intentional biological actions in 22 siblings of children with autism and 22 matched controls. The presented actions were either congruent or incongruent with the actor’s emotional cue. Prior studies reported that typically developing children and adults, but not children with autism, show increased activation to incongruent actions (relative to congruent), within the pSTS and dorsolateral prefrontal cortex. We report that unaffected siblings did not show a compensatory response, or a preference for incongruent over congruent trials, in any brain region. Moreover, interaction analyses revealed a sub-region of the pSTS in which control children showed an incongruency preference to a significantly greater degree than siblings, which suggests a localized deficit in siblings. A sample of children with autism also did not show differential activation in the pSTS, providing further evidence that it is an area of selective disruption in children with autism and siblings. While reduced activation to both conditions was unique to the autism sample, lack of differentiation to incongruent and congruent intentional actions was common to both children with ASD and unaffected siblings.