SNARC (spatial–numerical association of response codes) meets SPARC (spatial–pitch association of response codes): Automaticity and interdependency in compatibility effects

Concepts, including the mental number line, or addressing pitch as high and low, suggest that the spatial–numerical and spatial–pitch association of response codes (SNARC and SPARC) effects are domain-specific and thus independent. Alternatively, there may be dependencies between these effects, because they share common automatic or controlled decision mechanisms. In two experiments, participants were presented with spoken numbers in different pitches; their numerical value, pitch, and response compatibility were varied systematically. This allowed us to study SNARC and SPARC effects in a factorial design (see also Fischer, Riello, Giordano, &; Rusconi, 2013 Fischer, M. H., Riello, M., Giordano, B. L., &; Rusconi, E. (2013). Singing numbers?…?in cognitive space--a dual-task study of the link between pitch, space, and numbers. Topics in Cognitive Science, 5(2), 354–366. Retrieved from http://doi.org/10.1111/tops.12017[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). Participants judged the stimuli on numerical magnitude, pitch, or parity (odd–even). In all tasks, the SNARC and SPARC effects had superadditive interactions. These were interpreted as both effects sharing a common mechanism. The task variation probes the mechanism: In the magnitude judgement task, numerical magnitude was explicit, whereas pitch was implicit; in the pitch judgement task, it was vice versa. In the parity judgement task, both dimensions were implicit. Regardless of whether they were implicit or explicit, both SNARC and SPARC effects occurred in all tasks. We concluded that by not requiring focal attention the common mechanism operates automatically.     

Pitch discrimination without awareness in congenital amusia: Evidence from event-related potentials

Congenital amusia is a lifelong disorder characterized by a difficulty in perceiving and producing music despite normal intelligence and hearing. Behavioral data have indicated that it originates from a deficit in fine-grained pitch discrimination, and is expressed by the absence of a P3b event-related brain response for pitch differences smaller than a semitone and a bigger N2b–P3b brain response for large pitch differences as compared to controls. However, it is still unclear why the amusic brain overreacts to large pitch changes. Furthermore, another electrophysiological study indicates that the amusic brain can respond to changes in melodies as small as a quarter-tone, without awareness, by exhibiting a normal mismatch negativity (MMN) brain response. Here, we re-examine the event-related N2b–P3b components with the aim to clarify the cause of the larger amplitude observed by Peretz, Brattico, and Tervaniemi (2005), by experimentally matching the number of deviants presented to the controls according to the number of deviants detected by amusics. We also re-examine the MMN component as well as the N1 in an acoustical context to investigate further the pitch discrimination deficit underlying congenital amusia. In two separate conditions, namely ignore and attend, we measured the MMN, the N1, the N2b and the P3b to tones that deviated by an eight of a tone (25 cents) or whole tone (200 cents) from a repeated standard tone. The results show a normal MMN, a seemingly normal N1, a normal P3b for the 200 cents pitch deviance, and no P3b for the small 25 cents pitch differences in amusics. These results indicate that the amusic brain responds to small pitch differences at a pre-attentive level of perception, but is unable to detect consciously those same pitch deviances at a later attentive level. The results are consistent with previous MRI and fMRI studies indicating that the auditory cortex of amusic individuals is functioning normally.     

Differential recognition of pitch patterns in discrete and gliding stimuli in congenital amusia: evidence from Mandarin speakers

This study examined whether "melodic contour deafness" (insensitivity to the direction of pitch movement) in congenital amusia is associated with specific types of pitch patterns (discrete versus gliding pitches) or stimulus types (speech syllables versus complex tones). Thresholds for identification of pitch direction were obtained using discrete or gliding pitches in the syllable /ma/ or its complex tone analog, from nineteen amusics and nineteen controls, all healthy university students with Mandarin Chinese as their native language. Amusics, unlike controls, had more difficulty recognizing pitch direction in discrete than in gliding pitches, for both speech and non-speech stimuli. Also, amusic thresholds were not significantly affected by stimulus types (speech versus non-speech), whereas controls showed lower thresholds for tones than for speech. These findings help explain why amusics have greater difficulty with discrete musical pitch perception than with speech perception, in which continuously changing pitch movements are prevalent.     

先天性音乐障碍的音高空间表征能力研究

利用音高空间表征任务、心理旋转任务和音高辨别任务,探讨先天性音乐障碍者音高空间表征能力、心理旋转加工能力和音高辨别能力。结果发现：先天性音乐障碍组音高空间表征任务标准差、心理旋转任务错误率和音高辨别任务错误率均显著高于控制组。 这些结果表明先天性音乐障碍并非音乐特异性障碍,而是音乐相关性障碍。     

Correspondence effects with torches: Grasping affordance or visual feature asymmetry?

Three experiments were conducted to determine whether an object-based correspondence effect for torch (flashlight) stimuli reported by Pellicano et al. [(2010 Pellicano, A., Iani, C., Borghi, A. M., Rubichi, S., & Nicoletti, R. (2010). Simon-like and functional affordance effects with tools: The effects of object perceptual discrimination and object action state. Quarterly Journal of Experimental Psychology, 63, 2190–2201. doi: 10.1080/17470218.2010.486903[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). Simon-like and functional affordance effects with tools: The effects of object perceptual discrimination and object action state. Quarterly Journal of Experimental Psychology, 63, 2190–2201] is due to a grasping affordance provided by the handle or asymmetry of feature markings on the torch. In Experiment 1 the stimuli were the same as those from Pellicano et al.'s Experiment 2, whereas in Experiments 2 and 3 the stimuli were modified versions with the graspable handle removed. Participants in all experiments performed upright/inverted orientation judgements on the torch stimuli. The results of Experiment 1 replicated those of Pellicano et al.: A small but significant object-based correspondence effect was evident, mainly when the torch was in an active state. With the handle of the torch removed in Experiment 2, making the barrel markings more asymmetric in the display, the correspondence effect was larger. Experiment 3 directly demonstrated an effect of barrel-marking asymmetry on the correspondence effect: When only the half of the markings nearest the light end of the torch was included, the correspondence effect reversed to favour the light end. The results are in agreement with a visual feature-asymmetry account and are difficult to reconcile with a grasping-affordance account.     

Effects of visual cue and response assignment on spatial stimulus coding in stimulus–response compatibility

Tlauka and McKenna (2000 Tlauka, M. and McKenna, F. P. 2000. Hierarchical knowledge influences stimulus–response compatibility effects. Quarterly Journal of Experimental Psychology, 53A: 85–103.  [Google Scholar]) reported a reversal of the traditional stimulus–response compatibility (SRC) effect (faster responding to a stimulus presented on the same side than to one on the opposite side) when the stimulus appearing on one side of a display is a member of a superordinate unit that is largely on the opposite side. We investigated the effects of a visual cue that explicitly shows a superordinate unit, and of assignment of multiple stimuli within each superordinate unit to one response, on the SRC effect based on superordinate unit position. Three experiments revealed that stimulus–response assignment is critical, while the visual cue plays a minor role, in eliciting the SRC effect based on the superordinate unit position. Findings suggest bidirectional interaction between perception and action and simultaneous spatial stimulus coding according to multiple frames of reference, with contribution of each coding to the SRC effect flexibly varying with task situations.     

Do you see what I’m singing? Visuospatial movement biases pitch perception

The nature of the connection between musical and spatial processing is controversial. While pitch may be described in spatial terms such as “high” or “low”, it is unclear whether pitch and space are associated but separate dimensions or whether they share representational and processing resources. In the present study, we asked participants to judge whether a target vocal note was the same as (or different from) a preceding cue note. Importantly, target trials were presented as video clips where a singer sometimes gestured upward or downward while singing that target note, thus providing an alternative, concurrent source of spatial information. Our results show that pitch discrimination was significantly biased by the spatial movement in gesture, such that downward gestures made notes seem lower in pitch than they really were, and upward gestures made notes seem higher in pitch. These effects were eliminated by spatial memory load but preserved under verbal memory load conditions. Together, our findings suggest that pitch and space have a shared representation such that the mental representation of pitch is audiospatial in nature.     

Task-defined associations are mode specific for selection of relevant dimension but mode independent for selection of location mapping

When compatible and incompatible mappings of a location-relevant task are mixed, or a location-relevant task is mixed with a task for which stimulus location is irrelevant, the benefit of the compatible mapping is eliminated for physical locations and enhanced for location words. Two experiments examined the influence of presenting the location information for the mixed conditions in different stimulus modes (physical location or word). Experiment 1 showed that the effects of mixing location-relevant and location-irrelevant tasks on the spatial compatibility and Simon effects are reduced when the location information is presented in different modes for the two tasks. Experiment 2 showed, in contrast, that the mode distinction had little influence on the effects of mixed compatible and incompatible mappings for location-relevant tasks: The compatibility effect was eliminated for physical locations and enhanced for words, as when there is no mode distinction. Thus, when location is relevant for one task and colour for the other, the task-defined associations of locations to responses are mode specific, but when location is relevant for both tasks, the associations are mode independent.     

Motor inhibition associated with the affordance of briefly displayed objects

Research has demonstrated that left- and right-hand responses are facilitated when they are performed with the hand compatible with the orientation of a viewed object. This suggests that graspable objects automatically activate the motor representations that correspond to their orientation. It has recently been proposed that similar positive stimulus–response compatibility effects (PCE) may turn into negative compatibility effects (NCE) when a prime object is displayed very briefly. These NCEs are suggested to reflect motor inhibition mechanisms—motor activation triggered by briefly viewed objects may be treated by the motor system as unwanted, and thus it is rapidly inhibited. We examined whether the motor activation triggered by the orientation of a task-irrelevant object is similarly inhibited when the object is displayed briefly. In Experiment 1, a NCE was observed between the orientation of an object and the responding hand when the object was displayed for 30 or 70 ms. The effect turned into a PCE when the object was displayed for 370 ms. Experiment 2 confirmed that this motor inhibition effect was produced by the handle affordance of the object rather than some abstract visual properties of the object.     

Stimulus–response correspondence effect as a function of temporal overlap between relevant and irrelevant information processing

The stimulus–response correspondence (SRC) effect refers to advantages in performance when stimulus and response correspond in dimensions or features, even if the common features are irrelevant to the task. Previous research indicated that the SRC effect depends on the temporal course of stimulus information processing. The current study investigated how the temporal overlap between relevant and irrelevant stimulus processing influences the SRC effect. In this experiment, the irrelevant stimulus (a previously associated tone) preceded the relevant stimulus (a coloured rectangle). The irrelevant and relevant stimuli onset asynchrony was varied to manipulate the temporal overlap between the irrelevant and relevant stimuli processing. Results indicated that the SRC effect size varied as a quadratic function of the temporal overlap between the relevant stimulus and irrelevant stimulus. This finding extends previous experimental observations that the SRC effect size varies in an increasing or decreasing function with reaction time. The current study demonstrated a quadratic function between effect size and the temporal overlap.     

Stimulus–response compatibility for mixed mappings and tasks with unique responses

For two stimulus locations mapped to two keypresses, reaction time is shorter when the mapping is compatible than when it is not (the stimulus–response compatibility, SRC, effect). A similar result, called the Simon effect, occurs when stimulus location is irrelevant, and colour is relevant. When compatibly mapped trials are intermixed with incompatibly mapped trials or Simon task trials, the compatibility effect is eliminated, and the Simon effect is influenced by the location mapping. In five experiments, we examined whether similar mixing effects occur when the two spatial mappings or location-relevant and location-irrelevant tasks use distinct keypresses on the left and right hands. Mixing had considerably less influence on the SRC and Simon effects than it does when the intermixed trial types or tasks share the same responses, even though response time was lengthened to a similar extent. Mixing two tasks for which stimulus location was irrelevant yielded no within-task Simon effect, but the effect was also absent when four stimuli were assigned to two responses on a single hand. The relative lack of influence of mixing on the SRC and Simon effects when the tasks have unique responses implies that suppression of direct activation of the corresponding response occurs primarily when the tasks share responses.     

Stimulus–classification and stimulus–action associations: Effects of repetition learning and durability

It has been shown that acquired stimulus–response bindings result from at least two types of associations from the stimulus to the task (stimulus–task or stimulus–classification; S–C) and from the stimulus to the motor response (stimulus–response or stimulus–action; S–A). These types of associations have been shown to independently affect behaviour. This finding suggests that they are processed in different pathways or different parts of a pathway at the neural level. Here we test a hypothesis that such associations may be differentially affected by repetition learning and that such effects may be detected by measuring their durability against overwriting. We show that both S–C and S–A associations are in fact strengthened when learning is boosted by increasing repetitions of the primes. However, the results further suggest that associations between stimuli and actions have less durable effects on behaviour and that the durability of S–C and S–A associations is independent of repetition learning. This is an important finding for the understanding of the underlying mechanisms of associative learning and particularly raises the question of which processes may affect flexibility of learning.     

Transfer of orthogonal stimulus–response mappings to an orthogonal Simon task

When up–down stimulus locations are mapped to left–right keypresses, an overall advantage for the up–right/down–left mapping is often obtained that varies as a function of response eccentricity. This orthogonal stimulus–response compatibility (SRC) effect also occurs when stimulus location is irrelevant, a phenomenon called the orthogonal Simon effect, and has been attributed to correspondence of stimulus and response code polarities. The Simon effect for horizontal stimulus–response (S–R) arrangements has been shown to be affected by short-term S–R associations established through the mapping used for a prior SRC task in which stimulus location was relevant. We examined whether such associations also transfer between orthogonal SRC and Simon tasks and whether correspondence of code polarities continues to contribute to performance in the Simon task. In Experiment 1, the orthogonal Simon effect was larger after practising with an up–right/down–left mapping of visual stimuli to responses than with the alternative mapping, for which the orthogonal Simon effect tended to reverse. Experiment 2 showed similar results when practice was with high (up) and low (down) pitch tones, though the influence of practice mapping was not as large as that in Experiment 1, implying that the short-term S–R associations acquired in practice are at least in part not modality specific. In Experiment 3, response eccentricity and practice mapping were shown to have separate influences on the orthogonal Simon effect, as expected if both code polarity and acquired S–R associations contribute to performance.     

Flanker and Simon effects interact at the response selection stage

The present study aimed at investigating the processing stage underlying stimulus–stimulus (S–S) congruency effects by examining the relation of a particular type of congruency effect (i.e., the flanker effect) with a stimulus–response (S–R) spatial correspondence effect (i.e., the Simon effect). Experiment 1 used a unilateral flanker task in which the flanker also acted as a Simon-like accessory stimulus. Results showed a significant S–S Congruency × S–R Correspondence interaction: An advantage for flanker–response spatially corresponding trials was observed in target–flanker congruent conditions, whereas, in incongruent conditions, there was a noncorresponding trials' advantage. The analysis of the temporal trend of the correspondence effects ruled out a temporal-overlap account for the observed interaction. Moreover, results of Experiment 2, in which the flanker did not belong to the target set, demonstrated that this interaction cannot be attributed to perceptual grouping of the target–flanker pairs and referential coding of the target with respect to the flanker in the congruent and incongruent conditions, respectively. Taken together, these findings are consistent with a response selection account of congruency effects: Both the position and the task-related attribute of the flanker would activate the associated responses. In noncorresponding-congruent trials and corresponding-incongruent trials, this would cause a conflict at the response selection stage.     

When speech enhances Spatial Musical Association of Response Codes: Joint spatial associations of pitch and timbre in nonmusicians

Previous studies have shown that the effect of the Spatial Musical Association of Response Codes (SMARC) depends on various features, such as task conditions (whether pitch height is implicit or explicit), response dimension (horizontal vs. vertical), presence or absence of a reference tone, and former musical training of the participants. In the present study, we investigated the effects of pitch range and timbre: in particular, how timbre (piano vs. vocal) contributes to the horizontal and vertical SMARC effect in nonmusicians under varied pitch range conditions. Nonmusicians performed a timbre judgement task in which the pitch range was either small (6 or 8 semitone steps) or large (9 or 12 semitone steps) in a horizontal and a vertical response setting. For piano sounds, SMARC effects were observed in all conditions. For the vocal sounds, in contrast, SMARC effects depended on pitch range. We concluded that the occurrence of the SMARC effect, especially in horizontal response settings, depends on the interaction of the timbre (vocal and piano) and pitch range if vocal and instrumental sounds are combined in one experiment: the human voice enhances the attention, both to the vocal and the instrumental sounds.     

Spatial representation of pitch height: the SMARC effect

Through the preferential pairing of response positions to pitch, here we show that the internal representation of pitch height is spatial in nature and affects performance, especially in musically trained participants, when response alternatives are either vertically or horizontally aligned. The finding that our cognitive system maps pitch height onto an internal representation of space, which in turn affects motor performance even when this perceptual attribute is irrelevant to the task, extends previous studies on auditory perception and suggests an interesting analogy between music perception and mathematical cognition. Both the basic elements of mathematical cognition (i.e. numbers) and the basic elements of musical cognition (i.e. pitches), appear to be mapped onto a mental spatial representation in a way that affects motor performance.     

Response-specifying cue for action interferes with perception of feature-sharing stimuli

Perceiving a visual stimulus is more difficult when a to-be-executed action is compatible with that stimulus, which is known as blindness to response-compatible stimuli. The present study explored how the factors constituting the action event (i.e., response-specifying cue, response intention, and response feature) affect the occurrence of this blindness effect. The response-specifying cue varied along the horizontal and vertical dimensions, while the response buttons were arranged diagonally. Participants responded based on one dimension randomly determined in a trial-by-trial manner. The response intention varied along a single dimension, whereas the response location and the response-specifying cue varied within both vertical and horizontal dimensions simultaneously. Moreover, the compatibility between the visual stimulus and the response location and the compatibility between that stimulus and the response-specifying cue was separately determined. The blindness effect emerged exclusively based on the feature correspondence between the response-specifying cue of the action task and the visual target of the perceptual task. The size of this stimulus–stimulus (S–S) blindness effect did not differ significantly across conditions, showing no effect of response intention and response location. This finding emphasizes the effect of stimulus factors, rather than response factors, of the action event as a source of the blindness to response-compatible stimuli.     

Action–effect congruence during observational learning leads to faster action sequence learning

Common coding theory suggests that any action (pressing a piano key) is intimately linked with its resultant sensory effect (an auditory musical tone). We conducted two experiments to explore the effect of varying auditory action–effect patterns during complex action learning. In Experiment 1, participants were assigned to 1 of 4 groups, watched a silent video of a hand playing a sequence on a piano keyboard with no auditory action effect (observation) and were asked to practise and perform the sequence on an identical keyboard with varying action effects (reproduction). During reproduction, Group 1 heard no auditory tones (identical to observed video), Group 2 heard typical scale-ascending piano tones with each key press, Group 3 heard fixed but out-of-sequence piano tones with each key press, and Group 4 heard random piano tones with each key press. In Experiment two, new participants were assigned to 1 of 2 groups and watched an identical video; however, the video in this experiment contained typical, scale-ascending piano sounds. During reproduction, Group 1 heard no auditory tones while Group 2 heard typical, scale-ascending piano tones with each key press (identical to observed video). Our results showed that participants whose action–effect patterns during reproduction matched those in the observed video learned the action sequence faster than participants whose action–effect patterns during reproduction differed from those in the observed video. Additionally, our results suggest that adding an effect during reproduction (when one is absent during observation) is somewhat more detrimental to action sequence learning than removing an effect during reproduction (when one is present during observation).