Action–effect congruence during observational learning leads to faster action sequence learning

Common coding theory suggests that any action (pressing a piano key) is intimately linked with its resultant sensory effect (an auditory musical tone). We conducted two experiments to explore the effect of varying auditory action–effect patterns during complex action learning. In Experiment 1, participants were assigned to 1 of 4 groups, watched a silent video of a hand playing a sequence on a piano keyboard with no auditory action effect (observation) and were asked to practise and perform the sequence on an identical keyboard with varying action effects (reproduction). During reproduction, Group 1 heard no auditory tones (identical to observed video), Group 2 heard typical scale-ascending piano tones with each key press, Group 3 heard fixed but out-of-sequence piano tones with each key press, and Group 4 heard random piano tones with each key press. In Experiment two, new participants were assigned to 1 of 2 groups and watched an identical video; however, the video in this experiment contained typical, scale-ascending piano sounds. During reproduction, Group 1 heard no auditory tones while Group 2 heard typical, scale-ascending piano tones with each key press (identical to observed video). Our results showed that participants whose action–effect patterns during reproduction matched those in the observed video learned the action sequence faster than participants whose action–effect patterns during reproduction differed from those in the observed video. Additionally, our results suggest that adding an effect during reproduction (when one is absent during observation) is somewhat more detrimental to action sequence learning than removing an effect during reproduction (when one is present during observation).     

Stimulus–response correspondence effect as a function of temporal overlap between relevant and irrelevant information processing

The stimulus–response correspondence (SRC) effect refers to advantages in performance when stimulus and response correspond in dimensions or features, even if the common features are irrelevant to the task. Previous research indicated that the SRC effect depends on the temporal course of stimulus information processing. The current study investigated how the temporal overlap between relevant and irrelevant stimulus processing influences the SRC effect. In this experiment, the irrelevant stimulus (a previously associated tone) preceded the relevant stimulus (a coloured rectangle). The irrelevant and relevant stimuli onset asynchrony was varied to manipulate the temporal overlap between the irrelevant and relevant stimuli processing. Results indicated that the SRC effect size varied as a quadratic function of the temporal overlap between the relevant stimulus and irrelevant stimulus. This finding extends previous experimental observations that the SRC effect size varies in an increasing or decreasing function with reaction time. The current study demonstrated a quadratic function between effect size and the temporal overlap.     

Correspondence effects with torches: Grasping affordance or visual feature asymmetry?

Three experiments were conducted to determine whether an object-based correspondence effect for torch (flashlight) stimuli reported by Pellicano et al. [(2010 Pellicano, A., Iani, C., Borghi, A. M., Rubichi, S., & Nicoletti, R. (2010). Simon-like and functional affordance effects with tools: The effects of object perceptual discrimination and object action state. Quarterly Journal of Experimental Psychology, 63, 2190–2201. doi: 10.1080/17470218.2010.486903[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). Simon-like and functional affordance effects with tools: The effects of object perceptual discrimination and object action state. Quarterly Journal of Experimental Psychology, 63, 2190–2201] is due to a grasping affordance provided by the handle or asymmetry of feature markings on the torch. In Experiment 1 the stimuli were the same as those from Pellicano et al.'s Experiment 2, whereas in Experiments 2 and 3 the stimuli were modified versions with the graspable handle removed. Participants in all experiments performed upright/inverted orientation judgements on the torch stimuli. The results of Experiment 1 replicated those of Pellicano et al.: A small but significant object-based correspondence effect was evident, mainly when the torch was in an active state. With the handle of the torch removed in Experiment 2, making the barrel markings more asymmetric in the display, the correspondence effect was larger. Experiment 3 directly demonstrated an effect of barrel-marking asymmetry on the correspondence effect: When only the half of the markings nearest the light end of the torch was included, the correspondence effect reversed to favour the light end. The results are in agreement with a visual feature-asymmetry account and are difficult to reconcile with a grasping-affordance account.     

Effects of visual cue and response assignment on spatial stimulus coding in stimulus–response compatibility

Tlauka and McKenna (2000 Tlauka, M. and McKenna, F. P. 2000. Hierarchical knowledge influences stimulus–response compatibility effects. Quarterly Journal of Experimental Psychology, 53A: 85–103.  [Google Scholar]) reported a reversal of the traditional stimulus–response compatibility (SRC) effect (faster responding to a stimulus presented on the same side than to one on the opposite side) when the stimulus appearing on one side of a display is a member of a superordinate unit that is largely on the opposite side. We investigated the effects of a visual cue that explicitly shows a superordinate unit, and of assignment of multiple stimuli within each superordinate unit to one response, on the SRC effect based on superordinate unit position. Three experiments revealed that stimulus–response assignment is critical, while the visual cue plays a minor role, in eliciting the SRC effect based on the superordinate unit position. Findings suggest bidirectional interaction between perception and action and simultaneous spatial stimulus coding according to multiple frames of reference, with contribution of each coding to the SRC effect flexibly varying with task situations.     

Task-defined associations are mode specific for selection of relevant dimension but mode independent for selection of location mapping

When compatible and incompatible mappings of a location-relevant task are mixed, or a location-relevant task is mixed with a task for which stimulus location is irrelevant, the benefit of the compatible mapping is eliminated for physical locations and enhanced for location words. Two experiments examined the influence of presenting the location information for the mixed conditions in different stimulus modes (physical location or word). Experiment 1 showed that the effects of mixing location-relevant and location-irrelevant tasks on the spatial compatibility and Simon effects are reduced when the location information is presented in different modes for the two tasks. Experiment 2 showed, in contrast, that the mode distinction had little influence on the effects of mixed compatible and incompatible mappings for location-relevant tasks: The compatibility effect was eliminated for physical locations and enhanced for words, as when there is no mode distinction. Thus, when location is relevant for one task and colour for the other, the task-defined associations of locations to responses are mode specific, but when location is relevant for both tasks, the associations are mode independent.     

Motor inhibition associated with the affordance of briefly displayed objects

Research has demonstrated that left- and right-hand responses are facilitated when they are performed with the hand compatible with the orientation of a viewed object. This suggests that graspable objects automatically activate the motor representations that correspond to their orientation. It has recently been proposed that similar positive stimulus–response compatibility effects (PCE) may turn into negative compatibility effects (NCE) when a prime object is displayed very briefly. These NCEs are suggested to reflect motor inhibition mechanisms—motor activation triggered by briefly viewed objects may be treated by the motor system as unwanted, and thus it is rapidly inhibited. We examined whether the motor activation triggered by the orientation of a task-irrelevant object is similarly inhibited when the object is displayed briefly. In Experiment 1, a NCE was observed between the orientation of an object and the responding hand when the object was displayed for 30 or 70 ms. The effect turned into a PCE when the object was displayed for 370 ms. Experiment 2 confirmed that this motor inhibition effect was produced by the handle affordance of the object rather than some abstract visual properties of the object.     

Stimulus–response compatibility for mixed mappings and tasks with unique responses

For two stimulus locations mapped to two keypresses, reaction time is shorter when the mapping is compatible than when it is not (the stimulus–response compatibility, SRC, effect). A similar result, called the Simon effect, occurs when stimulus location is irrelevant, and colour is relevant. When compatibly mapped trials are intermixed with incompatibly mapped trials or Simon task trials, the compatibility effect is eliminated, and the Simon effect is influenced by the location mapping. In five experiments, we examined whether similar mixing effects occur when the two spatial mappings or location-relevant and location-irrelevant tasks use distinct keypresses on the left and right hands. Mixing had considerably less influence on the SRC and Simon effects than it does when the intermixed trial types or tasks share the same responses, even though response time was lengthened to a similar extent. Mixing two tasks for which stimulus location was irrelevant yielded no within-task Simon effect, but the effect was also absent when four stimuli were assigned to two responses on a single hand. The relative lack of influence of mixing on the SRC and Simon effects when the tasks have unique responses implies that suppression of direct activation of the corresponding response occurs primarily when the tasks share responses.     

Stimulus–classification and stimulus–action associations: Effects of repetition learning and durability

It has been shown that acquired stimulus–response bindings result from at least two types of associations from the stimulus to the task (stimulus–task or stimulus–classification; S–C) and from the stimulus to the motor response (stimulus–response or stimulus–action; S–A). These types of associations have been shown to independently affect behaviour. This finding suggests that they are processed in different pathways or different parts of a pathway at the neural level. Here we test a hypothesis that such associations may be differentially affected by repetition learning and that such effects may be detected by measuring their durability against overwriting. We show that both S–C and S–A associations are in fact strengthened when learning is boosted by increasing repetitions of the primes. However, the results further suggest that associations between stimuli and actions have less durable effects on behaviour and that the durability of S–C and S–A associations is independent of repetition learning. This is an important finding for the understanding of the underlying mechanisms of associative learning and particularly raises the question of which processes may affect flexibility of learning.     

Transfer of orthogonal stimulus–response mappings to an orthogonal Simon task

When up–down stimulus locations are mapped to left–right keypresses, an overall advantage for the up–right/down–left mapping is often obtained that varies as a function of response eccentricity. This orthogonal stimulus–response compatibility (SRC) effect also occurs when stimulus location is irrelevant, a phenomenon called the orthogonal Simon effect, and has been attributed to correspondence of stimulus and response code polarities. The Simon effect for horizontal stimulus–response (S–R) arrangements has been shown to be affected by short-term S–R associations established through the mapping used for a prior SRC task in which stimulus location was relevant. We examined whether such associations also transfer between orthogonal SRC and Simon tasks and whether correspondence of code polarities continues to contribute to performance in the Simon task. In Experiment 1, the orthogonal Simon effect was larger after practising with an up–right/down–left mapping of visual stimuli to responses than with the alternative mapping, for which the orthogonal Simon effect tended to reverse. Experiment 2 showed similar results when practice was with high (up) and low (down) pitch tones, though the influence of practice mapping was not as large as that in Experiment 1, implying that the short-term S–R associations acquired in practice are at least in part not modality specific. In Experiment 3, response eccentricity and practice mapping were shown to have separate influences on the orthogonal Simon effect, as expected if both code polarity and acquired S–R associations contribute to performance.     

Flanker and Simon effects interact at the response selection stage

The present study aimed at investigating the processing stage underlying stimulus–stimulus (S–S) congruency effects by examining the relation of a particular type of congruency effect (i.e., the flanker effect) with a stimulus–response (S–R) spatial correspondence effect (i.e., the Simon effect). Experiment 1 used a unilateral flanker task in which the flanker also acted as a Simon-like accessory stimulus. Results showed a significant S–S Congruency × S–R Correspondence interaction: An advantage for flanker–response spatially corresponding trials was observed in target–flanker congruent conditions, whereas, in incongruent conditions, there was a noncorresponding trials' advantage. The analysis of the temporal trend of the correspondence effects ruled out a temporal-overlap account for the observed interaction. Moreover, results of Experiment 2, in which the flanker did not belong to the target set, demonstrated that this interaction cannot be attributed to perceptual grouping of the target–flanker pairs and referential coding of the target with respect to the flanker in the congruent and incongruent conditions, respectively. Taken together, these findings are consistent with a response selection account of congruency effects: Both the position and the task-related attribute of the flanker would activate the associated responses. In noncorresponding-congruent trials and corresponding-incongruent trials, this would cause a conflict at the response selection stage.     

When speech enhances Spatial Musical Association of Response Codes: Joint spatial associations of pitch and timbre in nonmusicians

Previous studies have shown that the effect of the Spatial Musical Association of Response Codes (SMARC) depends on various features, such as task conditions (whether pitch height is implicit or explicit), response dimension (horizontal vs. vertical), presence or absence of a reference tone, and former musical training of the participants. In the present study, we investigated the effects of pitch range and timbre: in particular, how timbre (piano vs. vocal) contributes to the horizontal and vertical SMARC effect in nonmusicians under varied pitch range conditions. Nonmusicians performed a timbre judgement task in which the pitch range was either small (6 or 8 semitone steps) or large (9 or 12 semitone steps) in a horizontal and a vertical response setting. For piano sounds, SMARC effects were observed in all conditions. For the vocal sounds, in contrast, SMARC effects depended on pitch range. We concluded that the occurrence of the SMARC effect, especially in horizontal response settings, depends on the interaction of the timbre (vocal and piano) and pitch range if vocal and instrumental sounds are combined in one experiment: the human voice enhances the attention, both to the vocal and the instrumental sounds.     

Response-specifying cue for action interferes with perception of feature-sharing stimuli

Perceiving a visual stimulus is more difficult when a to-be-executed action is compatible with that stimulus, which is known as blindness to response-compatible stimuli. The present study explored how the factors constituting the action event (i.e., response-specifying cue, response intention, and response feature) affect the occurrence of this blindness effect. The response-specifying cue varied along the horizontal and vertical dimensions, while the response buttons were arranged diagonally. Participants responded based on one dimension randomly determined in a trial-by-trial manner. The response intention varied along a single dimension, whereas the response location and the response-specifying cue varied within both vertical and horizontal dimensions simultaneously. Moreover, the compatibility between the visual stimulus and the response location and the compatibility between that stimulus and the response-specifying cue was separately determined. The blindness effect emerged exclusively based on the feature correspondence between the response-specifying cue of the action task and the visual target of the perceptual task. The size of this stimulus–stimulus (S–S) blindness effect did not differ significantly across conditions, showing no effect of response intention and response location. This finding emphasizes the effect of stimulus factors, rather than response factors, of the action event as a source of the blindness to response-compatible stimuli.     

SNARC (spatial–numerical association of response codes) meets SPARC (spatial–pitch association of response codes): Automaticity and interdependency in compatibility effects

Concepts, including the mental number line, or addressing pitch as high and low, suggest that the spatial–numerical and spatial–pitch association of response codes (SNARC and SPARC) effects are domain-specific and thus independent. Alternatively, there may be dependencies between these effects, because they share common automatic or controlled decision mechanisms. In two experiments, participants were presented with spoken numbers in different pitches; their numerical value, pitch, and response compatibility were varied systematically. This allowed us to study SNARC and SPARC effects in a factorial design (see also Fischer, Riello, Giordano, &; Rusconi, 2013 Fischer, M. H., Riello, M., Giordano, B. L., &; Rusconi, E. (2013). Singing numbers?…?in cognitive space--a dual-task study of the link between pitch, space, and numbers. Topics in Cognitive Science, 5(2), 354–366. Retrieved from http://doi.org/10.1111/tops.12017[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]). Participants judged the stimuli on numerical magnitude, pitch, or parity (odd–even). In all tasks, the SNARC and SPARC effects had superadditive interactions. These were interpreted as both effects sharing a common mechanism. The task variation probes the mechanism: In the magnitude judgement task, numerical magnitude was explicit, whereas pitch was implicit; in the pitch judgement task, it was vice versa. In the parity judgement task, both dimensions were implicit. Regardless of whether they were implicit or explicit, both SNARC and SPARC effects occurred in all tasks. We concluded that by not requiring focal attention the common mechanism operates automatically.     

Automatic imitation of intransitive actions

Previous research has indicated a potential discontinuity between monkey and human ventral premotor-parietal mirror systems, namely that monkey mirror systems process only transitive (object-directed) actions, whereas human mirror systems may also process intransitive (non-object-directed) actions. The present study investigated this discontinuity by seeking evidence of automatic imitation of intransitive actions--hand opening and closing--in humans using a simple reaction time (RT), stimulus-response compatibility paradigm. Left-right and up-down spatial compatibility were controlled by ensuring that stimuli were presented and responses executed in orthogonal planes, and automatic imitation was isolated from simple and complex orthogonal spatial compatibility by varying the anatomical identity of the stimulus hand and response hemispace, respectively. In all conditions, action compatible responding was faster than action incompatible responding, and no effects of spatial compatibility were observed. This experiment therefore provides evidence of automatic imitation of intransitive actions, and support for the hypothesis that human and monkey mirror systems differ with respect to the processing of intransitive actions.     

Conflict in object affordance revealed by grip force

Viewing objects can result in automatic, partial activation of motor plans associated with them—“object affordance”. Here, we recorded grip force simultaneously from both hands in an object affordance task to investigate the effects of conflict between coactivated responses. Participants classified pictures of objects by squeezing force transducers with their left or right hand. Responses were faster on trials where the object afforded an action with the same hand that was required to make the response (congruent trials) compared to the opposite hand (incongruent trials). In addition, conflict between coactivated responses was reduced if it was experienced on the preceding trial, just like Gratton adaptation effects reported in “conflict” tasks (e.g., Eriksen flanker). This finding suggests that object affordance demonstrates conflict effects similar to those shown in other stimulus–response mapping tasks and thus could be integrated into the wider conceptual framework on overlearnt stimulus–response associations. Corrected erroneous responses occurred more frequently when there was conflict between the afforded response and the response required by the task, providing direct evidence that viewing an object activates motor plans appropriate for interacting with that object. Recording continuous grip force, as here, provides a sensitive way to measure coactivated responses in affordance tasks.