Scheduling and programming of rapid finger sequences: tests and elaborations of the hierarchical editor model

Is a response sequence executed only after the sequence has been fully programmed, as discrete processing models predict, or does execution begin before programming has been completed, as continuous processing models predict? To address this issue, we tested a discrete processing model of human motor performance, the hierarchical editor model of Rosenbaum, Inhoff, and Gordon (1984). This model was developed to account for data from experiments in which people perform one of two possible finger sequences, depending on the identity of a choice signal. The model assumes a hierarchically organized motor program that is first "edited" to resolve any uncertainties and is then "executed" to produce the desired responses. Three experiments reported here show that, contrary to the model's predictions and some well-known motor programming results (Sternberg, Monsell, Knoll, & Wright, 1978), the reaction time to begin a response sequence actually decreases with the length of the sequence under some choice conditions. We account for these results with a model that allows execution to begin while editing is still in progress. A key assumption in the model is that subjects schedule execution so that means and variances of interresponse times are minimized.     

Nonhierarchical control of rapid movement sequences: a comment on Rosenbaum, Kenny, and Derr

In a recent article, Rosenbaum, Kenny, and Derr (1983) described a hierarchical storage and execution model for a class of repetitive, discrete response sequences. With a few modifications, this model can match the performance of subjects performing sequences from this class. The authors claimed that this provides an "existence proof" for hierarchical control during movement execution, at least for these sequences. My purpose is to show by counterexample that this claim is too strong. I present a logogen activation model for the rapid execution of stored motor sequences which assumes that (a) logogens corresponding to responses are activated via association and repetition; (b) activation decays; and (c) interresponse time is inversely related to activation of the correct response at each position in the sequence. This model can also fit the results of Rosenbaum et al. A much richer data base, designed to discriminate between competing formulations, will be needed to prove the existence of the hierarchical, tree-traversal control process proposed by Rosenbaum et al.     

The problem of serial order in behavior: Lashley's legacy

In a prescient paper Karl Lashley (1951) rejected reflex chaining accounts of the sequencing of behavior and argued instead for a more cognitive account in which behavioral sequences are typically controlled with central plans. An important feature of such plans, according to Lashley, is that they are hierarchical. Lashley offered several sources of evidence for the hierarchical organization for behavioral plans, and others afterward provided more evidence for this hypothesis. We briefly review that evidence here and then shift from a focus on the structure of plans (Lashley's point of concentration) to the processes by which plans are formed in real time. Two principles emerge from the studies we review. One is that plans are not formed from scratch for each successive movement sequence but instead are formed by making whatever changes are needed to distinguish the movement sequence to be performed next from the movement sequence that has just been performed. This plan-modification view is supported by two phenomena discovered in our laboratory: the parameter remapping effect, and the handpath priming effect. The other principle we review is that even single movements appear to be controlled with hierarchically organized plans. At the top level are the starting and goal postures. At the lower level are the intermediate states comprising the transition from the starting posture to the goal posture. The latter principle is supported by another phenomenon discovered in our lab, the end-state comfort effect, and by a computational model of motor planning which accounts for a large number of motor phenomena. Interestingly, the computational model hearkens back to a classical method of generating cartoon animations that relies on the production of keyframes first and the production of interframes (intermediate frames) second.     

On choosing between movement sequences: comments on Rose (1988)

One method of investigating human motor programming is to determine how the choice reaction time for a memorized response sequence depends on the composition of that sequence as well as the other sequence that may be required. Using this method, Rose (1988) found that the total number of responses in the two possible response sequences predicts the choice reaction time to initiate either one. On the basis of this result, Rose claimed that the hierarchical editor (HED) model of motor programming, developed by Rosenbaum, Inhoff, and Gordon (1984), may have to be reevaluated. In this commentary I argue that Rose's results are inconsistent with a precursor of the HED model, not with the HED model itself, that the HED model actually provides a better fit to Rose's data than her total-number-of-responses model, that in general, choice reaction time does not increase with the total number of possible responses, and that structural relations between alternative movement sequences are the main determinants of choice reaction time. Taken as a whole, the results suggest that possible responses are not held in completely readied form before being selected for execution. A further implication is that the storage capacity of the motor output buffer (the MOB) is extremely limited.     

Decreased load on general motor preparation and visual-working memory while preparing familiar as compared to unfamiliar movement sequences

Learning movement sequences is thought to develop from an initial controlled attentive phase to a more automatic inattentive phase. Furthermore, execution of sequences becomes faster with practice, which may result from changes at a general motor processing level rather than at an effector specific motor processing level. In the current study, we examined whether these changes are already present during preparation. Fixed series of six keypresses, either familiar or unfamiliar, had to be prepared and executed/withheld after a go/nogo signal. Reaction time results confirmed that familiar sequences were executed faster than unfamiliar sequences. Results derived from the electroencephalogram showed a decreased demand on general motor preparation and visual-working memory before familiar sequences as compared to unfamiliar sequences. We propose that with familiar sequences the presetting segments of responses is less demanding than with unfamiliar sequences, as familiar sequences can be regarded as less complex than unfamiliar sequences. Finally, the decreasing demand on visual-working memory before familiar sequences suggests that sequence learning indeed develops from an attentive to an automatic phase.     

The programming of structural properties of movement sequences

Summary The present paper investigates the role of abstract structural properties in the programming and execution of movement sequences. Three experiments, using converging methods, demonstrate that the motor system represents the abstract structural properties of movement sequences. The first two experiments show that hierarchical structures over a sequence of tapping movements can be used to prepare the motor program, even if the specific elements of the sequence are still unknown. Experiment 2 also shows that the preliminary programming of structural properties of a movement sequence takes more time than the programming of specific elements ( start elements). Experiment 3 suggests that abstract structural properties can be generalized from a special sequence and that they are transferable to other sequences. Abstract structural properties are assumed to be an important component of generalized motor programs.     

What determines the impact of context on sequential action?

In the current study we build on earlier observations that memory-based sequential action is better in the original learning context than in other contexts. We examined whether changes in the perceptual context have differential impact across distinct processing phases (preparation versus execution of a motor chunk) within an ongoing movement sequence. Participants were trained on two discrete keying sequences, each of which was systematically presented in its own unique color during a practice session with either limited or extended practice. In a subsequent test session, sequences were performed with the same, with reversed, and with completely novel sequence-specific colors. The results confirm context-dependence in sequential action, the relevance of practice for its development, and its selective expression for the preparation but not the execution of highly practiced motor chunks. As such, the current study provides novel insights into the determinants of context-dependent sequential action. We finish by outlining the overall status of context-dependence in sequential motor behavior, and specify a general working model.     

Central and peripheral coordination in movement sequences

Summary Motor coordination has been too poorly defined to be a useful construct in studying the control of movement. In general, motor coordination involves controlling both the timing and the kinematics of movement. Yet the motor behaviors typically used for the study of coordination have required controlling only the timing or the spatial aspects of a movement. To understand better the basis of motor behavior, this study examined movement sequences, a class of movement in which both the timing and the kinematics must be controlled. In one experiment we studied a reaching and grasping movement sequence to characterize the central coordination of movement sequences. In another experiment we studied a throwing movement sequence to characterize the peripheral (kinesthetic) coordination of movement sequences. An heuristic model is presented to explain how central and peripheral mechanisms of coordination might interact to produce accurate movement.     

NEURONAL CODING OF SERIAL ORDER:

Abstract— How does the brain create rule-governed sequences of behavior? An answer to this question may come from a surprising source: the neostriatum (caudate nucleus and putamen), Traditionally, the neostriatum has been considered pun of the brain's motor system, but its contribution to the preparation or execution of movement is recognized generally to concern high-level motor functions. Recent work implicates the neostriatum in disorders of sequential action and thought, as in the repetition of thoughts or habits in human obsessive-compulsive disorder and movements or speech in Tourette's syndrome. Yet there is no direct evidence to support the idea that the neostriatum controls sequences of behavior. Using ethological and neurophysiological techniques to study neural activity in the rat neoslriatum during syntactic grooming sequences, we found that neuronal activity in the anterolateral neostriatum depended on the execution of syntactic sequences of grooming actions. The individual grooming movements themselves did not activate the neoslriatum; activation was determined by the syntactic sequence in which grooming movements were performed. These data provide the first direct evidence that the neoslriatum coordinates the control of rule-governed behavioral sequences.     

Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective

Research on human and animal behavior has long emphasized its hierarchical structure—the divisibility of ongoing behavior into discrete tasks, which are comprised of subtask sequences, which in turn are built of simple actions. The hierarchical structure of behavior has also been of enduring interest within neuroscience, where it has been widely considered to reflect prefrontal cortical functions. In this paper, we reexamine behavioral hierarchy and its neural substrates from the point of view of recent developments in computational reinforcement learning. Specifically, we consider a set of approaches known collectively as hierarchical reinforcement learning, which extend the reinforcement learning paradigm by allowing the learning agent to aggregate actions into reusable subroutines or skills. A close look at the components of hierarchical reinforcement learning suggests how they might map onto neural structures, in particular regions within the dorsolateral and orbital prefrontal cortex. It also suggests specific ways in which hierarchical reinforcement learning might provide a complement to existing psychological models of hierarchically structured behavior. A particularly important question that hierarchical reinforcement learning brings to the fore is that of how learning identifies new action routines that are likely to provide useful building blocks in solving a wide range of future problems. Here and at many other points, hierarchical reinforcement learning offers an appealing framework for investigating the computational and neural underpinnings of hierarchically structured behavior.     

Aiming at a far target under different viewing conditions: visual control in basketball jump shooting

Most research on visual search in aiming at far targets assumes preprogrammed motor control implying that relevant visual information is detected prior to the final shooting or throwing movements. Eye movement data indirectly support this claim for stationary tasks. Using the basketball jump shot as experimental task we investigated whether in dynamic tasks in which the target can be seen until ball release, continuous, instead of preprogrammed, motor control is possible. We tested this with the temporal occlusion paradigm: 10 expert shooters took shots under four viewing conditions, namely, no vision, full vision, early vision (vision occluded during the final +/-350 ms before ball release), and late vision (vision occluded until these final +/-350 ms). Late-vision shooting appeared to be as good as shooting with full vision while early-vision performance was severely impaired. The results imply that the final shooting movements were controlled by continuous detection and use of visual information until ball release. The data further suggest that visual and movement control of aiming at a far target develop in close correspondence with the style of execution.     

Inhibition in motor imagery: a novel action mode switching paradigm

Motor imagery requires that actual movements are prevented (i.e., inhibited) from execution. To investigate at what level inhibition takes place in motor imagery, we developed a novel action mode switching paradigm. Participants imagined (indicating only start and end) and executed movements from start buttons to target buttons, and we analyzed trial sequence effects. Trial sequences depended on current action mode (imagination or execution), previous action mode (pure blocks/same mode, mixed blocks/same mode, or mixed blocks/other mode), and movement sequence (action repetition, hand repetition, or hand alternation). Results provided evidence for global inhibition (indicated by switch benefits in execution-imagination (E-I)-sequences in comparison to I-I-sequences), effector-specific inhibition (indicated by hand repetition costs after an imagination trial), and target inhibition (indicated by target repetition benefits in I-I-sequences). No evidence for subthreshold motor activation or action-specific inhibition (inhibition of the movement of an effector to a specific target) was obtained. Two (global inhibition and effector-specific inhibition) of the three observed mechanisms are active inhibition mechanisms. In conclusion, motor imagery is not simply a weaker form of execution, which often is implied in views focusing on similarities between imagination and execution.     

Learning a keying sequence you never executed: Evidence for independent associative and motor chunk learning

A substantial amount of research has addressed how people learn and control movement sequences. Recent results suggested that practice with discrete key pressing sequences results in two types of sequence learning: associative learning and motor chunk development (Verwey & Abrahamse, 2012). In the present study, we addressed whether in keying sequences of limited length associative learning develops also when the use of the chunking mode is prevented by introducing during practice random deviants. In line with the notion of two different learning mechanisms, the present results indicate that associative sequence learning develops when motor chunks cannot be developed during practice. This confirms the notion that motor chunks do not rely on these associations. In addition, experience with a particular execution mode during the practice phase seems to benefit subsequent use of that mode with unfamiliar and random sequences. Also, participants with substantial video-gaming experience were faster in executing discrete keying sequences in the chunking mode. These last two results may point to the development of a general ability to produce movement sequences in the chunking mode.     

Segmentation and coupling in complex movements

In two experiments we explore the structure of complex sequences of drawing movements. We find that in these movements a single parameter--the velocity gain factor--relates the geometrical and kinematic aspects of the movement trajectory via a two-thirds power law. In Experiment 1 we investigate the relation between the velocity gain factor and the linear extent of the trajectory. In Experiment 2 we demonstrate that the gain factor provides a criterion for segmenting the movement into distinct units of motor action, and we investigate the effects of the speed of execution on this segmentation. A theoretical analysis shows that the results of both Experiments 1 and 2 can be given a unitary interpretation by assuming a coupling function of variable strength between segments. The general problem of representing motor programs is discussed within this theoretical framework.     

On the importance of avoiding shortcuts in applying cognitive models to hierarchical data

Psychological experiments often yield data that are hierarchically structured. A number of popular shortcut strategies in cognitive modeling do not properly accommodate this structure and can result in biased conclusions. To gauge the severity of these biases, we conducted a simulation study for a two-group experiment. We first considered a modeling strategy that ignores the hierarchical data structure. In line with theoretical results, our simulations showed that Bayesian and frequentist methods that rely on this strategy are biased towards the null hypothesis. Secondly, we considered a modeling strategy that takes a two-step approach by first obtaining participant-level estimates from a hierarchical cognitive model and subsequently using these estimates in a follow-up statistical test. Methods that rely on this strategy are biased towards the alternative hypothesis. Only hierarchical models of the multilevel data lead to correct conclusions. Our results are particularly relevant for the use of hierarchical Bayesian parameter estimates in cognitive modeling.     

Tactile suppression in goal-directed movement

Sharing numerous characteristics with suppression in the other senses, tactile suppression is a reliable phenomenon that accompanies movement. By investigating the simplest of movements (e.g., finger flexions), early research tried to explain the origins of the phenomenon in terms of motor command generation together with sensory reafference. Here, we review recent research that has delved into (naturalistic) goal-directed movements. In connection with goal-directed movement, tactile suppression is evident as a decrease in behavioural performance measured shortly prior to, and during, movement execution. It is also reflected in a consistent response bias highlighting the (perceptual) uncertainty of the movement. Goal-directed movement supports the forward model and establishes contextual influences as the defining influences on tactile suppression. Depending on the task at hand, people prioritize a certain percept during movement. Future research, we argue, should focus on studying naturalistic movements, or sequences of movements, that share a common meaning or goal.     

Working memory for patterned sequences of auditory objects in a songbird

The capacity to remember sequences is critical to many behaviors, such as navigation and communication. Adult humans readily recall the serial order of auditory items, and this ability is commonly understood to support, in part, the speech processing for language comprehension. Theories of short-term serial recall posit either use of absolute (hierarchically structured) or relative (associatively structured) position information. To date, neither of these classes of theories has been tested in a comparative auditory model. European starlings, a species of songbird, use temporally structured acoustic signals to communicate, and thus have the potential to serve as a model system for auditory working memory. Here, we explore the strategies that starlings use to detect the serial order of ecologically valid acoustic communication signals and the limits on their capacities to do so. Using a two-alternative choice operant procedure, we demonstrate that starlings can attend to the serial ordering of at least four song elements (motifs) and can use this information to classify differently ordered sequences of motifs. Removing absolute position cues from sequences while leaving relative position cues intact, causes recognition to fail. We then show that starlings can, however, recognize motif-sequences using only relative position cues, but only under rigid circumstances. The data are consistent with a strong learning bias against relative position information, and suggest that recognition of structured vocal signals in this species is inherently hierarchical.     

Does motor programming necessitate response execution?

The complexity of a movement is known to affect the time it takes to initiate the movement. This effect is thought to reflect changes in the duration of processes that operate on a motor program. This question addressed here is whether programming a movement compels the start of its overt execution. If it does, then the programming processes may be said to occur after the "point of no return." We report results from an empirical procedure and a theoretical analysis designed to study processes before and after this point separately. According to our results, changes in the complexity of a movement affect only the prior set of processes. From this we argue that motor programming does not necessitate response execution and that the point of no return occurs very late in the information-processing system.     

Overcoming the acting/reasoning dualism in intelligent behavior

In a paper that recently appeared in this journal, we proposed a model that aims at providing a comprehensive account of our ability to intelligently use tools, bridging sensorimotor and reasoning-based explanations of this ability. Central to our model is the notion of generalized motor programs for tool use, which we defined as a synthesis between classic motor programs, as described in the scientific literature, and Peircean habits. In his commentary, Osiurak proposes a critique of the notion of generalized motor program, and suggests that the limitations of our model can be solved by integrating it with the view that motor programs are generated by a previous mechanical reasoning, independent from sensorimotor knowledge. Here we reply that while on the one hand our reference to Peircean habits gets over the temptation to consider motor programs as fixed internal entities, it also rejects the view, endorsed by Osiurak, that intelligent practice is a mixture of antecedent abstract reasoning and subsequent motor execution.     

Forgetting motor programmes: Retrieval dynamics in procedural memory

When motor sequences are stored in memory in a categorised manner, selective retrieval of some sequences can induce forgetting of the non-retrieved sequences. We show that such retrieval-induced forgetting (RIF) occurs not only in cued recall but also in a test assessing memory indirectly by providing novel test cues without involving recall of items. Participants learned several sequential finger movements (SFMs), each consisting of the movement of two fingers of either the left or the right hand. Subsequently, they performed retrieval practice on half of the sequences of one hand. A final task then required participants to enter letter dyads. A subset of these dyads corresponded to the previously learned sequences. RIF was present in the response times during the entering of the dyads. The finding of RIF in the slowed-down execution of motor programmes overlapping with initially trained motor sequences suggests that inhibition resolved interference between procedural representations of the acquired motor sequences of one hand during retrieval practice.