Eye movements during transitive action observation have sequential structure

In this paper, we explore how observers of a reach-to-grasp action can identify and distinguish between the agent and patient (i.e. target) of the action. We investigate the hypothesis that there is a characteristic sequential structure to the observer’s pattern of saccades, with the agent being fixated first, and then the target. We report an experiment which indicates that this sequence of saccades, while not ubiquitous, is overwhelmingly more likely than chance. The experiment also sheds some light on the mechanisms which allow the observer to saccade from the agent of the action to the target.     

A computational model of event segmentation from perceptual prediction

People tend to perceive ongoing continuous activity as series of discrete events. This partitioning of continuous activity may occur, in part, because events correspond to dynamic patterns that have recurred across different contexts. Recurring patterns may lead to reliable sequential dependencies in observers' experiences, which then can be used to guide perception. The current set of simulations investigated whether this statistical structure within events can be used 1) to develop stable internal representations that facilitate perception and 2) to learn when to update such representations in a self-organizing manner. These simulations demonstrate that experience with recurring patterns enables a system to accurately predict upcoming stimuli within an event, to identify boundaries between such events based on transient increases in prediction error, and to use such boundaries to improve prediction about subsequent activities.     

Implicit learning of predictive relationships in three-element visual sequences by young and old adults

Knowledge of sequential relationships enables future events to be anticipated and processed efficiently. Research with the serial reaction time task (SRTT) has shown that sequence learning often occurs implicitly without effort or awareness. Here, the authors report 4 experiments that use a triplet-learning task (TLT) to investigate sequence learning in young and older adults. In the TLT, people respond only to the last target event in a series of discrete, 3-event sequences or triplets. Target predictability is manipulated by varying the triplet frequency (joint probability) and/or the statistical relationships (conditional probabilities) among events within the triplets. Results reveal that both groups learned, though older adults showed less learning of both joint and conditional probabilities. Young people used the statistical information in both cues, but older adults relied primarily on information in the 2nd cue alone. The authors conclude that the TLT complements and extends the SRTT and other tasks by offering flexibility in the kinds of sequential statistical regularities that may be studied as well as by controlling event timing and eliminating motor response sequencing.     

The basis of transfer in artificial grammar learning

In two experiments, we examined the extent to which knowledge of sequential dependencies and/or patterns of repeating elements is used during transfer in artificial grammar learning. According to one view of transfer, learners abstract the grammar's sequential dependencies and then learn a mapping to new vocabulary at test (Dienes, Altmann, & Gao, 1999). Elements that are repeated have no special status on this view, and so a logical prediction is that learners should transfer as well after exposure to a grammar without repetitions as after exposure to a grammar with them. On another view, repetition structure is the very basis of transfer (Brooks & Vokey, 1991; Mathews & Roussel, 1997). Learners were trained on grammars with or without repeating elements to test these competing views. Learners demonstrated considerable knowledge of sequential dependencies in their training vocabulary but did not use such knowledge to transfer to a new vocabulary. Transfer only occurred in the presence of repetition structure, demonstrating this to be the basis of transfer.     

Non‐adjacent Dependencies Processing in Human and Non‐human Primates

Human and non‐human primates share the ability to extract adjacent dependencies and, under certain conditions, non‐adjacent dependencies (i.e., predictive relationships between elements that are separated by one or several intervening elements in a sequence). In this study, we explore the online extraction dynamics of non‐adjacent dependencies in humans and baboons using a serial reaction time task. Participants had to produce three‐target sequences containing deterministic relationships between the first and last target locations. In Experiment 1, participants from the two species could extract these non‐adjacent dependencies, but humans required less exposure than baboons. In Experiment 2, the data show for the first time in a non‐human primate species the successful generalization of sequential non‐adjacent dependencies over novel intervening items. These findings provide new evidence to further constrain current theories about the nature and the evolutionary origins of the learning mechanisms allowing the extraction of non‐adjacent dependencies.     

Cognitive factors affecting sequential dependencies in differential eyelid conditioning

Sequential dependencies were examined as a function of learning, masking, and subject factors in a two-stage differential eyelid conditioning experiment. In Stage 1, all subjects were differentially conditioned to two Vanderplas and Garvin (1959) forms. In Stage 2, the conditional stimuli (CSs) were one of eight words, four reinforced (CS+) and four unreinforced (CS?), with taxonomic category (animalsvs. musical instruments) as the discriminandum, and different subjects were given semantic, physical, or no additional CS processing tasks. The results showed sizeable and highly significant sequential dependencies in the form of greater conditional response probabilities when CS + as opposed to CS? trials were recently experienced. These effects decreased overtrials, were greater in the groups given additional processing loads, and were in some cases greater in C-form responders than in V-form responders and greater in subjects who were unable to verbalize the differential contingencies. It was concluded that the magnitude of sequential dependencies was inversely related to the subject’s awareness of the differential contingencies and/or ability to use the contingency information to respond appropriately to the CS cues.     

Perception of randomness and predicting uncertain events

Using randomly generated sequences of binary events we asked participants to make predictions about the next event. It turned out that while predicting uncertain events, people do not behave unsystematically. Our research identifies four types of relatively consistent strategies for predicting uncertain binary events: a strategy immune to short-run sequential dependencies consisting of the persistent prediction of long-run majority events, hereafter called the long-run momentum strategy; a strategy immune to short-run sequential dependencies consisting of the persistent prediction of long-run minority events, called the long-run contrarian strategy; a strategy sensitive to short-run sequential dependencies consisting of the prediction of short-run majority events, called the short-run momentum strategy; and a strategy sensitive to short-run sequential dependencies consisting of the prediction of short-run minority events, called the short-run contrarian strategy. When the character of events remains unknown, the most common strategy is the short-run momentum strategy. With the increase of a perceived randomness of the situation, people tend more often to use the short-run contrarian strategy. People differ in their general beliefs about the continuation or reversal of a trend in various natural and social processes. Trend believers, when facing sequences of binary events commonly perceived as random, tend to use momentum strategies, whereas those who believe in the trend's reversal tend to use contrarian strategies.     

Temporal contextual cuing of visual attention

Previous research has shown how spatial attention is guided to a target location, but little is understood about how attention is allocated to an event in time. The authors introduce a paradigm to manipulate the sequential structure of visual events independent of responses. They asked whether this temporal context could be implicitly learned and used to guide attention to a relative point in time or location, or both, in space. Experiments show that sequentially structured event durations, event identities, and spatiotemporal event sequences can guide attention to a point in time as well as to a target event's identity and location. Cuing was found to rely heavily on the element immediately preceding the target, although cuing from earlier items also was evident. Learning was implicit in all cases. These results show that the sequential structure of the visual world plays an important role in guiding visual attention to target events.     

Shifts of attention in the identification and discrimination of intensity

Subjects’ ability to resolve stimuli varying in loudness is studied by manipulating task requirements and sequential dependencies in signal presentations. In the first experiment, subjects are required both to identify and to discriminate the same set of 11 randomly presented signals. On the one hand, resolution performance is virtually unchanged under the differing task instructions. On the other hand, large effects on sensitivity are obtained in Experiments 1, 2, and 3 in both identification and discrimination tasks when sequential dependencies are manipulated. The results are interpreted in terms of a shifting attention band in the intensity dimension, as suggested by Luce, Green, and Weber (1976). Preliminary evidence that subjects can voluntarily focus the attention band on a given region of the intensity continuum is obtained.     

Identification of sequential dependencies in conversations: A Pascal program

A Pascal program that provides an analysis of the sequential dependencies between conversational events (e.g., speech acts) generated by two sources is described It produces z scores, chirsquare values, and confidence limits for the difference between estimated unconditional probabilities of the occurrence of selected events (valid events) and estimated conditional probabilities of these events following specified target events immediately (at Lag 1) and, if required, more distantly (at Lags 2, 3, etc.). Valid events and target events may be independently selected from the set of events generated by the first source, the second source, or both. Events not selected are ignored. The user may also elect to exclude the target event from the set of valid events (an adjustment for autocorrelation). The program is especially appropriate for conversational events because it does not assume that two events from the same category can never follow one another.     

Implicit Transfer of Reversed Temporal Structure in Visuomotor Sequence Learning

Some spatio‐temporal structures are easier to transfer implicitly in sequential learning. In this study, we investigated whether the consistent reversal of triads of learned components would support the implicit transfer of their temporal structure in visuomotor sequence learning. A triad comprised three sequential button presses ([1][2][3]) and seven consecutive triads comprised a sequence. Participants learned sequences by trial and error, until they could complete it 20 times without error. Then, they learned another sequence, in which each triad was reversed ([3][2][1]), partially reversed ([2][1][3]), or switched so as not to overlap with the other conditions ([2][3][1] or [3][1][2]). Even when the participants did not notice the alternation rule, the consistent reversal of the temporal structure of each triad led to better implicit transfer; this was confirmed in a subsequent experiment. These results suggest that the implicit transfer of the temporal structure of a learned sequence can be influenced by both the structure and consistency of the change.     

Sequence Memory in Music Performance

Abstract— How do people remember and produce complex sequences like music or speech? Music provides an example of excellent sequence memory under fast performance conditions; novices as well as skilled musicians can perform memorized music rapidly, without making mistakes. In addition, musical pitches repeat often within a melodic sequence in different orders, yet people do not confuse the sequential ordering; temporal properties of musical pitches aid sequence memory. I describe a contextual model of sequence memory that is sensitive to the rate at which musical sequences are produced and to individual differences among performers. Age and musical experience differentiate adults' and children's memory for musical sequences during performance. Performers' memory for the sequential structure of one melody transfers or generalizes to other melodies in terms of the sequence of pitch events, their temporal properties, and their movements. Motion-analysis techniques provide further views of the time course of the cognitive processes that make sequence memory for music so accurate.     

From Blickets to Synapses: Inferring Temporal Causal Networks by Observation

How do human infants learn the causal dependencies between events? Evidence suggests that this remarkable feat can be achieved by observation of only a handful of examples. Many computational models have been produced to explain how infants perform causal inference without explicit teaching about statistics or the scientific method. Here, we propose a spiking neuronal network implementation that can be entrained to form a dynamical model of the temporal and causal relationships between events that it observes. The network uses spike‐time dependent plasticity, long‐term depression, and heterosynaptic competition rules to implement Rescorla–Wagner‐like learning. Transmission delays between neurons allow the network to learn a forward model of the temporal relationships between events. Within this framework, biologically realistic synaptic plasticity rules account for well‐known behavioral data regarding cognitive causal assumptions such as backwards blocking and screening‐off. These models can then be run as emulators for state inference. Furthermore, this mechanism is capable of copying synaptic connectivity patterns between neuronal networks by observing the spontaneous spike activity from the neuronal circuit that is to be copied, and it thereby provides a powerful method for transmission of circuit functionality between brain regions.     

Serial reaction time learning in preschool- and school-age children

Visuomotor sequence learning was assessed in 4- to 10-year-old children using a serial reaction time (SRT) task with both random and sequenced trials. One-half of the children received exposure to the sequence prior to performing the reaction time (RT) task. In Experiment 1, 7- and 10-year-old children demonstrated sequence-specific decreases in RT. As in the adult SRT literature, participants with explicit awareness of the sequence at the end of the session showed larger sequence-specific reaction time decrements than those without explicit awareness. Contrary to expectation, preexposure to the sequence did not reliably predict the level of awareness attained. Results from Experiment 2 indicate that 4-year-olds also demonstrate significant sequence learning on a variant of the SRT task. This article provides preliminary data regarding developmental changes in sequential learning and the development and use of implicit and explicit knowledge. Age-related differences emerged primarily in explicit rather than implicit knowledge.     

Implicit learning

Implicit learning appears to be a fundamental and ubiquitous process in cognition. Although defining and operationalizingimplicit learning remains a central theoretical challenge, scientists' understanding of implicit learning has progressed significantly. Beyond establishing the existence of "learning without awareness," current research seeks to identify the cognitive processes that support implicit learning and addresses the relationship between learning and awareness of what was learned. The emerging view of implicit learning emphasizes the role of associative learning mechanisms that exploit statistical dependencies in the environment in order to generate highly specific knowledge representations.     

Pure feedback effects in absolute identification

To reveal the pure effects of trial-by-trial feedback on judgmental accuracy and sequential dependencies independent of global anchoring effects and other influences, we presented subjects with sequences consisting alternately (within an experimental session) of short runs of trials with feedback (feedback sequences) and without feedback (no-feedback sequences). In Experiments 1 and 2 (absolute identification of sound intensity and sound frequency, respectively), judgmental accuracy was the same in the feedback and the no-feedback sequences, contrary to previous results. Also, in the feedback sequences, the dependency of the current response on the immediately preceding stimulus was larger than that in the no-feedback sequences, while the dependency on the previous response was larger in the no-feedback sequences. In Experiment 3 (absolute identification of sound frequency), we attempted to separate the effects of the number of response categories on sequential dependencies from the effects of the number of stimuli. The results showed that the number of response categories had a larger effect than the number of stimuli on most aspects of performance, but that both affected sequential dependencies. These results are generally consistent with a theory of absolute identification in which feedback affects judgmental accuracy by improving long-term memory for judgmental anchors, while feedback affects sequential dependencies by altering response biases.     

The effect of interstimulus interval on sequential effects in absolute identification

In absolute identification experiments, the participant is asked to identify stimuli drawn from a small set of items which differ on a single physical dimension (e.g., 10 tones which vary in frequency). Responses in these tasks show a striking pattern of sequential dependencies: The current response assimilates towards the immediately preceding stimulus but contrasts with the stimuli further back in the sequence. This pattern has been variously interpreted as resulting from confusion of items in memory, shifts in response criteria, or the action of selective attention, and these interpretations have been incorporated into competing formal models of absolute identification performance. In two experiments, we demonstrate that lengthening the time between trials increases contrast to both the previous stimulus and the stimulus two trials back. This surprising pattern of results is difficult to reconcile with the idea that sequential dependencies result from memory confusion or from criterion shifts, but is consistent with an account that emphasizes selective attention.     

ACT-PRO action protocol analyzer: A tool for analyzing discrete action protocols

This article presents a top-down approach for analyzing sequential events in behavioral data. Analysis of behavioral sequential data often entails identifying patterns specified by the researchers. Algorithms were developed and applied to analyze a kind of behavioral data, calleddiscrete action protocol data. Discrete action protocols consist of discrete user actions, such as mouse clicks and keypresses. Unfortunately, the process of analyzing the huge volume of actions (typically, > 10⁵) is very labor intensive. To facilitate this process, we developed an action protocol analyzer (ACT-PRO) that provides two levels of pattern matching. Level one uses formal grammars to identify sequential patterns. Level two matches these patterns to a hierarchical structure. ACT-PRO can be used to determine how well data fit the patterns specified by an experimenter. Complementarily, it can be used to focus an experimenter’s attention on data that do not fit the prespecified patterns.     

Comparing the effects of implicit and explicit temporal expectation on choice response time and response conflict

People can use temporally structured sensory information to anticipate future events. Temporal information can be presented implicitly through probability manipulation without participants’ awareness of the manipulation, or explicitly conveyed through instructions. We examined how implicit and explicit temporal information established temporal expectations that influenced choice response times and response conflict (measured as flanker effects). We implicitly manipulated temporal structure by block-wise varying the likely timing of a target. In the short-interval block, a target was presented frequently (80 % of trials) after a short (400 ms) cue-to-target interval and infrequently (20 % of trials) after a long (1200 ms) interval; the probability assignment was reversed in the long-interval block. Building on this baseline condition (Experiment 1), we augmented the temporal information by filling the cue-to-target intervals with tones (Experiment 2), explicitly informed participants of the prevalent time interval (Experiment 3) and provided trial-by-trial reminders of the prevalent time interval (Experiment 4). The temporal probability manipulation alone (of which participants were unaware) influenced choice response times but only when the temporal information was augmented with tones, whereas providing the explicit knowledge of the temporal manipulation, with or without trial-by-trial reminders, robustly influenced choice response times. Response conflict was unaffected by these conditions. These results suggest that temporal expectation can be established by the implicit learning of a temporal structure given that sufficiently strong temporal information is presented as well as by the explicit knowledge of the temporal structure. This established temporal expectation influences choice response times without necessarily affecting the strength of response conflict.     

Filling gaps: Decision principles and structure in sentence comprehension

The correct grammatical characterization of sentences containing filler-gap dependencies is a topic of considerable theoretical interest in linguistics. In some grammatical frameworks, these dependencies are representef in terms of conditions on the permissible indexing of structures (or alternative structure evaluation conditions) which a representation must adhere to in order to be well-formed. In other frameworks, constraints on permissible filler-gap dependencies are simply inherent in the set of phrase structure rules contained in the grammar of a language.The processing of sentences with multiple (potential) filler-gap dependencies was investigated in two experiments. The first experiment provided evidence for three claims. First, the human sentence processor abides by a strategy of assigning the most recent potential filler to a gap. Hence, ‘recent filler’ sentences where this assignment proves to be correct takes less time to comprehend than ‘distant filler’ sentences where this decision turns out to be incorrect. Second, the recent filler strategy is itself just a special case of a more general strategy of assigning the most salient potential filler to a gap. Third, unambiguous sentences in which a filler-gap assignment is disambiguated by ‘control’ information specified by individual verbs gives rise to the same recent filler errors as ambiguous sentences. This suggests that tentative filler-gap assignments are made by the processor before all of the relevant constraints on permissible filler-gap dependencies are consulted by the processor.The second experiment tested an alternative hypothesis that the more complex ‘distant filler’ sentences took longer to comprehend in the first experiment only because these sentences often contained verbs which license two adjacent gaps. The experiment showed that there was a significant recent filler effect in sentences that did not contain adjacent gaps and that this effect did not interact with verb class.The finding that the processor delays use of verb-control information is extremely surprising. It may be explained by the fact that this information is only relevant to one type of gap (‘equi-gaps’) and what type of gap the processor is dealing with often can not be determined unambiguously at the time when it initially encounters a gap in its left-to-right processing of a sentence.If our interpretation of these findings is correct, they argue for a considerable amount of structure in the sentence comprehension system. Further, they favor a view of sentence processing in which processing operations involving constraints on the permissible indexing (or evaluation) of structures lag behind the processor's structure building operations. Hence, the results favor those grammatical theories which preserve this distinction over grammatical theories which provide a uniform characterization of all syntactic well-formedness conditions.