The Effect of Prominence and Cue Association on Retrieval Processes: A Computational Account

We present a comprehensive empirical evaluation of the ACT‐R–based model of sentence processing developed by Lewis and Vasishth (2005) (LV05). The predictions of the model are compared with the results of a recent meta‐analysis of published reading studies on retrieval interference in reflexive‐/reciprocal‐antecedent and subject–verb dependencies (Jäger, Engelmann, & Vasishth, 2017). The comparison shows that the model has only partial success in explaining the data; and we propose that its prediction space is restricted by oversimplifying assumptions. We then implement a revised model that takes into account differences between individual experimental designs in terms of the prominence of the target and the distractor in memory‐ and context‐dependent cue‐feature associations. The predictions of the original and the revised model are quantitatively compared with the results of the meta‐analysis. Our simulations show that, compared to the original LV05 model, the revised model accounts for the data better. The results suggest that effects of prominence and variable cue‐feature associations need to be considered in the interpretation of existing empirical results and in the design and planning of future experiments. With regard to retrieval interference in sentence processing and to the broader field of psycholinguistic studies, we conclude that well‐specified models in tandem with high‐powered experiments are needed in order to uncover the underlying cognitive processes.     

Patterns of comprehension performance in agrammatic Broca's aphasia: a test of the Trace Deletion Hypothesis

We tested the core prediction of the Trace Deletion Hypothesis (TDH) of agrammatic Broca's aphasia, which contends that such patients' comprehension performance is normal for active reversible sentences but at chance level for passive reversible sentences. We analyzed the comprehension performance of 38 Italian Broca's aphasics with verified damage to Broca's area, who completed sentence-to-picture matching tasks using active and passive reversible sentences as stimuli. The results failed to confirm the predictions made by TDH: only a small minority (15%) performed at chance on passive sentences and better than chance on active sentences. Furthermore, the distribution of the 38 subjects' performance on passive sentences differed from that predicted by the TDH since many more subjects performed at better-than-chance levels than expected. We discuss the implication of these results for claims about the distribution of language processing mechanisms in the brain.     

A Computational Evaluation of Two Models of Retrieval Processes in Sentence Processing in Aphasia

Can sentence comprehension impairments in aphasia be explained by difficulties arising from dependency completion processes in parsing? Two distinct models of dependency completion difficulty are investigated, the Lewis and Vasishth (2005) activation-based model and the direct-access model (DA; McElree, 2000). These models' predictive performance is compared using data from individuals with aphasia (IWAs) and control participants. The data are from a self-paced listening task involving subject and object relative clauses. The relative predictive performance of the models is evaluated using k-fold cross-validation. For both IWAs and controls, the activation-based model furnishes a somewhat better quantitative fit to the data than the DA. Model comparisons using Bayes factors show that, assuming an activation-based model, intermittent deficiencies may be the best explanation for the cause of impairments in IWAs, although slowed syntax and lexical delayed access may also play a role. This is the first computational evaluation of different models of dependency completion using data from impaired and unimpaired individuals. This evaluation develops a systematic approach that can be used to quantitatively compare the predictions of competing models of language processing.     

The effects of slowed speech on auditory comprehension in aphasia

The present study investigates the effects of slowed speech on auditory comprehension in aphasia. Specifically, an attempt was made to isolate the effects of added time on comprehension at the language processing stages of auditory perception, by increasing the duration of the vowel segments in each word; word recognition and semantic analysis, by adding silences between words; and syntactic analysis, by adding silences at constituent phrase boundaries. Sentences were also read at a slow rate to see the effects of naturally slowed speech on sentence comprehension. Test sentences consisted of simple active and passive declarative sentences, and complex sentences with embedded medial and final relative clauses. Sentences were either semantically reversible or nonreversible. Thirty-four aphasic patients who varied in both severity and type of aphasia were tested on a picture verification task. Results indicated that slowing facilitated language comprehension significantly only in the syntactic condition. Neither syntactic complexity nor semantic reversibility interacted with slowed speech to facilitate auditory language comprehension. Further, it was only the Wernicke's aphasics who showed significant improvement with time added at constituent boundaries. These results suggest that time alone does not facilitate language comprehension in aphasia, but that rather it is the interaction of time with syntactic processing which improves comprehension.     

Direct Evidence of Memory Retrieval as a Source of Difficulty in Non‐Local Dependencies in Language

Linguistic dependencies between non‐adjacent words have been shown to cause comprehension difficulty, compared with local dependencies. According to one class of sentence comprehension accounts, non‐local dependencies are difficult because they require the retrieval of the first dependent from memory when the second dependent is encountered. According to these memory‐based accounts, making the first dependent accessible at the time when the second dependent is encountered should help alleviate the difficulty associated with the processing of non‐local dependencies. In a dual‐task paradigm, participants read sentences that did or did not contain a non‐local dependency (i.e., object‐ and subject‐extracted cleft constructions) while simultaneously remembering a word. The memory task was aimed at making the word held in memory accessible throughout the sentence. In an object‐extracted cleft (e.g., It was Ellen whom John consulted…), the object (Ellen) must be retrieved from memory when consulted is encountered. In the critical manipulation, the memory word was identical to the verb's object (ELLEN). In these conditions, the extraction effect was reduced in the comprehension accuracy data and eliminated in the reading time data. These results add to the body of evidence supporting memory‐based accounts of syntactic complexity.     

Specificity of operations in sentence recognition

When we read a sentence do we abstract out its semantic core and discard the surface representation, or is the surface representation an integral part of the encoded sentence? This question was studied by measurement of transfer to a second reading of a sentence of skill acquired in a first encounter with the sentence. The transfer relation was studied when the typography, language, or modality of the first and second embodiments were varied. The results favor a theory that emphasizes recognition in terms of the pattern analyzing operations that are directed at surface lexical representations. A distinction is brought out between operational or procedural memory and substantive or semantic memory. In many cases memory of procedures may be sufficient to recover the information encoded. Stage-wise processing theories do not seem to have place for such notions.     

Anticipation in Real‐World Scenes: The Role of Visual Context and Visual Memory

The human sentence processor is able to make rapid predictions about upcoming linguistic input. For example, upon hearing the verb eat, anticipatory eye‐movements are launched toward edible objects in a visual scene (Altmann & Kamide, 1999). However, the cognitive mechanisms that underlie anticipation remain to be elucidated in ecologically valid contexts. Previous research has, in fact, mainly used clip‐art scenes and object arrays, raising the possibility that anticipatory eye‐movements are limited to displays containing a small number of objects in a visually impoverished context. In Experiment 1, we confirm that anticipation effects occur in real‐world scenes and investigate the mechanisms that underlie such anticipation. In particular, we demonstrate that real‐world scenes provide contextual information that anticipation can draw on: When the target object is not present in the scene, participants infer and fixate regions that are contextually appropriate (e.g., a table upon hearing eat). Experiment 2 investigates whether such contextual inference requires the co‐presence of the scene, or whether memory representations can be utilized instead. The same real‐world scenes as in Experiment 1 are presented to participants, but the scene disappears before the sentence is heard. We find that anticipation occurs even when the screen is blank, including when contextual inference is required. We conclude that anticipatory language processing is able to draw upon global scene representations (such as scene type) to make contextual inferences. These findings are compatible with theories assuming contextual guidance, but posit a challenge for theories assuming object‐based visual indices.     

Verbal working memory and sentence comprehension

This target article discusses the verbal working memory system used in sentence comprehension. We review the concept of working memory as a short-duration system in which small amounts of information are simultaneously stored and manipulated in the service of accomplishing a task. We summarize the argument that syntactic processing in sentence comprehension requires such a storage and computational system. We then ask whether the working memory system used in syntactic processing is the same as that used in verbally mediated tasks that involve conscious controlled processing. Evidence is brought to bear from various sources: the relationship between individual differences in working memory and individual differences in the efficiency of syntactic processing; the effect of concurrent verbal memory load on syntactic processing; and syntactic processing in patients with poor short-term memory, patients with poor working memory, and patients with aphasia. Experimental results from these normal subjects and patients with various brain lesions converge on the conclusion that there is a specialization in the verbal working memory system for assigning the syntactic structure of a sentence and using that structure in determining sentence meaning that is separate from the working memory system underlying the use of sentence meaning to accomplish other functions. We present a theory of the divisions of the verbal working memory system and suggestions regarding its neural basis.     

Processing of representations in declarative and procedural working memory

The article investigates the relation between declarative and procedural working memory (WM; Oberauer, 2009). Two experiments test the assumption that representations in the two subsystems are selected for processing in analogous ways. Participants carried out a series of decisions on memorized lists of digits. For each decision, they had to select declarative and procedural representations. Regarding declarative representations, participants selected a memory set and a digit within this set as the input to each decision. With respect to the procedural representations, they selected a task set to be applied to the selected digit and a response within that task set. We independently manipulated the number of lists and the number of tasks to be switched among (one, two, or three; Experiment 1) and preparation time for a list switch (Experiment 2). For three effects commonly observed in task-switch studies, analogues in declarative WM were found: list-switch costs, mixing costs, and residual switch costs. List- and task-switch costs were underadditive, suggesting that declarative and procedural representations are selected separately and in parallel. The findings support the hypothesis of two analogous WM subsystems.     

Processing of representations in declarative and procedural working memory

The article investigates the relation between declarative and procedural working memory (WM; Oberauer, 2009). Two experiments test the assumption that representations in the two subsystems are selected for processing in analogous ways. Participants carried out a series of decisions on memorized lists of digits. For each decision, they had to select declarative and procedural representations. Regarding declarative representations, participants selected a memory set and a digit within this set as the input to each decision. With respect to the procedural representations, they selected a task set to be applied to the selected digit and a response within that task set. We independently manipulated the number of lists and the number of tasks to be switched among (one, two, or three; Experiment 1) and preparation time for a list switch (Experiment 2). For three effects commonly observed in task-switch studies, analogues in declarative WM were found: list-switch costs, mixing costs, and residual switch costs. List- and task-switch costs were underadditive, suggesting that declarative and procedural representations are selected separately and in parallel. The findings support the hypothesis of two analogous WM subsystems.     

What Differentiates Declarative and Procedural Memories: Reply to Cohen,Poldrack, and Eichenbaum (1997)

Cohen, Poldrack, and Eichenbaum (1997; hereafter CPE) offer an account of the nature of individual items in memory and how they relate to one another. They argue that there are two separate memory systems, procedural and declarative (Cohen & Eichenbaum, 1993; Cohen & Squire, 1980). These systems differ in their neuroanatomic substrates, in their operating characteristics, and in the nature of the representations they use. CPE argue that representations in the declarative memory system are compositional, meaning that declarative representations may be composed of other declarative representations. Declarative memories are also flexible, meaning they can be accessed in contexts that differ from those in which they were encoded. Procedural memories, on the other hand, are neither compositional nor flexible. I will argue that there is not sufficient reason to argue that procedural and declarative memories have these distinct characteristics. Both procedural and declarative memories are arguably compositional, and both can appear flexible or inflexible, depending on testing conditions.     

Slowed Speech Input has a Differential Impact on On-line and Off-line Processing in Children’s Comprehension of Pronouns

The central question underlying this study revolves around how children process co-reference relationships—such as those evidenced by pronouns (him) and reflexives (himself)—and how a slowed rate of speech input may critically affect this process. Previous studies of child language processing have demonstrated that typical language developing (TLD) children as young as 4 years of age process co-reference relations in a manner similar to adults on-line. In contrast, off-line measures of pronoun comprehension suggest a developmental delay for pronouns (relative to reflexives). The present study examines dependency relations in TLD children (ages 5–13) and investigates how a slowed rate of speech input affects the unconscious (on-line) and conscious (off-line) parsing of these constructions. For the on-line investigations (using a cross-modal picture priming paradigm), results indicate that at a normal rate of speech TLD children demonstrate adult-like syntactic reflexes. At a slowed rate of speech the typical language developing children displayed a breakdown in automatic syntactic parsing (again, similar to the pattern seen in unimpaired adults). As demonstrated in the literature, our off-line investigations (sentence/picture matching task) revealed that these children performed much better on reflexives than on pronouns at a regular speech rate. However, at the slow speech rate, performance on pronouns was substantially improved, whereas performance on reflexives was not different than at the regular speech rate. We interpret these results in light of a distinction between fast automatic processes (relied upon for on-line processing in real time) and conscious reflective processes (relied upon for off-line processing), such that slowed speech input disrupts the former, yet improves the latter.     

Real-time examinations of lexical processing in aphasics

We argue that the lesion localizing value of disruptions to modular information processing systems emerges most clearly from on-line analyses of processing. In this respect we seek to show that left anterior (but not left posterior) damage causes slowed information access and we discuss the manner in which this slowing might yield some of the specific syntactic limitations charted in Broca's aphasia. The general possibility we raise is that the cortical area implicated in Broca's aphasia is not necessarily the locus of syntactic representations, but rather sustains particular time-based operating characteristics that in turn sustain normal real-time parsing.This research was supported in part by NIH grants NS11408 and NS06209 to Boston University School of Medicine and by the Research Service of the Veterans Administration.     

Non-symbolic arithmetic abilities and mathematics achievement in the first year of formal schooling

We examine the nature of motor representations evoked during comprehension of written sentences describing hand actions. We distinguish between two kinds of hand actions: a functional action, applied when using the object for its intended purpose, and a volumetric action, applied when picking up or holding the object. In Experiment 1, initial activation of both action representations was followed by selection of the functional action, regardless of sentence context. Experiment 2 showed that when the sentence was followed by a picture of the object, clear context-specific effects on evoked action representations were obtained. Experiment 3 established that when a picture of an object was presented alone, the time course of both functional and volumetric actions was the same. These results provide evidence that representations of object-related hand actions are evoked as part of sentence processing. In addition, we discuss the conditions that elicit context-specific evocation of motor representations.     

Single-word predictions of upcoming language during comprehension: Evidence from the cumulative semantic interference task

Comprehenders predict upcoming speech and text on the basis of linguistic input. How many predictions do comprehenders make for an upcoming word? If a listener strongly expects to hear the word “sock”, is the word “shirt” partially expected as well, is it actively inhibited, or is it ignored? The present research addressed these questions by measuring the “downstream” effects of prediction on the processing of subsequently presented stimuli using the cumulative semantic interference paradigm. In three experiments, subjects named pictures (sock) that were presented either in isolation or after strongly constraining sentence frames (“After doing his laundry, Mark always seemed to be missing one …”). Naming sock slowed the subsequent naming of the picture shirt – the standard cumulative semantic interference effect. However, although picture naming was much faster after sentence frames, the interference effect was not modulated by the context (bare vs. sentence) in which either picture was presented. According to the only model of cumulative semantic interference that can account for such a pattern of data, this indicates that comprehenders pre-activated and maintained the pre-activation of best sentence completions (sock) but did not maintain the pre-activation of less likely completions (shirt). Thus, comprehenders predicted only the most probable completion for each sentence.     

Mimicking aphasic semantic errors in normal speech production: evidence from a novel experimental paradigm

Semantic errors are commonly found in semantic dementia (SD) and some forms of stroke aphasia and provide insights into semantic processing and speech production. Low error rates are found in standard picture naming tasks in normal controls. In order to increase error rates and thus provide an experimental model of aphasic performance, this study utilised a novel method- tempo picture naming. Experiment 1 showed that, compared to standard deadline naming tasks, participants made more errors on the tempo picture naming tasks. Further, RTs were longer and more errors were produced to living items than non-living items a pattern seen in both semantic dementia and semantically-impaired stroke aphasic patients. Experiment 2 showed that providing the initial phoneme as a cue enhanced performance whereas providing an incorrect phonemic cue further reduced performance. These results support the contention that the tempo picture naming paradigm reduces the time allowed for controlled semantic processing causing increased error rates. This experimental procedure would, therefore, appear to mimic the performance of aphasic patients with multi-modal semantic impairment that results from poor semantic control rather than the degradation of semantic representations observed in semantic dementia [Jefferies, E. A., & Lambon Ralph, M. A. (2006). Semantic impairment in stoke aphasia vs. semantic dementia: A case-series comparison. Brain, 129, 2132-2147]. Further implications for theories of semantic cognition and models of speech processing are discussed.     

The relationship between age,processing speed,working memory capacity,and language comprehension

A total of 50 elderly individuals and 48 college students were tested on several measures of processing speed and of working memory capacity. Language processing was tested with an on‐line measure of sentence processing efficiency, an end‐of‐sentence acceptability judgement task, and a paragraph comprehension test. Elderly individuals performed more poorly than college students on the speed of processing and working memory measures and had longer listening times overall on the sentence processing measures. Elderly individuals did not, however, have overall longer listening times at the most capacity‐demanding regions of the harder sentence types. Correlational analyses failed to establish a relationship between the increase in syntactic processing load at the capacity‐demanding region of the harder sentence type and the measures of working memory capacity, but did establish a relationship between paragraph comprehension and working memory capacity. The data are argued to provide evidence that the WM system used to structure sentences syntactically is separate from that used in other aspects of language comprehension.     

An activation-based model of sentence processing as skilled memory retrieval

We present a detailed process theory of the moment-by-moment working-memory retrievals and associated control structure that subserve sentence comprehension. The theory is derived from the application of independently motivated principles of memory and cognitive skill to the specialized task of sentence parsing. The resulting theory construes sentence processing as a series of skilled associative memory retrievals modulated by similarity-based interference and fluctuating activation. The cognitive principles are formalized in computational form in the Adaptive Control of Thought–Rational (ACT–R) architecture, and our process model is realized in ACT–R. We present the results of 6 sets of simulations: 5 simulation sets provide quantitative accounts of the effects of length and structural interference on both unambiguous and garden-path structures. A final simulation set provides a graded taxonomy of double center embeddings ranging from relatively easy to extremely difficult. The explanation of center-embedding difficulty is a novel one that derives from the model' complete reliance on discriminating retrieval cues in the absence of an explicit representation of serial order information. All fits were obtained with only 1 free scaling parameter fixed across the simulations; all other parameters were ACT–R defaults. The modeling results support the hypothesis that fluctuating activation and similarity-based interference are the key factors shaping working memory in sentence processing. We contrast the theory and empirical predictions with several related accounts of sentence-processing complexity.     

Analogous selection processes in declarative and procedural working memory: N-2 list-repetition and task-repetition costs

Working memory (WM) holds and manipulates representations for ongoing cognition. Oberauer (Psychology of Learning and Motivation, 51, 45–100, 2009) distinguishes between two analogous WM sub-systems: a declarative WM which handles the objects of thought, and a procedural WM which handles the representations of (cognitive) actions. Here, we assessed whether analogous effects are observed when participants switch between memory sets (declarative representations) and when they switch between task sets (procedural representations). One mechanism assumed to facilitate switching in procedural WM is the inhibition of previously used, but currently irrelevant task sets, as indexed by n-2 task-repetition costs (Mayr & Keele, Journal of Experimental Psychology: General, 129(1), 4–26, 2000). In this study we tested for an analogous effect in declarative WM. We assessed the evidence for n-2 list-repetition costs across eight experiments in which participants switched between memory lists to perform speeded classifications, mental arithmetic, or a local recognition test. N-2 list-repetition costs were obtained consistently in conditions assumed to increase interference between memory lists, and when lists formed chunks in long-term memory. Further analyses across experiments revealed a substantial contribution of episodic memory to n-2 list-repetition costs, thereby questioning the interpretation of n-2 repetition costs as reflecting inhibition. We reanalyzed the data of eight task-switching experiments, and observed that episodic memory also contributes to n-2 task-repetition costs. Taken together, these results show analogous processing principles in declarative and procedural WM, and question the relevance of inhibitory processes for efficient switching between mental sets.     

Thematic analysis in agrammatic comprehension: syntactic structures and task demands.

Aphasics' ability to assign thematic roles during sentence comprehension was investigated. The study was motivated by the hypothesis that capacity limitations, e.g., due to slowed processing operations, interact systematically with certain aspects of structural analysis more than with others. Experiments using a sentence-picture matching paradigm were conducted with several agrammatic Broca's and paragrammatic Wernicke's patients. Experiment 1 tested comprehension of various syntatic structures. The astoundingly good performance in Experiment 1 for both patient groups was attributed to the low task demands of the experiment. Raising the task demands, by delaying presentation of the relevant picture set in Experiment 2, resulted in a significant decrement of comprehension performance for agrammatic Broca's aphasics, but not for Wernicke's aphasics. Putting the main verb in the sentence-final position, as in Experiment 3, affected Wernicke's aphasics', but not Broca's aphasics' performance. In principle, one explanation for the performance differences between Experiments 1 and 2 for Broca's patient might be a deficit in these patients' verbal memory capacity. Therefore, verbal memory span on different word categories was tested in an additional experiment. No difference, however, was found between the Broca's patients' and the Wernicke's patients' capacity for verbal memory despite their different sentence comprehension profiles. The present data are taken to support the view that Broca's aphasics' comprehension behavior is due to the processing demands imposed by structural inference chains and not to a general reduction in verbal memory.