Detection of errors during speech production: a review of speech monitoring models

In this paper three theories of speech monitoring are evaluated. The perception-based approach proposes that the same mechanism employed in understanding other-produced language, the speech comprehension system, is also used to monitor one's own speech production. A conceptual, an inner, and an auditory loop convey information to a central, conscious monitor which scrutinizes the adequacy of the ongoing speech flow. In this model, only the end-products in the speech production sequences, the preverbal (propositional) message, the phonetic plan, and the auditory results, are verified. The production-based account assumes multiple local, autonomous monitoring devices, which can look inside formulation components. Moreover, these devices might be tuned to various signals from the actual speech motor execution, e.g. efferent, tactile, and proprioceptive feedback. Finally, node structure theory views error detection as a natural outflow of the activation patterns in the node system for speech production. Errors result in prolonged activation of uncommitted nodes, which in turn may incite error awareness. The approaches differ on the points of consciousness, volition and control, the number of monitoring channels, and their speed, flexibility, and capacity, and whether they can account for concurrent language comprehension disorders. From the empirical evidence presently available, it is argued for a central perception-based monitor, potentially augmented with a few automatic, production-based error detection devices.     

Theories of monitoring and the timing of repairs in spontaneous speech.

This study reports the first data to be published on the timing of self-repairs in spontaneous speech, giving means and confidence intervals for cut-off-to-repair, error-to-repair, and cut-off-to-repair times for different types of repair based on 1525 repairs made in the conversational turns of 61 callers to a radio talk show. The three most detailed models of monitoring are discussed in the introduction, with emphasis on their temporal implications. Many of the cut-off-to-repair times observed were faster than would be predicted by any model in the literature. Laver's (1980) theory of monitoring is shown to be incongruent with the observed times, as is Levelt's (1983, 1989) main interruption rule. The results show that people can plan corrections to their speech while talking, and suggest that Kempen and Hoenkamp's (1987) concept of incremental processing can be extended to repairs.     

Comprehension‐Based Skill Acquisition

We present a comprehension‐based computational model of UNIX user skill acquisition and performance in a training context (UNICOM). The work extends a comprehension‐based theory of planning to account for skill acquisition and learning. Individual models of 22 UNIX users were constructed and used to simulate user performance on successive command production problems in a training context. Comparisons of model and the human empirical data result in a high degree of agreement, validating the ability of UNICOM to predict user response to training.     

Effects of Time Pressure on Mechanisms of Speech Production and Self-Monitoring

The purpose of the present study was to examine effects of time pressure on mechanisms of speech production and self-monitoring. The most widely accepted monitoring theory (Levelt, 1989) suggests that monitoring proceeds through language perception, that is, speech error detection is primarily based on the parsing of one's own inner and overt speech. Twenty-four subjects described visual networks at two different rates (normal and fast). The time pressure manipulation affected a number of temporal characteristics: the error to cutoff and cutoff to repair times were shorter in the fast than in the normal condition. The results indicate that the monitor adjusts its speed of error detection and repair planning to the faster speech output rate. The time pressure manipulation did not affect the accuracy of error detection. The implications for the perception theory of monitoring are discussed.     

A sensorimotor theory of temporal tracking and beat induction

In this paper, we develop a theory of the neurobiological basis of temporal tracking and beat induction as a form of sensory-guided action. We propose three principal components for the neurological architecture of temporal tracking: (1) the central auditory system, which represents the temporal information in the input signal in the form of a modulation power spectrum; (2) the musculoskeletal system, which carries out the action and (3) a controller, in the form of a parieto-cerebellar-frontal loop, which carries out the synchronisation between input and output by means of an internal model of the musculoskeletal dynamics. The theory is implemented in the form of a computational algorithm which takes sound samples as input and synchronises a simple linear mass-spring-damper system to simulate audio-motor synchronisation. The model may be applied to both the tracking of isochronous click sequences and beat induction in rhythmic music or speech, and also accounts for the approximate Weberian property of timing.     

Disruption of verbal STM by irrelevant speech, articulatory suppression, and manual tapping: Do they have a common source?

Under appropriate conditions, immediate serial verbal recall is impaired by irrelevant speech, articulatory suppression, and syncopated tapping. Interpretation of these variables in terms of the phonological loop component of working memory assumes separate phonological storage and articulatory rehearsal processes. In contrast, the Object-Oriented Episodic Record (O-OER) of Jones and the feature theory of Neath interpret these and other phenomena in terms of a unitary multimodal system. Three experiments investigate these disrupting tasks, with each experiment emphasizing one parameter. In each case, recall of phonologically similar and dissimilar letter sequences is compared as a marker of the presence or absence of phonological coding. In Experiment 1, subjects heard or articulated a single item, or tapped a single key at equal intervals. Only articulatory suppression impaired performance; it also abolished the effects of phonological similarity. Experiment 2 was identical, except that items were heard, or generated in a syncopated rhythm. Both suppression and tapping impaired performance to an equivalent extent and obliterated the effect of phonological similarity. Syncopated irrelevant speech caused a modest but significant impairment in performance. Experiment 3 was identical to Experiment 1, except that six tokens were used. Irrelevant speech and tapping had a clear impact on recall, but neither removed the phonological similarity effect. Again articulatory suppression had a major impact on performance and removed the effect of phonological similarity. We conclude that the pattern of results readily fits the phonological loop hypothesis, provided one accepts Saito's proposal that generating syncopated sequences uses common processes with speech production. It is not clear how the results can be explained by either the O-OER or the feature hypothesis.     

Multitimescale dynamical interactions between speech rhythm and gesture

Temporal patterns in human movement, and in speech in particular, occur on multiple timescales. Regularities in such patterns have been observed between speech gestures, which are relatively quick movements of articulators (e.g., tongue fronting and lip protrusion), and also between rhythmic units (e.g., syllables and metrical feet), which occur more slowly. Previous work has shown that patterns in both domains can be usefully modeled with oscillatory dynamical systems. To investigate how rhythmic and gestural domains interact, an experiment was conducted in which speakers performed a phrase repetition task, and gestural kinematics were recorded using electromagnetic articulometry. Variance in relative timing of gestural movements was correlated with variance in rhythmic timing, indicating that gestural and rhythmic systems interact in the process of planning and producing speech. A model of rhythmic and gestural planning oscillators with multifrequency coupling is presented, which can simulate the observed covariability between rhythmic and gestural timing.     

Language deficits and the theory of syntax

A new structural account of agrammatism is proposed, which analyzes the deficit in terms of one current theory of syntax. First, the motivation for accounts of this kind is given. Then, a variety of experimental findings from sentence comprehension in agrammatism are examined and accounted for in a unified way. It is shown that a minimal change in the syntactic model (achieved by imposing a special condition on a construct called trace), results in a model which accounts for all the data at hand. A number of possible objections to this proposal is then examined, and reasons are given to dismiss these objections. Also, it is shown that this proposal is preferable to other structural accounts which have been recently proposed. Finally, the empirical consequences of this account are discussed, with a special emphasis on the implications for models of language processing.     

A Dynamical Approach to Gestural Patterning in Speech Production

In this article, we attempt to reconcile the linguistic hypothesis that speech involves an underlying sequencing of abstract, discrete, context-independent units, with the empirical observation of continuous, context-dependent interleaving of articulatory movements. To this end, we first review a previously proposed task-dynamic model for the coordination and control of the speech articulators. We then describe an extension of this model in which invariant speech units (gestural primitives) are identified with context-independent sets of parameters in a dynamical system having two functionally distinct but interacting levels. The intergestural level is defined according to a set of activation coordinates; the interarticulator level is defined according to both model articulator and tractvariable coordinates. In the framework of this extended model, coproduction effects in speech are described in terms of the blending dynamics defined among a set of temporally overlapping active units; the relative timing of speech gestures is formulated in terms of the serial dynamics that shape the temporal patterning of onsets and offsets in unit activations. Implications of this approach for certain phonological issues are discussed, and a range of relevant experimental data on speech and limb motor control is reviewed.     

Spoken language and arm gestures are controlled by the same motor control system

Arm movements can influence language comprehension much as semantics can influence arm movement planning. Arm movement itself can be used as a linguistic signal. We reviewed neurophysiological and behavioural evidence that manual gestures and vocal language share the same control system. Studies of primate premotor cortex and, in particular, of the so-called "mirror system", including humans, suggest the existence of a dual hand/mouth motor command system involved in ingestion activities. This may be the platform on which a combined manual and vocal communication system was constructed. In humans, speech is typically accompanied by manual gesture, speech production itself is influenced by executing or observing transitive hand actions, and manual actions play an important role in the development of speech, from the babbling stage onwards. Behavioural data also show reciprocal influence between word and symbolic gestures. Neuroimaging and repetitive transcranial magnetic stimulation (rTMS) data suggest that the system governing both speech and gesture is located in Broca's area. In general, the presented data support the hypothesis that the hand motor-control system is involved in higher order cognition.     

Motor movement matters: The flexible abstractness of inner speech

Inner speech is typically characterized as either the activation of abstract linguistic representations or a detailed articulatory simulation that lacks only the production of sound. We present a study of the speech errors that occur during the inner recitation of tongue-twister-like phrases. Two forms of inner speech were tested: inner speech without articulatory movements and articulated (mouthed) inner speech. Although mouthing one’s inner speech could reasonably be assumed to require more articulatory planning, prominent theories assume that such planning should not affect the experience of inner speech and, consequently, the errors that are “heard” during its production. The errors occurring in articulated inner speech exhibited the phonemic similarity effect and the lexical bias effect—two speech-error phenomena that, in overt speech, have been localized to an articulatory-feature-processing level and a lexical—phonological level, respectively. In contrast, errors in unarticulated inner speech did not exhibit the phonemic similarity effect—just the lexical bias effect. The results are interpreted as support for a flexible abstraction account of inner speech. This conclusion has ramifications for the embodiment of language and speech and for the theories of speech production.     

A computer model of the temporal course of agrammatic sentence understanding: The effects of variation in severity and sentence complexity

In this article, we present SYNCHRON, a computer model of Kolk and van Grunsven's (1985) hypothesis of agrammatic comprehension deficits. According to this hypothesis, parsing fails in agrammatic aphasies since syntactic representational elements that need to be active simultaneously are often not coactive because of disturbances in timing due to brain damage. SYNCHRON has been especially designed to account for two neglected aspects of agrammatic comprehension: degrees of severity and the sentence-complexity effect. We report an attempt to simulate data from two sentence-picture-matching studies in which a qualitatively similar sentence-complexity effect was found at two different average levels of severity. This pattern was reproduced when the timing disorder was assumed to affect syntactic phrasal categories, but not when it was assumed to affect function-word categories. When phrasal categories were involved, fit was not much affected, whether the damage was assumed to slow down the time for an element to be retrieved, or to decrease the time for an element to remain available.     

The Left Hand Doesn't Know What the Right Hand Is Doing: The Disruptive Effects of Attention to the Hands in Skilled Typewriting

ABSTRACT— Everyone knows that attention to the details disrupts skilled performance, but little empirical evidence documents this fact. We show that attention to the hands disrupts skilled typewriting. We had skilled typists type words preceded by cues that told them to type only the letters assigned to one hand or to type all of the letters. Cuing the hands disrupted performance markedly, slowing typing and increasing the error rate (Experiment 1); these deleterious effects were observed even when no keystrokes were actually inhibited (Experiment 3). However, cuing the same letters with colors was not disruptive (Experiment 2). We account for the disruption with a hierarchical control model, in which an inner loop controls the hands and an outer loop controls what is typed. Typing letters using only one hand requires the outer loop to monitor the inner loop's output; the outer loop slows inner-loop cycle time to increase the likelihood of inhibiting responses with the unwanted hand. This produces the disruption.     

Plato's Revenge: Moral Deliberation As Dialogical Activity

In this article I offer an account of normative thought inspired by Plato's proposal in the Theaetetus that judgement is ‘speech spoken … silently.’ After arguing that force conventionalism is the speech act theory best suited for modeling dialogic inner speech, I close the article by sketching the picture of normative thought that results. Though I defend a particular theory of normative speech elsewhere, the core insights of this article can be used by other theorists as well. The arguments offered here also serve as an important step forward for the more general program of using social speech to better understand thought.     

Re-examining selective adaptation: Fatiguing feature detectors,or distributional learning?

When a listener hears many good examples of a /b/ in a row, they are less likely to classify other sounds on, e.g., a /b/-to-/d/ continuum as /b/. This phenomenon is known as selective adaptation and is a well-studied property of speech perception. Traditionally, selective adaptation is seen as a mechanistic property of the speech perception system, and attributed to fatigue in acoustic-phonetic feature detectors. However, recent developments in our understanding of non-linguistic sensory adaptation and higher-level adaptive plasticity in speech perception and language comprehension suggest that it is time to re-visit the phenomenon of selective adaptation. We argue that selective adaptation is better thought of as a computational property of the speech perception system. Drawing on a common thread in recent work on both non-linguistic sensory adaptation and plasticity in language comprehension, we furthermore propose that selective adaptation can be seen as a consequence of distributional learning across multiple levels of representation. This proposal opens up new questions for research on selective adaptation itself, and also suggests that selective adaptation can be an important bridge between work on adaptation in low-level sensory systems and the complicated plasticity of the adult language comprehension system.     

Dynamic Text Comprehension

Abstract— Reading is one of the most complex and uniquely human of cognitive activities. Our understanding of the processes and factors involved in text comprehension is quite impressive, but it also is fragmented, with a proliferation of "mini-theories" for specific components that in reality are intertwined and interact with one another. Theories of dynamic text comprehension (DTC) aim to capture the integration of these components. They depict reading comprehension as an ongoing process involving fluctuations in the activation of concepts as the reader proceeds through the text, resulting in a gradually emerging interpretation of the material. Features of texts and characteristics of the reader jointly and interactively affect these fluctuations, influencing and being influenced by the reader's understanding and memory of what is read. We illustrate the DTC approach by describing one theory, called the Landscape model, and summarize how its simulations match empirical data. We conclude with some implications of the DTC framework for basic and applied reading research.     

Theory learning as stochastic search in the language of thought

We present an algorithmic model for the development of children's intuitive theories within a hierarchical Bayesian framework, where theories are described as sets of logical laws generated by a probabilistic context-free grammar. We contrast our approach with connectionist and other emergentist approaches to modeling cognitive development. While their subsymbolic representations provide a smooth error surface that supports efficient gradient-based learning, our symbolic representations are better suited to capturing children's intuitive theories but give rise to a harder learning problem, which can only be solved by exploratory search. Our algorithm attempts to discover the theory that best explains a set of observed data by performing stochastic search at two levels of abstraction: an outer loop in the space of theories and an inner loop in the space of explanations or models generated by each theory given a particular dataset. We show that this stochastic search is capable of learning appropriate theories in several everyday domains and discuss its dynamics in the context of empirical studies of children's learning.     

A symbolic-connectionist theory of relational inference and generalization

The authors present a theory of how relational inference and generalization can be accomplished within a cognitive architecture that is psychologically and neurally realistic. Their proposal is a form of symbolic connectionism: a connectionist system based on distributed representations of concept meanings, using temporal synchrony to bind fillers and roles into relational structures. The authors present a specific instantiation of their theory in the form of a computer simulation model, Learning and Inference with Schemas and Analogies (LISA). By using a kind of self-supervised learning, LISA can make specific inferences and form new relational generalizations and can hence acquire new schemas by induction from examples. The authors demonstrate the sufficiency of the model by using it to simulate a body of empirical phenomena concerning analogical inference and relational generalization.     

老年人医疗语篇理解的影响因素和补偿模型

医疗语篇理解是老年人实现自我照护的关键因素,改善老年人医疗语篇理解有助于提高其生存质量并节约医疗成本。本文整理了老年人医疗语篇理解的影响因素,包括加工能力等老年人个体因素和可读性等语篇设计因素,并在此基础上提出了老年人医疗语篇理解补偿模型。该模型尝试阐述上述各影响因素中,那些随龄衰退的因素(如加工能力),如何从别的因素中得到补偿(如医疗知识),从而支撑老年人的医疗语篇理解。补偿模型有利于干预研究的设计,但也还有待更多的实证研究。     

Is comprehension necessary for error detection? A conflict-based account of monitoring in speech production

Despite the existence of speech errors, verbal communication is successful because speakers can detect (and correct) their errors. The standard theory of speech-error detection, the perceptual-loop account, posits that the comprehension system monitors production output for errors. Such a comprehension-based monitor, however, cannot explain the double dissociation between comprehension and error-detection ability observed in the aphasic patients. We propose a new theory of speech-error detection which is instead based on the production process itself. The theory borrows from studies of forced-choice-response tasks the notion that error detection is accomplished by monitoring response conflict via a frontal brain structure, such as the anterior cingulate cortex. We adapt this idea to the two-step model of word production, and test the model-derived predictions on a sample of aphasic patients. Our results show a strong correlation between patients’ error-detection ability and the model’s characterization of their production skills, and no significant correlation between error detection and comprehension measures, thus supporting a production-based monitor, generally, and the implemented conflict-based monitor in particular. The successful application of the conflict-based theory to error-detection in linguistic, as well as non-linguistic domains points to a domain-general monitoring system.