How Should a Speech Recognizer Work?

Although researchers studying human speech recognition (HSR) and automatic speech recognition (ASR) share a common interest in how information processing systems (human or machine) recognize spoken language, there is little communication between the two disciplines. We suggest that this lack of communication follows largely from the fact that research in these related fields has focused on the mechanics of how speech can be recognized. In Marr's (1982) terms, emphasis has been on the algorithmic and implementational levels rather than on the computational level. In this article, we provide a computational-level analysis of the task of speech recognition, which reveals the close parallels between research concerned with HSR and ASR. We illustrate this relation by presenting a new computational model of human spoken-word recognition, built using techniques from the field of ASR that, in contrast to current existing models of HSR, recognizes words from real speech input.     

Grammatical context constrains lexical competition in spoken word recognition

When perceiving spoken language, listeners must match the incoming acoustic phonetic input to lexical representations in memory. Models that quantify this process propose that the input activates multiple lexical representations in parallel and that these activated representations compete for recognition (Weber & Scharenborg, 2012). In two experiments, we assessed how grammatically constraining contexts alter the process of lexical competition. The results suggest that grammatical context constrains the lexical candidates that are activated to grammatically appropriate competitors. Stimulus words with little competition from items of the same grammatical class benefit more from the addition of grammatical context than do words with more within-class competition. The results provide evidence that top-down contextual information is integrated in the early stages of word recognition. We propose adding a grammatical class level of analysis to existing models of word recognition to account for these findings.     

From recognition to decisions: Extending and testing recognition-based models for multialternative inference

The recognition heuristic is a noncompensatory strategy for inferring which of two alternatives, one recognized and the other not, scores higher on a criterion. According to it, such inferences are based solely on recognition. We generalize this heuristic to tasks with multiple alternatives, proposing a model of how people identify the consideration sets from which they make their final decisions. In doing so, we address concerns about the heuristic’s adequacy as a model of behavior: Past experiments have led several authors to conclude that there is no evidence for a noncompensatory use of recognition but clear evidence that recognition is integrated with other information. Surprisingly, however, in no study was this competing hypothesis—the compensatory integration of recognition—formally specified as a computational model. In four studies, we specify five competing models, conducting eight model comparisons. In these model comparisons, the recognition heuristic emerges as the best predictor of people’s inferences.     

可操作性在物体表征中的作用

近10年来, 越来越多的研究表明, 对可操作物体的识别不仅依赖物体的视觉信息, 同时也依赖操作它的动作信息和知识, 即可操作性。来自行为实验、脑成像和脑损伤病人的研究都表明, 物体的可操作性在物体识别中会被激活, 并起着重要的作用。可操作性的研究不但重新解释了生物与非生物、名词与动词的分离现象, 而且对于研究物体表征, 以及视觉物体识别的神经通路有着重要的理论意义。     

Fixation-dependent memory for natural scenes: an experimental test of scanpath theory

Many modern theories propose that perceptual information is represented by the sensorimotor activity elicited by the original stimulus. Scanpath theory (Noton & Stark, 1971) predicts that reinstating a sequence of eye fixations will help an observer recognize a previously seen image. However, the only studies to investigate this are correlational ones based on calculating scanpath similarity. We therefore describe a series of 5 experiments that constrain the fixations during encoding or recognition of images in order to manipulate scanpath similarity. Participants encoded a set of images and later had to recognize those that they had seen. They spontaneously selected regions that they had fixated during encoding (Experiment 1), and this was a predictor of recognition accuracy. Yoking the parts of the image available at recognition to the encoded scanpath led to better memory performance than randomly selected image regions (Experiment 2), and this could not be explained by the spatial distribution of locations (Experiment 3). However, there was no recognition advantage for re-viewing one's own fixations versus someone else's (Experiment 4) or for retaining their serial order (Experiment 5). Therefore, although it is beneficial to look at encoded regions, there is no evidence that scanpaths are stored or that scanpath recapitulation is functional in scene memory. This paradigm provides a controlled way of studying the integration of scene content, spatial structure, and oculomotor signals, with consequences for the perception, representation, and retrieval of visual information.     

Individual differences in visual word recognition: insights from the English Lexicon Project

Empirical work and models of visual word recognition have traditionally focused on group-level performance. Despite the emphasis on the prototypical reader, there is clear evidence that variation in reading skill modulates word recognition performance. In the present study, we examined differences among individuals who contributed to the English Lexicon Project (http://elexicon.wustl.edu), an online behavioral database containing nearly 4 million word recognition (speeded pronunciation and lexical decision) trials from over 1,200 participants. We observed considerable within- and between-session reliability across distinct sets of items, in terms of overall mean response time (RT), RT distributional characteristics, diffusion model parameters (Ratcliff, Gomez, & McKoon, 2004), and sensitivity to underlying lexical dimensions. This indicates reliably detectable individual differences in word recognition performance. In addition, higher vocabulary knowledge was associated with faster, more accurate word recognition performance, attenuated sensitivity to stimuli characteristics, and more efficient accumulation of information. Finally, in contrast to suggestions in the literature, we did not find evidence that individuals were trading-off their utilization of lexical and nonlexical information.     

Differences in familiarity according to the cognitive reserve of healthy elderly people / Diferencias en familiaridad en función de la reserva cognitiva en ancianos sanos

Abstract

This study examines the relationship between cognitive reserve and familiarity processes in recognition memory. We hypothesize that people with high cognitive reserve are able to better compensate in alternative information retrieval processes. Forty-five participants, divided into high and low cognitive reserve groups, conducted a recognition experiment where they were asked to discriminate between studied and non-studied words that varied in perceptual familiarity. The results indicated that participants were able to use perceptual familiarity to improve their level of recognition. More importantly, people with high cognitive reserve used familiarity better than those with low cognitive reserve. The results provide the first empirical evidence indicating that people with high cognitive reserve are more efficient at balancing recollection and familiarity processes, and thus maintain a better performance level than those with low cognitive reserve.     

Change blindness and visual memory: Visual representations get rich and act poor

Change blindness is often taken as evidence that visual representations are impoverished, while successful recognition of specific objects is taken as evidence that they are richly detailed. In the current experiments, participants performed cover tasks that required each object in a display to be attended. Change detection trials were unexpectedly introduced and surprise recognition tests were given for nonchanging displays. For both change detection and recognition, participants had to distinguish objects from the same basic‐level category, making it likely that specific visual information had to be used for successful performance. Although recognition was above chance, incidental change detection usually remained at floor. These results help reconcile demonstrations of poor change detection with demonstrations of good memory because they suggest that the capability to store visual information in memory is not reflected by the visual system's tendency to utilize these representations for purposes of detecting unexpected changes.     

Are nonconscious processes sufficient to produce false memories?

Seamon, Luo, and Gallo (1998) reported evidence that nonconscious processes could produce false recognition in a converging-associates task, whereby subjects falsely remember a nonstudied lure (e.g., sleep) after studying a list of related words (bed, rest, awake...). Zeelenberg, Plomp, and Raaijmakers (2003) failed to observe this false recognition effect when list word recognition was at chance. We critically evaluate the evidence for nonsconscious processing and report the results of a new experiment designed to overcome previous methodological limitations. Consistent with Seamon et al., we found that conscious activation of a related lure during study was not necessary for its subsequent recognition; consistent with Zeelenberg et al., we found no evidence for recognition of related lures under conditions where there was no memory for studied words. It is currently unknown whether conscious recollection of the studied items is necessary for false recognition or if nonconscious activation of the lure is sufficient.     

Recall,recognition, and confidence patterns in eyewitness testimony

The diversity of methods, contents and tests used in the study of eyewitness memory may have contributed to discrepancies in results in this field. In this experiment, using incidental or intentional learning, we examine the recall and recognition of actions and details concerning the central and peripheral information of a kidnapping. A similar pattern emerges in free recall, hits and recognition confidence: scores are much higher in actions than in central details and there are almost no differences between peripheral actions and details, showing that the distribution of attentional resources is different for actions and details in central than in peripheral information. Although in recall the degree of error was low, in recognition false alarms, especially those in central actions, reduced the level of accuracy to even lower than chance performance in both incidental and intentional groups, also showing that subjects accept false but plausible contents with a high level of confidence. Copyright © 1999 John Wiley & Sons, Ltd.     

Caricature generalization benefits for faces learned with enhanced idiosyncratic shape or texture

Recent findings show benefits for learning and subsequent recognition of faces caricatured in shape or texture, but there is little evidence on whether this caricature learning advantage generalizes to recognition of veridical counterparts at test. Moreover, it has been reported that there is a relatively higher contribution of texture information, at the expense of shape information, for familiar compared to unfamiliar face recognition. The aim of this study was to examine whether veridical faces are recognized better when they were learned as caricatures compared to when they were learned as veridicals—what we call a caricature generalization benefit. Photorealistic facial stimuli derived from a 3-D camera system were caricatured selectively in either shape or texture by 50 %. Faces were learned across different images either as veridicals, shape caricatures, or texture caricatures. At test, all learned and novel faces were presented as previously unseen frontal veridicals, and participants performed an old–new task. We assessed accuracies, reaction times, and face-sensitive event-related potentials (ERPs). Faces learned as caricatures were recognized more accurately than faces learned as veridicals. At learning, N250 and LPC were largest for shape caricatures, suggesting encoding advantages of distinctive facial shape. At test, LPC was largest for faces that had been learned as texture caricatures, indicating the importance of texture for familiar face recognition. Overall, our findings demonstrate that caricature learning advantages can generalize to and, importantly, improve recognition of veridical versions of faces.     

Processing of unattended visual information

A visual analog of auditory selective attention paradigms is described. Using that analog, we examined the amount of processing required to monitor unattended information. With the materials used, there was no evidence that any processing capacity was required to monitor the unattended information. Memory for the unattended information and the recognition of one’s name were examined to provide additional evidence concerning the fate of the unattended information.     

“To Name or Not to Name: That is the Question”: The Role of Response Inhibition in Reading

Reading is a complex process that includes the integration of information about letters (graphemes) and sounds (phonemes). In many circumstances, such as noisy environments, response inhibition is an additional factor that plays a marked role in successful oral reading. Response inhibition can take the form of task relevant inhibition (i.e., foils in a go/no-go task) and task irrelevant inhibition (i.e., distractor information). Here we investigated task relevant inhibition by having participants (N?=?30) take part in two tasks: go/no-go naming with nonwords foils (GNG-NW) and go/no-go naming with pseudohomophones foils (GNG-PH). Also, we investigated the addition of task irrelevant inhibition by having participants (N?=?28) take part in two tasks: GNG-NW?+?information masking and GNG-PH?+?information masking. We provide evidence that during a task relevant inhibition task, sub-word sound level information can be successfully inhibited, as evidenced by comparable response times for regular words and exception words, provided the foils do not contain familiar sound-based information (GNG-NW). In contrast, regular words were read aloud faster than exception words in a GNG-PH task, indicating that sub-word level interference occurs when the foils contain familiar sound-based information. The addition of task irrelevant inhibition (i.e., information masking at the phoneme level), served to increase response time overall, but did not impact the pattern of response times between regular words and exception words. Together these findings provide useful information regarding the role of response inhibition in word recognition and may be useful in computational models of word recognition and future work may benefit from accounting for the effects outlined in this paper.     

Retrieval conditions and false recognition: testing the distinctiveness heuristic

High levels of false recognition are observed after people study lists of semantic associates that all converge on a nonpresented lure word. In previous experiments, we have found that orienting participants to encode distinctive information about study list items by presenting them as pictures as opposed to words produces marked reductions in false recognition. We have suggested that these reductions reflect the operation of a distinctiveness heuristic: Participants demand access to detailed pictorial information in order to support a positive recognition decision. The present experiments provide additional evidence on this point and allow us to distinguish between the distinctiveness heuristic account and an alternative account based on the impoverished encoding of relational information that occurs when one is studying pictures. In Experiment 1, even when only half of the items in a study list were presented as pictures, a general suppression of false recognition was observed that could be attributable to impoverished encoding of relational information. Experiment 2 provided a critical test of the distinctiveness heuristic account: We manipulated test instructions and found that differences in false recognition rates between picture and word encoding were attenuated in a retrieval condition that did not encourage reliance on a distinctiveness heuristic.     

Interaction Between Phonological and Semantic Representations: Time Matters

Computational modeling and eye‐tracking were used to investigate how phonological and semantic information interact to influence the time course of spoken word recognition. We extended our recent models (Chen & Mirman, 2012; Mirman, Britt, & Chen, 2013) to account for new evidence that competition among phonological neighbors influences activation of semantically related concepts during spoken word recognition (Apfelbaum, Blumstein, & McMurray, 2011). The model made a novel prediction: Semantic input modulates the effect of phonological neighbors on target word processing, producing an approximately inverted‐U‐shaped pattern with a high phonological density advantage at an intermediate level of semantic input—in contrast to the typical disadvantage for high phonological density words in spoken word recognition. This prediction was confirmed with a new analysis of the Apfelbaum et al. data and in a visual world paradigm experiment with preview duration serving as a manipulation of strength of semantic input. These results are consistent with our previous claim that strongly active neighbors produce net inhibitory effects and weakly active neighbors produce net facilitative effects.     

Dual processes in recognition: Does a focus on measurement operations provide a sufficient foundation?

Current theoretical thinking about dual processes in recognition relies heavily on the measurement operations embodied within the process dissociation procedure. We critically evaluate the ability of this procedure to support this theoretical enterprise. We show that there are alternative processes that would produce a rough invariance in familiarity (a key prediction of the dual-processing approach) and that the process dissociation procedure does not have the power to differentiate between these alternative possibilities. We also show that attempts to relate parameters estimated by the process dissociation procedure to subjective reports (remember-know judgments) cannot differentiate between alternative dual-processing models and that there are problems with some of the historical evidence and with obtaining converging evidence. Our conclusion is that more specific theories incorporating ideas about representation and process are required.     

The Utility of Item-Level Analyses in Model Evaluation: A Reply to Seidenberg and Plaut

Seidenberg and Plaut (this issue) argue that the implications of our analyses (Spieler & Balota, 1997) for the two extant connectionist models of word naming are limited by two factors. First, variables outside the scope of these models influence naming performance, so it is not surprising that the models do not account for much of the variance at the item level. Second, there is error variance associated with large item-level data sets that obviously should not be captured by these models. We point out that there are a number of variables that have been incorporated within the targeted connectionist models that should provide these models an advantage over the simple predictor variables that we selected as a baseline to evaluate the efficacy of the models (e.g., log frequency, length in letters, and number of orthographic neighbors). We also point out that there is considerable consistency across four large-scale studies of item means. Finally, we provide evidence that even under conditions of a standard word-naming study (with a small set of items), simple word frequency, orthographic neighborhoods, and length accounted for more variance than the extant connectionist models. We conclude that item-level analyses provide an important source of evidence in the evaluation of current models and the development of future models of visual word recognition.     

Neutral details associated with emotional events are encoded: evidence from a cued recall paradigm

Enhanced emotional memory often comes at the cost of memory for surrounding background information. Narrowed-encoding theories suggest that this is due to narrowed attention for emotional information at encoding, leading to impaired encoding of background information. Recent work has suggested that an encoding-based theory may be insufficient. Here, we examined whether cued recall—instead of previously used recognition memory tasks—would reveal evidence that non-emotional information associated with emotional information was effectively encoded. Participants encoded positive, negative, or neutral objects on neutral backgrounds. At retrieval, they were given either the item or the background as a memory cue and were asked to recall the associated scene element. Counter to narrowed-encoding theories, emotional items were more likely than neutral items to trigger recall of the associated background. This finding suggests that there is a memory trace of this contextual information and that emotional cues may facilitate retrieval of this information.