Quantity judgments of auditory and visual stimuli by chimpanzees (Pan troglodytes)

Many species can choose between two visual sets of stimuli on the basis of quantity. This is true when sets are both visible, or are presented one set at a time or even one item at a time. However, we know comparatively little about how well nonhuman animals can compare auditory quantities. Here, three chimpanzees (Pan troglodytes) chose between two sets of food items when they only heard each item fall into different containers rather than seeing those items. This method prevented the chimpanzees from summing the amount of visible food they saw because there were no visual cues. Chimpanzees performed well, and their performance matched that of previous experiments with regard to obeying Weber's law. They also performed well with comparisons between a sequentially presented auditory set and a fully visible set, demonstrating that duration of presentation was not being used as a cue. In addition, they accommodated empty sets into these judgments, although not perfectly. Thus, chimpanzees can judge auditory quantities in flexible ways that show many similarities to how they compare visual quantities.     

Effects of Auditory/Visual and Lexical/Non-lexical Comparisons on Event-related Potentials in a Memory-scanning Task

A late parietal positivity (P3) and behavioural measures were studied during performance of a two-item memory-scanning task. Stimuli were digits presented as memorised items in one modality (auditory or visual) while the following probe, also a digit, was presented in the same or the other modality. In a separate set of experiments, P3 and behaviour were similarly studied using only visual stimuli that were either lexical (digits) or non-lexical (novel fonts with the same contours as the digits) to which subjects assigned numerical values. Reaction times (RTs) and P3 latencies were prolonged to non-lexical compared to lexical stimuli. Although RTs were longer to auditory than to visual stimuli, P3 latencies to memorised items were prolonged in response to visually compared to auditorily presented memorised items, and were further prolonged when preceding visual probes. P3 amplitudes were smaller to auditory than to visual stimuli, and were smaller for the second memorised item when lexical/non-lexical comparisons were involved. The most striking finding was scalp distribution variations indicating changes in relative contributions of brain structures involved in processing memorised items, according to the probes that followed. These findings are compatible, in general, with a phonological memorisation, but they suggest that the process is modified by memorising the item in the same terms as the expected probe that follows.     

Effects of same-modality interference on immediate serial recall of auditory and visual information.

Two studies investigated the effects of same-modality interference on the immediate serial recall of auditorily and visually presented stimuli. Typically, research in which this task is used has been conducted in quiet rooms, excluding auditory information that is extraneous to the auditorily presented stimuli. However, visual information such as background items clearly within the subject's view have not been excluded during visual presentation. Therefore, in both the present studies, the authors used procedures that eliminated extra-list visual interference and introduced extra-list auditory interference. When same-modality interference was eliminated, weak visual recency effects were found, but they were smaller than those that were generated by auditorily presented items. Further, mid-list and end-of-list recall of visually presented stimuli was unaffected by the amount of interfering visual information. On the other hand, the introduction of auditory interference increased mid-list recall of auditory stimuli. The results of Experiment 2 showed that the mid-list effect occurred with a moderate, but not with a minimal or maximal, level of auditory interference, indicating that moderate amounts of auditory interference had an alerting effect that is not present in typical visual interference.     

Absence of a cross-modal “suffix effect” in short-term memory

Three experiments are reported involving the presentation of lists of either letters or digits for immediate serial recall. The main variable was the presence or absence of a suffix-prefix, an item (tick or cross) occurring at the end of the list which had to be copied before recall of the stimulus list. With auditory stimuli and an auditory suffix-prefix there was a large and selective increase in the number of errors on the last few serial positions—the typical “suffix effect”. The suffix effect was not found with auditory stimuli and a visual suffix-prefix nor with a visual stimulus and an auditory suffix-prefix. These results are interpreted as supporting a model for short-term memory proposed by Crowder and Morton (1969) in which it is suggested that with serial recall information concerning the final items following auditory presentation has a different, precategorical, origin from that concerning other items.     

An experimental analysis of memory processing

Rhesus monkeys were trained and tested in visual and auditory list-memory tasks with sequences of four travel pictures or four natural/environmental sounds followed by single test items. Acquisitions of the visual list-memory task are presented. Visual recency (last item) memory diminished with retention delay, and primacy (first item) memory strengthened. Capuchin monkeys, pigeons, and humans showed similar visual-memory changes. Rhesus learned an auditory memory task and showed octave generalization for some lists of notes--tonal, but not atonal, musical passages. In contrast with visual list memory, auditory primacy memory diminished with delay and auditory recency memory strengthened. Manipulations of interitem intervals, list length, and item presentation frequency revealed proactive and retroactive inhibition among items of individual auditory lists. Repeating visual items from prior lists produced interference (on nonmatching tests) revealing how far back memory extended. The possibility of using the interference function to separate familiarity vs. recollective memory processing is discussed.     

Modality effects in retrieval of information from short-term memory

Retrieval from short-term memory was studied in a reaction time task, Ss decided whether or not a target item was present in a memorized list. When modality of presentation was manipulated, fastest reaction times resulted if the target item and all memory items were presented auditorily. All conditions involving pure visual presentation or mixed auditory-visual presentation resulted in reaction times which were longer than those for the pure auditory conditions out which did not differ from each other. This suggests that retrieval can be trom either a precategorical auditory memory or a postcategorical memory which does not preserve the sensory features of the memorized material.     

Modality effects and the structure of short-term verbal memory

The effects of auditory and visual presentation upon short-term retention of verbal stimuli are reviewed, and a model of the structure of short-term memory is presented. The main assumption of the model is that verbal information presented to the auditory and visual modalities is processed in separate streams that have different properties and capabilities. Auditory items are automatically encoded in both the A (acoustic) code, which, in the absence of subsequent input, can be maintained for some time without deliberate allocation of attention, and a P (phonological) code. Visual items are retained in both the P code and a visual code. Within the auditory stream, successive items are strongly associated; in contrast, in the visual modality, it is simultaneously presented items that are strongly associated. These assumptions about the structure of short-term verbal memory are shown to account for many of the observed effect of presentation modality.     

Parallel scanning of auditory and visual information

Two experiments involving memory retrieval of auditorilv and visually presented materials were performed. In Experiment I, subjects were presented with memory sets of 1, 2, or 4 stimuli and then with a test item to be classified as belonging or not belonging to the memory set. In Condition 1, each memory stimulus was a single, auditorily presented letter. In Condition 2, each memory stimulus was a visually presented letter. In Conditions 3 and 4, each memory stimulus was a pair of letters, one presented visually and the other auditorily. Mean reaction time (RT) for the classification task increased as a function of number of memory stimuli at equal rates for all four conditions. This was interpreted as evidence for a parallel scanning process in Conditions 3 and 4 where the auditory item and visual item of each memory stimulus pair can be scanned simultaneously. Experiment II compared memory retrieval for a simultaneous condition in which auditory and visual memory items were presented as pairs with a sequential condition in which mixed auditory-visual memory sets were presented one item at a time. RTs were shorter for the simultaneous condition. This was interpreted as evidence that parallel scanning may depend upon memory input parameters.     

Disruption of order information by irrelevant items: a serial recognition paradigm

Irrelevant speech effect (ISE) is defined as a decrement in visually presented digit-list short-term memory performance due to exposure to irrelevant auditory material. Perhaps the most successful theoretical explanation of the effect is the changing state hypothesis. This hypothesis explains the effect in terms of confusion between amodal serial order cues, and represents a view based on the interference caused by the processing of similar order information of the visual and auditory materials. An alternative view suggests that the interference occurs as a consequence of the similarity between the visual and auditory contents of the stimuli. An important argument for the former view is the observation that ISE is almost exclusively observed in tasks that require memory for serial order. However, most short-term memory tasks require that both item and order information be retained in memory. An ideal task to investigate the sensitivity of maintenance of serial order to irrelevant speech would be one that calls upon order information but not item information. One task that is particularly suited to address this issue is serial recognition. In a typical serial recognition task, a list of items is presented and then probed by the same list in which the order of two adjacent items has been transposed. Due to the re-presentation of the encoding string, serial recognition requires primarily the serial order to be maintained while the content of the presented items is deemphasized. In demonstrating a highly significant ISE of changing versus steady-state auditory items in a serial recognition task, the present finding lends support for and extends previous empirical findings suggesting that irrelevant speech has the potential to interfere with the coding of the order of the items to be memorized.     

Enumeration produces poor primacy for tactile presentation relative to visual and auditory presentation as the only modality effect

Methodological biases may help explain the modality effect, which is superior recall of auditory recency (end of list) items relative to visual recency items. In 1985 Nairne and McNabb used a counting procedure to reduce methodological biases, and they produced modality-like effects, such that recall of tactile recency items was superior to recall of visual recency items. The present study extended Nairne and McNabb's counting procedure and controlled several variables which may have enhanced recall of tactile end items or disrupted recall of visual end items in their study. Although the results of the present study indicated general serial position effects across tactile, visual, and auditory presentation modalities, the tactile condition showed lower recall for the initial items in the presentation list than the other two conditions. Moreover, recall of the final list item did not differ across the three presentation modalities; modality effects were not found. These results did not replicate the findings of Nairne and McNabb, or much of the past research showing superior recall of auditory recency items. Implications of these findings are discussed.     

Do you remember where sounds,pictures and words came from? The role of the stimulus format in object location memory

Contrasting results in visual and auditory spatial memory stimulate the debate over the role of sensory modality and attention in identity-to-location binding. We investigated the role of sensory modality in the incidental/deliberate encoding of the location of a sequence of items. In 4 separated blocks, 88 participants memorised sequences of environmental sounds, spoken words, pictures and written words, respectively. After memorisation, participants were asked to recognise old from new items in a new sequence of stimuli. They were also asked to indicate from which side of the screen (visual stimuli) or headphone channel (sounds) the old stimuli were presented in encoding. In the first block, participants were not aware of the spatial requirement while, in blocks 2, 3 and 4 they knew that their memory for item location was going to be tested. Results show significantly lower accuracy of object location memory for the auditory stimuli (environmental sounds and spoken words) than for images (pictures and written words). Awareness of spatial requirement did not influence localisation accuracy. We conclude that: (a) object location memory is more effective for visual objects; (b) object location is implicitly associated with item identity during encoding and (c) visual supremacy in spatial memory does not depend on the automaticity of object location binding.     

Acoustic confusions and order forgetting

This paper reports two experiments employing a PETERSON and PETERSON (1959) paradigm, aimed at showing order as opposed to item forgetting in acoustic confusion experiments. In the first experiment two subsets from the acoustically similar vocabulary C V B D G T P E, i.e. D V B and C G T, were selected. One group of Ss were presented with the items from subset D V B in different orders for 4 trials, another group with the subset C G T. Interpolated activity lasted twenty seconds and presentation was auditory. The second experiment was similar to the first, but presentation was visual and interpolated activity lasted only 5 sec. In both experiments the incidence of order errors significantly and considerably exceeded the incidence of intrusion errors. This leaves open the possibility that in acoustic confusion experiments, confusions can be accounted for by assuming order rather than item forgetting, contrary to CONRAD's (1965) hypothesis.     

Serial recall of colors: Two models of memory for serial order applied to continuous visual stimuli

This study investigated the effects of serial position and temporal distinctiveness on serial recall of simple visual stimuli. Participants observed lists of five colors presented at varying, unpredictably ordered interitem intervals, and their task was to reproduce the colors in their order of presentation by selecting colors on a continuous-response scale. To control for the possibility of verbal labeling, articulatory suppression was required in one of two experimental sessions. The predictions were derived through simulation from two computational models of serial recall: SIMPLE represents the class of temporal-distinctiveness models, whereas SOB-CS represents event-based models. According to temporal-distinctiveness models, items that are temporally isolated within a list are recalled more accurately than items that are temporally crowded. In contrast, event-based models assume that the time intervals between items do not affect recall performance per se, although free time following an item can improve memory for that item because of extended time for the encoding. The experimental and the simulated data were fit to an interference measurement model to measure the tendency to confuse items with other items nearby on the list—the locality constraint—in people as well as in the models. The continuous-reproduction performance showed a pronounced primacy effect with no recency, as well as some evidence for transpositions obeying the locality constraint. Though not entirely conclusive, this evidence favors event-based models over a role for temporal distinctiveness. There was also a strong detrimental effect of articulatory suppression, suggesting that verbal codes can be used to support serial-order memory of simple visual stimuli.     

The effect of an additional task on recall accuracy in serial short term memory: Preemption of rehearsal or acquisition?

In a serial short term memory test, being either visual or auditory, Ss got the additional instruction to make immediate binary classification responses (BcRs) to those items that occured after a marker which was presented somewhere in the list. The results show a general negative effect of BcRs on recall accuracy of both visually and auditory presented lists i.e. both on items which had been classified during presentation and on those which had occured before the marker arrived. The findings are contrary to the view that BcRs preclude acquisition of the items, but the hypothesis is confirmed that BcRs preempt intra trial rehearsal.     

Stimulus modality and working memory performance in Greek children with reading disabilities: Additional evidence for the pictorial superiority hypothesis

This study investigated the effects of stimulus presentation modality on working memory performance in children with reading disabilities (RD) and in typically developing children (TDC), all native speakers of Greek. It was hypothesized that the visual presentation of common objects would result in improved learning and recall performance as compared to the auditory presentation of stimuli. Twenty children, ages 10–12, diagnosed with RD were matched to 20 TDC age peers. The experimental tasks implemented a multitrial verbal learning paradigm incorporating three modalities: auditory, visual, and auditory plus visual. Significant group differences were noted on language, verbal and nonverbal memory, and measures of executive abilities. A mixed-model MANOVA indicated that children with RD had a slower learning curve and recalled fewer words than TDC across experimental modalities. Both groups of participants benefited from the visual presentation of objects; however, children with RD showed the greatest gains during this condition. In conclusion, working memory for common verbal items is impaired in children with RD; however, performance can be facilitated, and learning efficiency maximized, when information is presented visually. The results provide further evidence for the pictorial superiority hypothesis and the theory that pictorial presentation of verbal stimuli is adequate for dual coding.     

Boundary conditions for the modality effect: Type of material and time of test

Recall of auditory and visually presented information was studied using three different types of material. Although auditory presentation was superior for all three types of material, the size and pattern of the effect depended upon whether the visual presentation was successive or simultaneous. Successive visual presentation reduced recall performance compared to auditory presentation considerably for real sentences, while smaller effects were obtained for noun sequences and scrambled sentences. The overall supremacy in recall of auditory presented items was reduced further when compared to a simultaneous visual presentation. In the last two experiments, all effects of modality of presentation obtained in immediate recall were replicated, but were eliminated in delayed recall. The finding that this modality effect is eliminated in delayed recall is taken to indicate that semantic aspects of the material to be remembered are by and large unaffected by mode of presentation, while more superficial aspects are the ones influenced.     

The role of visual interference in producing the long-term modality effect

Typically, recall of the last of a list of auditory items greatly exceeds recall of the last of a list of visual items. This modality effect has been found in serial recall, free recall, and recall using the distractor paradigm in which each to-be-remembered item is preceded and followed by distractor activity. One source of the auditory advantage may be visual interference that reduces recall of visual stimuli. In three experiments, sources of visual interference were minimized. Although this manipulation reduced the modality effect, it did not eliminate the effect.     

A temporal distinctiveness theory of recency and modality effects

A temporal distinctiveness theory of contextually cued retrieval from memory is presented and applied to recency and modality effects. According to this theory, one part of the mnemonic trace of an item is a representation of the item's time of presentation. Time of presentation may be encoded with a coarse grain (so that it is consistent with a wide range of times) or with a fine grain (so that it is consistent with a narrow range of times). Retrieval proceeds by constructing temporally defined search sets that include representations of items consistent with the temporal bounds of the search set. The temporal width of the search set increases as the retention interval increases. Recency effects arise from retrieval of recently presented items from narrow search sets that include representations of few items; within the context of the search set, these items are distinctive and recalled well. Superiority in recall of recently presented auditory information in comparison with recently presented visual information is attributed to differences in the grain of time of presentation representations for aurally (fine grain) and visually (coarse grain) presented information. Four experiments confirm qualitative and quantitative predictions of the theory, including the prediction of auditory superiority at the beginning of the list when the initial items are temporally distinct.     

Effects of multimodal presentation and stimulus familiarity on auditory and visual processing

Two experiments examined the effects of multimodal presentation and stimulus familiarity on auditory and visual processing. In Experiment 1, 10-month-olds were habituated to either an auditory stimulus, a visual stimulus, or an auditory-visual multimodal stimulus. Processing time was assessed during the habituation phase, and discrimination of auditory and visual stimuli was assessed during a subsequent testing phase. In Experiment 2, the familiarity of the auditory or visual stimulus was systematically manipulated by prefamiliarizing infants to either the auditory or visual stimulus prior to the experiment proper. With the exception of the prefamiliarized auditory condition in Experiment 2, infants in the multimodal conditions failed to increase looking when the visual component changed at test. This finding is noteworthy given that infants discriminated the same visual stimuli when presented unimodally, and there was no evidence that multimodal presentation attenuated auditory processing. Possible factors underlying these effects are discussed.     

Teaching children with autism spectrum disorder to tact auditory stimuli

Studies on teaching tacts to individuals with autism spectrum disorder (ASD) have primarily focused on visual stimuli, despite published clinical recommendations to teach tacts of stimuli in other sensory domains as well. In the current study, two children with ASD were taught to tact auditory stimuli under two stimulus‐presentation arrangements: isolated (auditory stimuli presented without visual cues) and compound (auditory stimuli presented with visual cues). Results indicate that compound stimulus presentation was a more effective teaching procedure, but that it interfered with prior object‐name tacts. A modified compound arrangement in which object‐name tact trials were interspersed with auditory‐stimulus trials mitigated this interference.