Position coding effects in a 2D scenario: The case of musical notation

How does the cognitive system encode the location of objects in a visual scene? In the past decade, this question has attracted much attention in the field of visual-word recognition (e.g., “jugde” is perceptually very close to “judge”). Letter transposition effects have been explained in terms of perceptual uncertainty or shared “open bigrams”. In the present study, we focus on note position coding in music reading (i.e., a 2D scenario). The usual way to display music is the staff (i.e., a set of 5 horizontal lines and their resultant 4 spaces). When reading musical notation, it is critical to identify not only each note (temporal duration), but also its pitch (y-axis) and its temporal sequence (x-axis). To examine note position coding, we employed a same–different task in which two briefly and consecutively presented staves contained four notes. The experiment was conducted with experts (musicians) and non-experts (non-musicians). For the “different” trials, the critical conditions involved staves in which two internal notes that were switched vertically, horizontally, or fully transposed — as well as the appropriate control conditions. Results revealed that note position coding was only approximate at the early stages of processing and that this encoding process was modulated by expertise. We examine the implications of these findings for models of object position encoding.     

Exploring the association between visual perception abilities and reading of musical notation

In the reading of music, the acquisition of pitch information depends primarily upon the spatial position of notes as well as upon an individual's spatial processing ability. This study investigated the relationship between the ability to read single notes and visual-spatial ability. Participants with high and low single-note reading abilities were differentiated based upon differences in musical notation-reading abilities and their spatial processing; object recognition abilities were then assessed. It was found that the group with lower note-reading abilities made more errors than did the group with a higher note-reading abilities in the mental rotation task. In contrast, there was no apparent significant difference between the two groups in the object recognition task. These results suggest that note-reading may be related to visual spatial processing abilities, and not to an individual's ability with object recognition.     

The role of context in object recognition

In the real world, objects never occur in isolation; they co-vary with other objects and particular environments, providing a rich source of contextual associations to be exploited by the visual system. A natural way of representing the context of an object is in terms of its relationship to other objects. Alternately, recent work has shown that a statistical summary of the scene provides a complementary and effective source of information for contextual inference, which enables humans to quickly guide their attention and eyes to regions of interest in natural scenes. A better understanding of how humans build such scene representations, and of the mechanisms of contextual analysis, will lead to a new generation of computer vision systems.     

The role of object recognition in young infants' object segregation

Needham's (2001, this issue) new results confirm that young infants draw on experientially derived representations in resolving individuation ambiguities due to shared boundaries between adjacent objects. They extend previous findings in a surprising way: The memory representations that infants draw upon have bound together information about shape, color, and pattern. Our commentary on these important results draws a distinction between two senses of "recognition" and asks in which sense object recognition contributes to object individuation in these experiments.     

The role of edges in object recognition by pigeons

In three experiments, we explored how pigeons use edges, corresponding to orientation and depth discontinuities, in visual recognition tasks. In experiment 1, we compared the pigeon's ability to recognize line drawings of four different geons when trained with shaded images. The birds were trained with either a single view or five different views of each object. Because the five training views had markedly different appearances and locations of shaded surfaces, reflectance edges, etc, the pigeons might have been expected to rely more on the orientation and depth discontinuities that were preserved over rotation and in the line drawings. In neither condition, however, was there any transfer from the rendered images to the outline drawings. In experiment 2, some pigeons were trained with line drawings and shaded images of the same objects associated with the same response (consistent condition), whereas other pigeons were trained with a line drawing and a shaded image of two different objects associated with the same response (inconsistent condition). If the pigeons perceived any correspondence between the stimulus types, then birds in the consistent condition should have learned the discrimination more quickly than birds in the inconsistent condition. But, there was no difference in performance between birds in the consistent and inconsistent conditions. In experiment 3, we explored pigeons' processing of edges by comparing their discrimination of shaded images or line drawings of four objects. Once trained, the pigeons were tested with planar rotations of those objects. The pigeons exhibited different patterns of generalization depending on whether they were trained with line drawings or shaded images. The results of these three experiments suggest that pigeons may place greater importance on surface features indicating materials, such as food or water. Such substances do not have definite boundaries cued by edges which are thought to be central to human recognition.     

Visual perception of musical notation: Registering pitch symbols in memory

Music bears formal relations to language which suggest that perceptual processes in the two modes may also be similar. These experiments examined the way in which experienced musicians differed from non-musicians in their recognition of briefly exposed pitch notation. Experiments I and II together demonstrated that musicians are superior to non-musicians in their immediate written recall of stimuli containing more than three notes, but only when the stimulus is available to them for 150 ms or more. These results are accounted for well by a model proposed by Coltheart (1972) for letter perception under conditions of brief exposure. In this model, two coding processes act simultaneously on the stimulus, one a fast visual coding, and the other a slower, but more permanent abstract (or name) coding. In this case non-musicians appear to be lacking a second, abstract, coding which musicians possess. Experiments III and IV attempted to investigate the nature of the abstract code for musicians by presenting various types of interference in the linguistic or musical mode. Neither concurrent letter naming nor concurrent memorization of pitches appeared to cause a decrement in the original visual task, suggesting that musicians may not have been using simple naming or pitching transformations in coding the visual input.     

Auditory imagery from musical notation in expert musicians

Anecdotal evidence has suggested that musical notation can trigger auditory images. Expert musicians silently read scores containing well-known themes embedded into the notation of an embellished phrase and judged if a tune heard aloud thereafter was the original theme (i.e., melodic target) or not (i.e., melodic lure). Three experiments were conducted employing four score-reading conditions: normal nondistracted reading, concurrent rhythmic distraction, phonatory interference, and obstruction by auditory stimuli. The findings demonstrate that phonatory interference impaired recognition of original themes more than did the other conditions. We propose that notational audiation is the silent reading of musical notation resulting in auditory imagery. The research suggests that it also elicits kinesthetic-like phonatory processes.     

The role of color diagnosticity in object recognition and representation

The role of color diagnosticity in object recognition and representation was assessed in three Experiments. In Experiment 1a, participants named pictured objects that were strongly associated with a particular color (e.g., pumpkin and orange). Stimuli were presented in a congruent color, incongruent color, or grayscale. Results indicated that congruent color facilitated naming time, incongruent color impeded naming time, and naming times for grayscale items were situated between the congruent and incongruent conditions. Experiment 1b replicated Experiment 1a using a verification task. Experiment 2 employed a picture rebus paradigm in which participants read sentences one word at a time that included pictures of color diagnostic objects (i.e., pictures were substituted for critical nouns). Results indicated that the “reading” times of these pictures mirrored the pattern found in Experiment 1. In Experiment 3, an attempt was made to override color diagnosticity using linguistic context (e.g., a pumpkin was described as painted green). Linguistic context did not override color diagnosticity. Collectively, the results demonstrate that color information is regularly utilized in object recognition and representation for highly color diagnostic items.     

Dynamic melody recognition: Distinctiveness and the role of musical expertise

The hypothesis that melodies are recognized at moments when they exhibit a distinctive musical pattern was tested. In a melody recognition experiment, point-of-recognition (POR) data were gathered from 32 listeners (16 musicians and 16 nonmusicians) judging 120 melodies. A series of models of melody recognition were developed, resulting from a stepwise multiple regression of two classes of information relating to melodic familiarity and melodic distinctiveness. Melodic distinctiveness measures were assembled through statistical analyses of over 15,000 Western themes and melodies. A significant model, explaining 85% of the variance, entered measures primarily of timing distinctiveness and pitch distinctiveness, but excluding familiarity, as predictors of POR. Differences between nonmusician and musician models suggest a processing shift from momentary to accumulated information with increased exposure to music. Supplemental materials for this article may be downloaded from http://mc.psychonomic-journals.org/content/supplemental.     

The role of the human medial temporal lobe in object recognition and object discrimination

This paper reviews evidence from neuropsychological patient studies relevant to two questions concerning the functions of the medial temporal lobe in humans. The first is whether the hippocampus and the adjacent perirhinal cortex make different contributions to memory. Data are discussed from two patients with adult-onset bilateral hippocampal damage who show a sparing of item recognition relative to recall and certain types of associative recognition. It is argued that these data are consistent with Aggleton and Brown's (1999) proposal that familiarity-based recognition memory is not dependent on the hippocampus but is mediated by the perirhinal cortex and dorso-medial thalamic nucleus. The second question is whether the recognition memory deficit observed in medial temporal lobe amnesia can be explained by a deficit in perceptual processing and representation of objects rather than a deficit in memory per se. The finding that amnesics were impaired at recognizing, after short delays, patterns that they could successfully discriminate suggests that their memory impairment did not result from an object-processing deficit. The possibility remains, however, that the human perirhinal cortex plays a role in object processing, as well as in recognition memory, and data are presented that support this possibility.     

The role of surface-based representations of shape in visual object recognition

This study contrasted the role of surfaces and volumetric shape primitives in three-dimensional object recognition. Observers (N?=?50) matched subsets of closed contour fragments, surfaces, or volumetric parts to whole novel objects during a whole–part matching task. Three factors were further manipulated: part viewpoint (either same or different between component parts and whole objects), surface occlusion (comparison parts contained either visible surfaces only, or a surface that was fully or partially occluded in the whole object), and target–distractor similarity. Similarity was varied in terms of systematic variation in nonaccidental (NAP) or metric (MP) properties of individual parts. Analysis of sensitivity (d′) showed a whole–part matching advantage for surface-based parts and volumes over closed contour fragments—but no benefit for volumetric parts over surfaces. We also found a performance cost in matching volumetric parts to wholes when the volumes showed surfaces that were occluded in the whole object. The same pattern was found for both same and different viewpoints, and regardless of target–distractor similarity. These findings challenge models in which recognition is mediated by volumetric part-based shape representations. Instead, we argue that the results are consistent with a surface-based model of high-level shape representation for recognition.     

The role of the human medial temporal lobe in object recognition and object discrimination.

This paper reviews evidence from neuropsychological patient studies relevant to two questions concerning the functions of the medial temporal lobe in humans. The first is whether the hippocampus and the adjacent perirhinal cortex make different contributions to memory. Data are discussed from two patients with adult-onset bilateral hippocampal damage who show a sparing of item recognition relative to recall and certain types of associative recognition. It is argued that these data are consistent with Aggleton and Brown's (1999) proposal that familiarity-based recognition memory is not dependent on the hippocampus but is mediated by the perirhinal cortex and dorso-medial thalamic nucleus. The second question is whether the recognition memory deficit observed in medial temporal lobe amnesia can be explained by a deficit in perceptual processing and representation of objects rather than a deficit in memory per se. The finding that amnesics were impaired at recognizing, after short delays, patterns that they could successfully discriminate suggests that their memory impairment did not result from an object-processing deficit. The possibility remains, however, that the human perirhinal cortex plays a role in object processing, as well as in recognition memory, and data are presented that support this possibility.     

The role of sensory-motor information in object recognition: evidence from category-specific visual agnosia

The role of sensory-motor representations in object recognition was investigated in experiments involving AD, a patient with mild visual agnosia who was impaired in the recognition of visually presented living as compared to non-living entities. AD named visually presented items for which sensory-motor information was available significantly more reliably than items for which such information was not available; this was true when all items were non-living. Naming of objects from their associated sound was normal. These data suggest that both information about object form computed in the ventral visual system as well as sensory-motor information specifying the manner of manipulation contribute to object recognition.     

The role of color information on object recognition: a review and meta-analysis

In this study, we systematically review the scientific literature on the effect of color on object recognition. Thirty-five independent experiments, comprising 1535 participants, were included in a meta-analysis. We found a moderate effect of color on object recognition (d = 0.28). Specific effects of moderator variables were analyzed and we found that color diagnosticity is the factor with the greatest moderator effect on the influence of color in object recognition; studies using color diagnostic objects showed a significant color effect (d = 0.43), whereas a marginal color effect was found in studies that used non-color diagnostic objects (d = 0.18). The present study did not permit the drawing of specific conclusions about the moderator effect of the object recognition task; while the meta-analytic review showed that color information improves object recognition mainly in studies using naming tasks (d = 0.36), the literature review revealed a large body of evidence showing positive effects of color information on object recognition in studies using a large variety of visual recognition tasks. We also found that color is important for the ability to recognize artifacts and natural objects, to recognize objects presented as types (line-drawings) or as tokens (photographs), and to recognize objects that are presented without surface details, such as texture or shadow. Taken together, the results of the meta-analysis strongly support the contention that color plays a role in object recognition. This suggests that the role of color should be taken into account in models of visual object recognition.     

Quantifying the role of context in visual object recognition

An object's context may serve as a source of information for recognition when the object's image is degraded. The current study aimed to quantify this source of information. Stimuli were photographs of objects divided into quantized blocks. Participants decreased block size (increasing resolution) until identification. Critical resolution was compared across three conditions: (1) when the picture of the target object was shown in isolation, (2) in the object's contextual setting where that context was unfamiliar to the participant, and (3) where that context was familiar to the participant. A second experiment assessed the role of object familiarity without context. Results showed a profound effect of context: Participants identified objects in familiar contexts with minimal resolution. Unfamiliar contexts required higher-resolution images, but much less so than those without context. Experiment 2 found a much smaller effect of familiarity without context, suggesting that recognition in familiar contexts is primarily based on object-location memory.     

Holistic processing of musical notation: Dissociating failures of selective attention in experts and novices

Holistic processing (i.e., the tendency to process objects as wholes) is associated with face perception and also with expertise individuating novel objects. Surprisingly, recent work also reveals holistic effects in novice observers. It is unclear whether the same mechanisms support holistic effects in experts and in novices. In the present study, we measured holistic processing of music sequences using a selective attention task in participants who vary in music-reading expertise. We found that holistic effects were strategic in novices but were relatively automatic in experts. Correlational analyses revealed that individual holistic effects were predicted by both individual music-reading ability and neural responses for musical notation in the right fusiform face area (rFFA), but in opposite directions for experts and novices, suggesting that holistic effects in the two groups may be of different natures. To characterize expert perception, it is important not only to measure the tendency to process objects as wholes, but also to test whether this effect is dependent on task constraints.     

Revisiting Snodgrass and Vanderwart's object pictorial set: the role of surface detail in basic-level object recognition

Theories of object recognition differ to the extent that they consider object representations as being mediated only by the shape of the object, or shape and surface details, if surface details are part of the representation. In particular, it has been suggested that color information may be helpful at recognizing objects only in very special cases, but not during basic-level object recognition in good viewing conditions. In this study, we collected normative data (naming agreement, familiarity, complexity, and imagery judgments) for Snodgrass and Vanderwart's object database of 260 black-and-white line drawings, and then compared the data to exactly the same shapes but with added gray-level texture and surface details (set 2), and color (set 3). Naming latencies were also recorded. Whereas the addition of texture and shading without color only slightly improved naming agreement scores for the objects, the addition of color information unambiguously improved naming accuracy and speeded correct response times. As shown in previous studies, the advantage provided by color was larger for objects with a diagnostic color, and structurally similar shapes, such as fruits and vegetables, but was also observed for man-made objects with and without a single diagnostic color. These observations show that basic-level 'everyday' object recognition in normal conditions is facilitated by the presence of color information, and support a 'shape + surface' model of object recognition, for which color is an integral part of the object representation. In addition, the new stimuli (sets 2 and 3) and the corresponding normative data provide valuable materials for a wide range of experimental and clinical studies of object recognition.     

The subjective experience of object recognition: comparing metacognition for object detection and object categorization

Perceptual decisions seem to be made automatically and almost instantly. Constructing a unitary subjective conscious experience takes more time. For example, when trying to avoid a collision with a car on a foggy road you brake or steer away in a reflex, before realizing you were in a near accident. This subjective aspect of object recognition has been given little attention. We used metacognition (assessed with confidence ratings) to measure subjective experience during object detection and object categorization for degraded and masked objects, while objective performance was matched. Metacognition was equal for degraded and masked objects, but categorization led to higher metacognition than did detection. This effect turned out to be driven by a difference in metacognition for correct rejection trials, which seemed to be caused by an asymmetry of the distractor stimulus: It does not contain object-related information in the detection task, whereas it does contain such information in the categorization task. Strikingly, this asymmetry selectively impacted metacognitive ability when objective performance was matched. This finding reveals a fundamental difference in how humans reflect versus act on information: When matching the amount of information required to perform two tasks at some objective level of accuracy (acting), metacognitive ability (reflecting) is still better in tasks that rely on positive evidence (categorization) than in tasks that rely more strongly on an absence of evidence (detection).     

Making psycholinguistics musical: Self-paced reading time evidence for shared processing of linguistic and musical syntax

Linguistic processing, especially syntactic processing, is often considered a hallmark of human cognition; thus, the domain specificity or domain generality of syntactic processing has attracted considerable debate. The present experiments address this issue by simultaneously manipulating syntactic processing demands in language and music. Participants performed self-paced reading of garden path sentences, in which structurally unexpected words cause temporary syntactic processing difficulty. A musical chord accompanied each sentence segment, with the resulting sequence forming a coherent chord progression. When structurally unexpected words were paired with harmonically unexpected chords, participants showed substantially enhanced garden path effects. No such interaction was observed when the critical words violated semantic expectancy or when the critical chords violated timbral expectancy. These results support a prediction of the shared syntactic integration resource hypothesis (Patel, 2003), which suggests that music and language draw on a common pool of limited processing resources for integrating incoming elements into syntactic structures. Notations of the stimuli from this study may be downloaded from pbr.psychonomic-journals.org/content/supplemental.     

The role of syllables in word recognition among beginning Finnish readers: Evidence from eye movements during reading

The eye movements of Finnish first and second graders were monitored as they read sentences where polysyllabic words were either hyphenated at syllable boundaries, alternatingly coloured (every second syllable black, every second red) or had no explicit syllable boundary cues (e.g., ta-lo vs. talo vs. talo = “house”). The results showed that hyphenation at syllable boundaries slows down reading of first and second graders even though syllabification by hyphens is very common in Finnish reading instruction, as all first-grade textbooks include hyphens at syllable boundaries. When hyphens were positioned within a syllable (t-alo vs. ta-lo), beginning readers were even more disrupted. Alternate colouring did not affect reading speed, no matter whether colours signalled syllable structure or not. The results show that beginning Finnish readers prefer to process polysyllabic words via syllables rather than letter by letter. At the same time they imply that hyphenation encourages sequential syllable processing, which slows down the reading of children, who are already capable of parallel syllable processing or recognising words directly via the whole-word route.