Mid-level feature contributions to category-specific gaze guidance

Our research has previously shown that scene categories can be predicted from observers’ eye movements when they view photographs of real-world scenes. The time course of category predictions reveals the differential influences of bottom-up and top-down information. Here we used these known differences to determine to what extent image features at different representational levels contribute toward guiding gaze in a category-specific manner. Participants viewed grayscale photographs and line drawings of real-world scenes while their gaze was tracked. Scene categories could be predicted from fixation density at all times over a 2-s time course in both photographs and line drawings. We replicated the shape of the prediction curve found previously, with an initial steep decrease in prediction accuracy from 300 to 500 ms, representing the contribution of bottom-up information, followed by a steady increase, representing top-down knowledge of category-specific information. We then computed the low-level features (luminance contrasts and orientation statistics), mid-level features (local symmetry and contour junctions), and Deep Gaze II output from the images, and used that information as a reference in our category predictions in order to assess their respective contributions to category-specific guidance of gaze. We observed that, as expected, low-level salience contributes mostly to the initial bottom-up peak of gaze guidance. Conversely, the mid-level features that describe scene structure (i.e., local symmetry and junctions) split their contributions between bottom-up and top-down attentional guidance, with symmetry contributing to both bottom-up and top-down guidance, while junctions play a more prominent role in the top-down guidance of gaze.     

Musical instrument naming impairments: the crucial exception to the living/nonliving dichotomy in category-specific agnosia

In category-specific agnosia (CSA) patients typically have more trouble naming animals, fruits, and vegetables than tools, furniture, and articles of clothing. A crucial exception to this living vs nonliving rule involves the category of musical instruments. Patients with problems naming living objects often repeatedly fail to name musical instruments. In CSA it is crucial to equate living and nonliving object lists on object name frequency, complexity, and familiarity. The present study shows, however, that even the most rigorously controlled object lists can lead to erroneous conclusions if nonliving stimuli contain an overrepresentation of musical instruments. Naming capabilities of a herpes encephalitis patient were assessed using matched lists of living and nonliving objects and showed no indication of category-specific deficits. When exemplars were separated into biological objects, musical instruments and man-made artifacts, strong category-specificity emerged: artifact naming was flawless whereas musical instrument and biological object naming were both severely impaired. It is concluded that CSA is a veridical phenomenon but that our understanding of CSA is limited by adhering to the spurious living/nonliving distinction.     

Hierarchies, similarity, and interactivity in object recognition: "category-specific" neuropsychological deficits.

Category-specific impairments of object recognition and naming are among the most intriguing disorders in neuropsychology, affecting the retrieval of knowledge about either living or nonliving things. They can give us insight into the nature of our representations of objects: Have we evolved different neural systems for recognizing different categories of object? What kinds of knowledge are important for recognizing particular objects? How does visual similarity within a category influence object recognition and representation? What is the nature of our semantic knowledge about different objects? We review the evidence on category-specific impairments, arguing that deficits even for one class of object (e.g., living things) cannot be accounted for in terms of a single information processing disorder across all patients; problems arise at contrasting loci in different patients. The same apparent pattern of impairment can be produced by damage to different loci. According to a new processing framework for object recognition and naming, the hierarchical interactive theory (HIT), we have a hierarchy of highly interactive stored representations. HIT explains the variety of patients in terms of (1) lesions at different levels of processing and (2) different forms of stored knowledge used both for particular tasks and for particular categories of object.     

On the equivalence of superordinate concepts

Psychological studies of superordinates have generally treated them as equivalent. However, many languages distinguish mass superordinates (e.g., clothing) from count superordinates (e.g., vehicle). In the present paper, experimental evidence is presented which suggests that the two types of superordinates are conceptually distinct as well. One study showed that the members of mass superordinates more often co-occur. A second study showed that people more often interact with the members of mass superordinates in temporal proximity whereas people primarily interact with single members of count superordinates on a specific occasion. Also, properties that characterize an individual are a more salient aspect of count superordinates. These findings imply that mass superordinates refer to unindividuated groups of objects, united by spatial and functional contiguity. Two other studies supported this hypothesis by showing that the class inclusion relation between a single object and a category is stronger for count superordinates. Taken together, the findings suggest that mass superordinates are not true taxonomic categories. We relate the findings to previous views of superordinates and to the count/mass distinction in general.     

Object recognition and segmentation by a fragment-based hierarchy

How do we learn to recognize visual categories, such as dogs and cats? Somehow, the brain uses limited variable examples to extract the essential characteristics of new visual categories. Here, I describe an approach to category learning and recognition that is based on recent computational advances. In this approach, objects are represented by a hierarchy of fragments that are extracted during learning from observed examples. The fragments are class-specific features and are selected to deliver a high amount of information for categorization. The same fragments hierarchy is then used for general categorization, individual object recognition and object-parts identification. Recognition is also combined with object segmentation, using stored fragments, to provide a top-down process that delineates object boundaries in complex cluttered scenes. The approach is computationally effective and provides a possible framework for categorization, recognition and segmentation in human vision.     

Developmental changes in visual object recognition between 18 and 24 months of age

Two experiments examined developmental changes in children's visual recognition of common objects during the period of 18 to 24 months. Experiment 1 examined children's ability to recognize common category instances that presented three different kinds of information: (1) richly detailed and prototypical instances that presented both local and global shape information, color, textural and featural information, (2) the same rich and prototypical shapes but no color, texture or surface featural information, or (3) that presented only abstract and global representations of object shape in terms of geometric volumes. Significant developmental differences were observed only for the abstract shape representations in terms of geometric volumes, the kind of shape representation that has been hypothesized to underlie mature object recognition. Further, these differences were strongly linked in individual children to the number of object names in their productive vocabulary. Experiment 2 replicated these results and showed further that the less advanced children's object recognition was based on the piecemeal use of individual features and parts, rather than overall shape. The results provide further evidence for significant and rapid developmental changes in object recognition during the same period children first learn object names. The implications of the results for theories of visual object recognition, the relation of object recognition to category learning, and underlying developmental processes are discussed.     

Basic-level visual similarity and category specificity

The role of visual crowding in category deficits has been widely discussed (e.g.,;; ). Most studies have measured overlap at the superordinate level (compare different examples of 'animal') rather than at the basic level (compare different examples of 'dog'). In this study, we therefore derived two measures of basic-level overlap for a range of categories. The first was a computational measure generated by a self-organising neural network trained to process pictures of living and non-living things; the second was a rating of perceived visual similarity generated by human subjects to the item names. The computational measure indicated that the pattern of crowded/uncrowded does not honour a living/non-living distinction. Nevertheless, different superordinates showed varied degrees of basic-level overlap, suggesting that specific token choice affects some superordinates more than others e.g., individual fruit and vegetable tokens show greater variability than any other items, while tools and vehicles produce more reliable or overlapping basic-level visual representations. Finally, subject ratings correlated significantly with the computational measures indicating that the neural model represents structural properties of the objects that are psychologically meaningful.     

The impact of colour, spatial resolution, and presentation speed on category naming

Studies of neurological patients with category-specific agnosia have provided important contributions to our understanding of object recognition, although the meaning of such disorders is still hotly debated. One crucial line of research for our understanding of category effects, is through the examination of category biases in healthy normal subjects. This approach has, however, led to contradictory findings with advantages both for natural kinds and for man-made things being documented in healthy subjects. It has been proposed that task conditions may influence the direction of advantage (Gerlach, 2001) and in particular, that sub-optimal viewing conditions underpin natural kinds advantages, while man-made advantages emerge under more optimal viewing conditions. In two experiments with normal subjects, we examined the roles played by spatial resolution (blurring), stimulus type (colour and texture), and speed of presentation in picture naming across category. In both experiments, healthy subjects showed a natural kind advantage for original stimuli and for blurred colour stimuli (at slow and fast presentation speeds), while an advantage for man-made things emerged for line-drawings that were blurred and presented slowly. The implications for category-specific object recognition deficits are discussed.     

Trial-by-trial adjustments in control triggered by incidentally encoded semantic cues

Cognitive control mechanisms provide the flexibility to rapidly adapt to contextual demands. These contexts can be defined by top-down goals—but also by bottom-up perceptual factors, such as the location at which a visual stimulus appears. There are now several experiments reporting contextual control effects. Such experiments establish that contexts defined by low-level perceptual cues such as the location of a visual stimulus can lead to context-specific control, suggesting a relatively early focus for cognitive control. The current set of experiments involved a word–word interference task designed to assess whether a high-level cue, the semantic category to which a word belongs, can also facilitate contextual control. Indeed, participants exhibit a larger Flanker effect to items pertaining to a semantic category in which 75% of stimuli are incongruent than in response to items pertaining to a category in which 25% of stimuli are incongruent. Thus, both low-level and high-level stimulus features can affect the bottom-up engagement of cognitive control. The implications for current models of cognitive control are discussed.     

Category-specific visual agnosia: lesion to semantic memory versus extra-lesional variables in a case study and a connectionist model

There is a current debate on the causes of category-specific agnosia. The aim of this study was to examine the effects of lesional and extra-lesional variables on object recognition. Extra-lesional variables, such as visual complexity or familiarity, are factors that influence recognition. Using a connectionist model based on study, we provide evidence that extra-lesional variables can yield dissociations in the recognition rate of different categories. Furthermore, it is shown that lesional and extra-lesional variables can interact (p < .01) when both are simultaneously modeled. Category-specific agnosia might thus result from complex interactions.     

The role of line junctions in object recognition: The case of reading musical notation

Previous work has shown that line junctions are informative features for visual perception of objects, letters, and words. However, the sources of such sensitivity and their generalizability to other object categories are largely unclear. We addressed these questions by studying perceptual expertise in reading musical notation, a domain in which individuals with different levels of expertise are readily available. We observed that removing line junctions created by the contact between musical notes and staff lines selectively impaired recognition performance in experts and intermediate readers, but not in novices. The degree of performance impairment was predicted by individual fluency in reading musical notation. Our findings suggest that line junctions provide diagnostic information about object identity across various categories, including musical notation. However, human sensitivity to line junctions does not readily transfer from familiar to unfamiliar object categories, and has to be acquired through perceptual experience with the specific objects.     

Changes to the object recognition system in patients with dementia of the Alzheimer's type

Do DAT patients show category-specific deficits in object identification, and do they arise from semantic or visual damage? Participants decided whether line drawings of living and nonliving objects matched names at superordinate, basic, or subordinate levels. Patients were most impaired with superordinate decisions. Controls had most difficulty with subordinate decisions. No category-specific deficit was found with patients. Impaired superordinate decisions by the patients support semantic damage. If category-specific deficits arise from damaged semantics, they should have been found. Since they were not, and since patients performed subordinate decisions the best, a visual basis to category specificity is supported. Finally, a living advantage was found with normal observers which cannot be spurious due to differences in concept familiarity since living and nonliving objects were matched for this variable.     

The object advantage can be eliminated under equiluminant conditions

A key phenomenon supporting the existence of object-based attention is the object advantage, in which responses are faster for within-object, relative to equidistant between-object, shifts of attention. The origins of this effect have been variously ascribed to low-level “bottom-up” sensory processing and to a cognitive “top-down” strategy of within-object attention prioritization. The degree to which the object advantage depends on lower-level sensory processing was examined by differentially stimulating the magnocellular (M) and parvocellular (P) retino-geniculo-cortical visual pathways by using equiluminant and nonequiluminant conditions. We found that the object advantage can be eliminated when M activity is reduced using psychophysically equiluminant stimuli. This novel result in normal observers suggests that the origin of the object advantage is found in lower-level sensory processing rather than a general cognitive process, which should not be so sensitive to differential activation of the bottom-up P and M pathways. Eliminating the object advantage while maintaining a spatial-cueing advantage with reduced M activity suggests that the notion of independent M-driven spatial attention and P-driven object attention requires revision.     

Infants learn about objects from statistics and people

In laboratory experiments, infants are sensitive to patterns of visual features that co-occur (e.g., Fiser & Aslin, 2002). Once infants learn the statistical regularities, however, what do they do with that knowledge? Moreover, which patterns do infants learn in the cluttered world outside of the laboratory? Across 4 experiments, we show that 9-month-olds use this sensitivity to make inferences about object properties. In Experiment 1, 9-month-old infants expected co-occurring visual features to remain fused (i.e., infants looked longer when co-occurring features split apart than when they stayed together). Forming such expectations can help identify integral object parts for object individuation, recognition, and categorization. In Experiment 2, we increased the task difficulty by presenting the test stimuli simultaneously with a different spatial layout from the familiarization trials to provide a more ecologically valid condition. Infants did not make similar inferences in this more distracting test condition. However, Experiment 3 showed that a social cue did allow inferences in this more difficult test condition, and Experiment 4 showed that social cues helped infants choose patterns among distractor patterns during learning as well as during test. These findings suggest that infants can use feature co-occurrence to learn about objects and that social cues shape such foundational learning in distraction-filled environments.     

Somatosensory processes subserving perception and action

The functions of the somatosensory system are multiple. We use tactile input to localize and experience the various qualities of touch, and proprioceptive information to determine the position of different parts of the body with respect to each other, which provides fundamental information for action. Further, tactile exploration of the characteristics of external objects can result in conscious perceptual experience and stimulus or object recognition. Neuroanatomical studies suggest parallel processing as well as serial processing within the cerebral somatosensory system that reflect these separate functions, with one processing stream terminating in the posterior parietal cortex (PPC), and the other terminating in the insula. We suggest that, analogously to the organisation of the visual system, somatosensory processing for the guidance of action can be dissociated from the processing that leads to perception and memory. In addition, we find a second division between tactile information processing about external targets in service of object recognition and tactile information processing related to the body itself. We suggest the posterior parietal cortex subserves both perception and action, whereas the insula principally subserves perceptual recognition and learning.     

Spatial-frequency thresholds for object categorisation at basic and subordinate levels

In an attempt to understand how low-level visual information contributes to object categorisation, previous studies have examined the effects of spatially filtering images on object recognition at different levels of abstraction. Here, the quantitative thresholds for object categorisation at the basic and subordinate levels are determined by using a combination of the method of adjustment and a match-to-sample method. Participants were asked to adjust the cut-off of either a low-pass or high-pass filter applied to a target image until they reached the threshold at which they could match the target image to one of six simultaneously presented category names. This allowed more quantitative analysis of the spatial frequencies necessary for recognition than previous studies. Results indicate that a more central range of low spatial frequencies is necessary for subordinate categorisation than basic, though the difference is small, at about 0.25 octaves. Conversely, there was no effect of categorisation level on high-pass thresholds.     

View-invariant object category learning, recognition, and search: how spatial and object attention are coordinated using surface-based attentional shrouds

How does the brain learn to recognize an object from multiple viewpoints while scanning a scene with eye movements? How does the brain avoid the problem of erroneously classifying parts of different objects together? How are attention and eye movements intelligently coordinated to facilitate object learning? A neural model provides a unified mechanistic explanation of how spatial and object attention work together to search a scene and learn what is in it. The ARTSCAN model predicts how an object's surface representation generates a form-fitting distribution of spatial attention, or "attentional shroud". All surface representations dynamically compete for spatial attention to form a shroud. The winning shroud persists during active scanning of the object. The shroud maintains sustained activity of an emerging view-invariant category representation while multiple view-specific category representations are learned and are linked through associative learning to the view-invariant object category. The shroud also helps to restrict scanning eye movements to salient features on the attended object. Object attention plays a role in controlling and stabilizing the learning of view-specific object categories. Spatial attention hereby coordinates the deployment of object attention during object category learning. Shroud collapse releases a reset signal that inhibits the active view-invariant category in the What cortical processing stream. Then a new shroud, corresponding to a different object, forms in the Where cortical processing stream, and search using attention shifts and eye movements continues to learn new objects throughout a scene. The model mechanistically clarifies basic properties of attention shifts (engage, move, disengage) and inhibition of return. It simulates human reaction time data about object-based spatial attention shifts, and learns with 98.1% accuracy and a compression of 430 on a letter database whose letters vary in size, position, and orientation. The model provides a powerful framework for unifying many data about spatial and object attention, and their interactions during perception, cognition, and action.     

可操作性在物体表征中的作用

近10年来, 越来越多的研究表明, 对可操作物体的识别不仅依赖物体的视觉信息, 同时也依赖操作它的动作信息和知识, 即可操作性。来自行为实验、脑成像和脑损伤病人的研究都表明, 物体的可操作性在物体识别中会被激活, 并起着重要的作用。可操作性的研究不但重新解释了生物与非生物、名词与动词的分离现象, 而且对于研究物体表征, 以及视觉物体识别的神经通路有着重要的理论意义。