An expanded model for perceptual visual single object recognition system using expectation priming following neuroscientific evidence

Under numerous circumstances, humans recognize visual objects in their environment with remarkable response times and accuracy. Existing artificial visual object recognition systems have not yet surpassed human vision, especially in its universality of application. We argue that modeling the recognition process in an exclusive feedforward manner hinders those systems’ performance. To bridge that performance gap between them and human vision, we present a brief review of neuroscientific data, which suggests that considering an agent’s internal influences (from cognitive systems that peripherally interact with visual-perceptual processes) recognition can be improved. Then, we propose a model for visual object recognition which uses these systems’ information, such as affection, for generating expectation to prime the object recognition system, thus reducing its execution times. Later, an implementation of the model is described. Finally, we present and discuss an experiment and its results.     

Role of the secondary visual cortex in HMAX model for object recognition

The models inspired by visual systems of life creatures (e.g., human, mammals, etc.) have been very successful in addressing object recognition tasks. For example, Hierarchical Model And X (HMAX) effectively recognizes different objects by modeling the V1, V4, and IT regions of the human visual system. Although HMAX is one of the superior models in the field of object recognition, its implementation has been limited due to some disadvantages such as the unrepeatability of the process under constant conditions, extreme redundancy, high computational load, and time-consuming. In this paper, we aim at revising the HMAX approach by adding the model of the secondary region (V2) in the human visual system which leads to removing the mentioned drawbacks of standard HMAX. The added layer selects repeatable and more informative features that increase the accuracy of the proposed method by avoiding the redundancy existing in the conventional approaches. Furthermore, this feature selection strategy considerably reduces the huge computational load. Another contribution of our model is highlighted when a small number of training images is available where our model can efficiently cope with this issue. We evaluate our proposed approach using Caltech5 and GRAZ-02 database as two famous benchmarks for object recognition tasks. Additionally, the results are compared with standard HMAX that validate and highlight the efficiency of the proposed method.     

Attention to distinguishing features in object recognition

This study advances the hypothesis that, in the course of object recognition, attention is directed to distinguishing features: visual information that is diagnostic of object identity in a specific context. In five experiments, observers performed an object categorization task involving drawings of fish (Experiments 1–4) and photographs of natural sea animals (Experiment 5). Allocation of attention to distinguishing and non-distinguishing features was examined using primed-matching (Experiment 1) and visual probe (Experiments 2, 4, 5) methods, and manipulated by spatial precuing (Experiment 3). Converging results indicated that in performing the object categorization task, attention was allocated to the distinguishing features in a context-dependent manner, and that such allocation facilitated performance. Based on the view that object recognition, like categorization, is essentially a process of discrimination between probable alternatives, the implications of the findings for the role of attention to distinguishing features in object recognition are discussed.     

Dissociating the effects of angular disparity and image similarity in mental rotation and object recognition

Performance is often impaired linearly with increasing angular disparity between two objects in tasks that measure mental rotation or object recognition. But increased angular disparity is often accompanied by changes in the similarity between views of an object, confounding the impact of the two factors in these tasks. We examined separately the effects of angular disparity and image similarity on handedness (to test mental rotation) and identity (to test object recognition) judgments with 3-D novel objects. When similarity was approximately equated, an effect of angular disparity was only found for handedness but not identity judgments. With a fixed angular disparity, performance was better for similar than dissimilar image pairs in both tasks, with a larger effect for identity than handedness judgments. Our results suggest that mental rotation involves mental transformation procedures that depend on angular disparity, but that object recognition is predominately dependent on the similarity of image features.     

Negative priming of unattended part primes: Implications for models of holistic and analytic processing in object recognition

The “hybrid” model of object recognition (Hummel, 2001) proposes that unattended objects are processed holistically, while attended objects are processed both holistically and analytically. Supporting evidence for this claim was reported by Thoma, Hummel, and Davidoff (2004) who showed that, unlike whole object primes, unattended split object parts (presumed to require analytic processing) do not elicit repetition priming. Here we tested the generality of this finding by contrasting priming for whole and part prime stimuli as a function of prime informativeness and by modifying the design so that both unattended whole and part prime displays contained a single perceptual object. Unlike Thoma et al. (2004) the results showed negative (rather than an absence of) priming for unattended half object primes. These findings place new constraints on theoretical models of the role of attention in object recognition.     

The effect of body and part-based motion on the recognition of unfamiliar objects

We investigated the role of global (body) and local (parts) motion on the recognition of unfamiliar objects. Participants were trained to categorise moving objects and were then tested on their recognition of static images of these targets using a priming paradigm. Each static target shape was primed by a moving object that comprised either the same body and parts motion; same body, different parts motion; different body, same part motion as the learned target or was non-moving. Only the same body but not the same part motion facilitated shape recognition (Experiment 1), even when either motion was diagnostic of object identity (Experiment 2). When parts motion was more related to the object's body motion then it facilitated the recognition of the static target (Experiment 3). Our results suggest that global and local motions are independently accessed during object recognition and have important implications for how objects are represented in memory.     

The role of surface-based representations of shape in visual object recognition

This study contrasted the role of surfaces and volumetric shape primitives in three-dimensional object recognition. Observers (N?=?50) matched subsets of closed contour fragments, surfaces, or volumetric parts to whole novel objects during a whole–part matching task. Three factors were further manipulated: part viewpoint (either same or different between component parts and whole objects), surface occlusion (comparison parts contained either visible surfaces only, or a surface that was fully or partially occluded in the whole object), and target–distractor similarity. Similarity was varied in terms of systematic variation in nonaccidental (NAP) or metric (MP) properties of individual parts. Analysis of sensitivity (d′) showed a whole–part matching advantage for surface-based parts and volumes over closed contour fragments—but no benefit for volumetric parts over surfaces. We also found a performance cost in matching volumetric parts to wholes when the volumes showed surfaces that were occluded in the whole object. The same pattern was found for both same and different viewpoints, and regardless of target–distractor similarity. These findings challenge models in which recognition is mediated by volumetric part-based shape representations. Instead, we argue that the results are consistent with a surface-based model of high-level shape representation for recognition.     

Distorting Face Representations in Newborn Brains

What role does experience play in the development of face recognition? A growing body of evidence indicates that newborn brains need slowly changing visual experiences to develop accurate visual recognition abilities. All of the work supporting this “slowness constraint” on visual development comes from studies testing basic-level object recognition. Here, we present the results of controlled-rearing experiments that provide evidence for a slowness constraint on the development of face recognition, a prototypical subordinate-level object recognition task. We found that (1) newborn chicks can rapidly develop view-invariant face recognition and (2) the development of this ability relies on experience with slowly moving faces. When chicks were reared with quickly moving faces, they built distorted face representations that largely lacked invariance to viewpoint changes, effectively “breaking” their face recognition abilities. These results provide causal evidence that slowly changing visual experiences play a critical role in the development of face recognition, akin to basic-level object recognition. Thus, face recognition is not a hardwired property of vision but is learned rapidly as the visual system adapts to the temporal structure of the animal's visual environment.     

Stereo disparity facilitates view generalization during shape recognition for solid multipart objects

Current theories of object recognition in human vision make different predictions about whether the recognition of complex, multipart objects should be influenced by shape information about surface depth orientation and curvature derived from stereo disparity. We examined this issue in five experiments using a recognition memory paradigm in which observers (N = 134) memorized and then discriminated sets of 3D novel objects at trained and untrained viewpoints under either mono or stereo viewing conditions. In order to explore the conditions under which stereo-defined shape information contributes to object recognition we systematically varied the difficulty of view generalization by increasing the angular disparity between trained and untrained views. In one series of experiments, objects were presented from either previously trained views or untrained views rotated (15°, 30°, or 60°) along the same plane. In separate experiments we examined whether view generalization effects interacted with the vertical or horizontal plane of object rotation across 40° viewpoint changes. The results showed robust viewpoint-dependent performance costs: Observers were more efficient in recognizing learned objects from trained than from untrained views, and recognition was worse for extrapolated than for interpolated untrained views. We also found that performance was enhanced by stereo viewing but only at larger angular disparities between trained and untrained views. These findings show that object recognition is not based solely on 2D image information but that it can be facilitated by shape information derived from stereo disparity.     

Conceptual knowledge attenuates viewpoint dependency in visual object recognition

Learning verbal semantic knowledge for objects has been shown to attenuate recognition costs incurred by changes in view from a learned viewpoint. Such findings were attributed to the semantic or meaningful nature of the learned verbal associations. However, recent findings demonstrate surprising benefits to visual perception after learning even noninformative verbal labels for stimuli. Here we test whether learning verbal information for novel objects, independent of its semantic nature, can facilitate a reduction in viewpoint-dependent recognition. To dissociate more general effects of verbal associations from those stemming from the semantic nature of the associations, participants learned to associate semantically meaningful (adjectives) or nonmeaningful (number codes) verbal information with novel objects. Consistent with a role of semantic representations in attenuating the viewpoint-dependent nature of object recognition, the costs incurred by a change in viewpoint were attenuated for stimuli with learned semantic associations relative to those associated with nonmeaningful verbal information. This finding is discussed in terms of its implications for understanding basic mechanisms of object perception as well as the classic viewpoint-dependent nature of object recognition.     

No strong evidence for lateralisation of word reading and face recognition deficits following posterior brain injury

Face recognition and word reading are thought to be mediated by relatively independent cognitive systems lateralised to the right and left hemispheres, respectively. In this case, we should expect a higher incidence of face recognition problems in patients with right hemisphere injury and a higher incidence of reading problems in patients with left hemisphere injury. We tested this hypothesis in a group of 31 patients with unilateral right or left hemisphere infarcts in the territory of the posterior cerebral arteries. In most domains tested (e.g., visual attention, object recognition, visuo-construction, motion perception), we found that both patient groups performed significantly worse than a matched control group. In particular, we found a significant number of face recognition deficits in patients with left hemisphere injury and a significant number of patients with word reading deficits following right hemisphere injury. This suggests that face recognition and word reading may be mediated by more bilaterally distributed neural systems than is commonly assumed.     

Object recognition with severe spatial deficits in Williams syndrome: sparing and breakdown

Williams syndrome (WS) is a rare genetic disorder that results in severe visual-spatial cognitive deficits coupled with relative sparing in language, face recognition, and certain aspects of motion processing. Here, we look for evidence for sparing or impairment in another cognitive system-object recognition. Children with WS, normal mental-age (MA) and chronological age-matched (CA) children, and normal adults viewed pictures of a large range of objects briefly presented under various conditions of degradation, including canonical and unusual orientations, and clear or blurred contours. Objects were shown as either full-color views (Experiment 1) or line drawings (Experiment 2). Across both experiments, WS and MA children performed similarly in all conditions while CA children performed better than both WS group and MA groups with unusual views. This advantage, however, was eliminated when images were also blurred. The error types and relative difficulty of different objects were similar across all participant groups. The results indicate selective sparing of basic mechanisms of object recognition in WS, together with developmental delay or arrest in recognition of objects from unusual viewpoints. These findings are consistent with the growing literature on brain abnormalities in WS which points to selective impairment in the parietal areas of the brain. As a whole, the results lend further support to the growing literature on the functional separability of object recognition mechanisms from other spatial functions, and raise intriguing questions about the link between genetic deficits and cognition.     

Orientation-invariant object recognition: evidence from repetition blindness

The question of whether object recognition is orientation-invariant or orientation-dependent was investigated using a repetition blindness (RB) paradigm. In RB, the second occurrence of a repeated stimulus is less likely to be reported, compared to the occurrence of a different stimulus, if it occurs within a short time of the first presentation. This failure is usually interpreted as a difficulty in assigning two separate episodic tokens to the same visual type. Thus, RB can provide useful information about which representations are treated as the same by the visual system. Two experiments tested whether RB occurs for repeated objects that were either in identical orientations, or differed by 30, 60, 90, or 180 degrees . Significant RB was found for all orientation differences, consistent with the existence of orientation-invariant object representations. However, under some circumstances, RB was reduced or even eliminated when the repeated object was rotated by 180 degrees , suggesting easier individuation of the repeated objects in this case. A third experiment confirmed that the upside-down orientation is processed more easily than other rotated orientations. The results indicate that, although object identity can be determined independently of orientation, orientation plays an important role in establishing distinct episodic representations of a repeated object, thus enabling one to report them as separate events.     

Object affordance based multimodal fusion for natural Human-Robot interaction

Spoken language based natural Human-Robot Interaction (HRI) requires robots to have the ability to understand spoken language, and extract the intention-related information from the working scenario. For grounding the intention-related object in the working environment, object affordance recognition could be a feasible way. To this end, we propose a dataset and a deep CNN based architecture to learn the human-centered object affordance. Furthermore, we present an affordance based multimodal fusion framework to realize intended object grasping according to the spoken instructions of human users. The proposed framework contains an intention semantics extraction module which is employed to extract the intention from spoken language, a deep Convolutional Neural Network (CNN) based object affordance recognition module which is applied to recognize human-centered object affordance, and a multimodal fusion module which is adopted to bridge the extracted intentions and the recognized object affordances. We also complete multiple intended object grasping experiments on a PR2 platform to validate the feasibility and practicability of the presented HRI framework.     

Action alters shape categories

Two experiments show that action alters the shape categories formed by 2-year-olds. Experiment 1 shows that moving an object horizontally (or vertically) defines the horizontal (or vertical) axis as the main axis of elongation and systematically changes the range of shapes seen as similar. Experiment 2 shows that moving an object symmetrically (or asymmetrically) also alters shape categories. Previous work has shown marked developmental changes in object recognition between 1 and 3 years of age. These results suggest a role for action in this developmental process.     

Neural adaptation across viewpoint and exemplar in fusiform cortex

The visual system has the remarkable ability to generalize across different viewpoints and exemplars to recognize abstract categories of objects, and to discriminate between different viewpoints and exemplars to recognize specific instances of particular objects. Behavioral experiments indicate the critical role of the right hemisphere in specific-viewpoint and -exemplar visual form processing and the left hemisphere in abstract-viewpoint and -exemplar visual form processing. Neuroimaging studies indicate the role of fusiform cortex in these processes, however results conflict in their support of the behavioral findings. We investigated this inconsistency in the present study by examining adaptation across viewpoint and exemplar changes in the functionally defined fusiform face area (FFA) and in fusiform regions exhibiting adaptation. Subjects were adapted to particular views of common objects and then tested with objects appearing in four critical conditions: same-exemplar, same-viewpoint adapted, same-exemplar, different-viewpoint adapted, different-exemplar adapted, and not adapted. In line with previous results, the FFA demonstrated a release from neural adaptation for repeated different viewpoints and exemplars of an object. In contrast to previous work, a (non-FFA) right medial fusiform area also demonstrated a release from neural adaptation for repeated different viewpoints and exemplars of an object. Finally, a left lateral fusiform area demonstrated neural adaptation for repeated different viewpoints, but not exemplars, of an object. Test-phase task demands did not affect adaptation in these regions. Together, results suggest that dissociable neural subsystems in fusiform cortex support the specific identification of a particular object and the abstract recognition of that object observed from a different viewpoint. In addition, results suggest that areas within fusiform cortex do not support abstract recognition of different exemplars of objects within a category.     

The relationship between object files and conscious perception

Object files (OFs) are hypothesized mid-level representations which mediate our conscious perception of persisting objects-e.g. telling us 'which went where'. Despite the appeal of the OF framework, not previous research has directly explored whether OFs do indeed correspond to conscious percepts. Here we present at least one case wherein conscious percepts of 'which went where' in dynamic ambiguous displays diverge from the analogous correspondence computed by the OF system. Observers viewed a 'bouncing/streaming' display in which two identical objects moved such that they could have either bounced off or streamed past each other. We measured two dependent variables: (1) an explicit report of perceived bouncing or streaming; and (2) an implicit 'object-specific preview benefit' (OSPB), wherein a 'preview' of information on a specific object speeds the recognition of that information at a later point when it appears again on the same object (compared to when it reappears on a different object), beyond display-wide priming. When the displays were manipulated such that observers had a strong bias to perceive streaming (on over 95% of the trials), there was nevertheless a strong OSPB in the opposite direction-such that the object files appeared to have 'bounced' even though the percept 'streamed'. Given that OSPBs have been taken as a hallmark of the operation of object files, the five experiments reported here suggest that in at least some specialized (and perhaps ecologically invalid) cases, conscious percepts of 'which went where' in dynamic ambiguous displays can diverge from the mapping computed by the object-file system.     

Combining disparate views of objects: Viewpoint costs are reduced by stereopsis

An issue of central concern in the object recognition literature is whether changes in the viewpoint from which an object is depicted produces systematic costs in performance, or whether performance is (largely) unaffected by such changes. This issue has generated a vigorous and lengthy debate because viewpoint-dependent or viewpoint-independent performance has been seen as a reflection of the underlying object representations. The current experiment shows that the effect of viewpoint differences between objects is strongly affected by whether or not they are depicted with stereoscopic depth, a result that is predicted by neither of the main approaches to object recognition. Instead, it is proposed that viewpoint costs in object recognition experiments are a function of the extent to which the information a subject is provided with generalizes across views, without this holding any necessary implications for the nature of the underlying object representations.     

Are objects required for object-files? Roles of segmentation and spatiotemporal continuity in computing object persistence

Two central tasks of visual processing are (1) to segment undifferentiated retinal images into discrete objects, and (2) to represent those objects as the same persisting individuals over time and motion. Here we explore the interaction of these two types of processing in the context of object files—mid-level visual representations that “stick” to moving objects on the basis of spatiotemporal properties. Object files can be revealed by object-specific preview benefits (OSPBs), wherein a “preview” of information on a moving object speeds the recognition of that information at a later point when it appears again on the same object (compared to when it reappears on a different moving object), beyond display-wide priming. Here we explore the degree of segmentation required to establish object files in the first place. Surprisingly, we find that no explicit segmentation is required until after the previews disappear, when using purely motion-defined objects (consisting of random elements on a random background). Moreover, OSPBs are observed in such displays even after moderate (but not long) delays between the offset of the preview information and the onset of the motion. These effects indicate that object files can be established without initial static segmentation cues, so long as there is spatiotemporal continuity between the previews and the eventual appearance of the objects. We also find that top-down strategies can sometimes mimic OSPBs, but that these strategies can be eliminated by novel manipulations. We discuss how these results alter our understanding of the nature of object files, and also why researchers must take care to distinguish “true OSPBs” from “illusory OSPBs”.