What is a visual object? Evidence from target merging in multiple object tracking

The notion that visual attention can operate over visual objects in addition to spatial locations has recently received much empirical support, but there has been relatively little empirical consideration of what can count as an 'object' in the first place. We have investigated this question in the context of the multiple object tracking paradigm, in which subjects must track a number of independently and unpredictably moving identical items in a field of identical distractors. What types of feature clusters can be tracked in this manner? In other words, what counts as an 'object' in this task? We investigated this question with a technique we call target merging: we alter tracking displays so that distinct target and distractor locations appear perceptually to be parts of the same object by merging pairs of items (one target with one distractor) in various ways - for example, by connecting item locations with a simple line segment, by drawing the convex hull of the two items, and so forth. The data show that target merging makes the tracking task far more difficult to varying degrees depending on exactly how the items are merged. The effect is perceptually salient, involving in some conditions a total destruction of subjects' capacity to track multiple items. These studies provide strong evidence for the object-based nature of tracking, confirming that in some contexts attention must be allocated to objects rather than arbitrary collections of features. In addition, the results begin to reveal the types of spatially organized scene components that can be independently attended as a function of properties such as connectedness, part structure, and other types of perceptual grouping.     

Is vision continuous with cognition? The case for cognitive impenetrability of visual perception

Although the study of visual perception has made more progress in the past 40 years than any other area of cognitive science, there remain major disagreements as to how closely vision is tied to cognition. This target article sets out some of the arguments for both sides (arguments from computer vision, neuroscience, psychophysics, perceptual learning, and other areas of vision science) and defends the position that an important part of visual perception, corresponding to what some people have called early vision, is prohibited from accessing relevant expectations, knowledge, and utilities in determining the function it computes--in other words, it is cognitively impenetrable. That part of vision is complex and involves top-down interactions that are internal to the early vision system. Its function is to provide a structured representation of the 3-D surfaces of objects sufficient to serve as an index into memory, with somewhat different outputs being made available to other systems such as those dealing with motor control. The paper also addresses certain conceptual and methodological issues raised by this claim, such as whether signal detection theory and event-related potentials can be used to assess cognitive penetration of vision. A distinction is made among several stages in visual processing, including, in addition to the inflexible early-vision stage, a pre-perceptual attention-allocation stage and a post-perceptual evaluation, selection, and inference stage, which accesses long-term memory. These two stages provide the primary ways in which cognition can affect the outcome of visual perception. The paper discusses arguments from computer vision and psychology showing that vision is "intelligent" and involves elements of "problem solving." The cases of apparently intelligent interpretation sometimes cited in support of this claim do not show cognitive penetration; rather, they show that certain natural constraints on interpretation, concerned primarily with optical and geometrical properties of the world, have been compiled into the visual system. The paper also examines a number of examples where instructions and "hints" are alleged to affect what is seen. In each case it is concluded that the evidence is more readily assimilated to the view that when cognitive effects are found, they have a locus outside early vision, in such processes as the allocation of focal attention and the identification of the stimulus.     

Some puzzling findings in multiple object tracking (MOT): II. Inhibition of moving nontargets

We present three studies examining whether multiple object tracking (MOT) benefits from the active inhibition of nontargets, as proposed in Pylyshyn (2004, Visual Cognition). Using a probe-dot technique, the first study showed poorer probe detection on nontargets than on either the targets being tracked or in the empty space between objects. The second study used a matching nontracking task to control for possible masking of probes, independent of target tracking. The third study examined how localized the inhibition is to individual nontargets. The result of these three studies led to the conclusion that nontargets are subject to a highly localized object-based inhibition. Implications of this finding for the FINST visual index theory are discussed. We suggest that we need to distinguish between the differentiation (or individuation) of enduring token objects and the process of making the objects accessible through indexes, with only the latter being limited to 4 or 5 objects.     

Enumerating by pointing to locations: a new method for measuring the numerosity of visual object representations

The fast and accurate enumeration of a small set of objects, called subitizing, is thought to involve a different mechanism from other numerosity judgments, such as those based on estimation. In this report, we examine the subitizing limit using a novel enumeration task that obtained the perceived locations of enumerated objects. Observers were shown brief masked displays (50, 200, and 350 ms) of 2-9 small black discs randomly placed on a gray screen and then asked to place a marker where each disc had been located. The number of these markers provided an estimate of the number of items processed. This "pointing" methodology enabled observers to accurately "enumerate" displays containing up to six items in contrast with the four-item limit typically found when using standard reporting methods (and replicated here in Experiment 2). These results suggest a different account of the limits found in most subitizing and enumeration studies.     

Situating vision in the world

Recently, there has been a great deal of interest in what has been called 'situated cognition', which has included claims that certain forms of representation are inadequate for modeling active organisms or agents such as humans and robots. In this article, I suggest that a weakness in classical theories of visual representation is the way in which representations connect with the real world, which may account for many of the concerns expressed by the situated cognition community. Specifically, I claim that what current theories lack is any provision for a certain form of direct, preconceptual connection between objects in the visual world (visual objects or proto-objects) and their representations in the visual system. This type of connection is akin to what philosophers and semanticists have referred to as an 'indexical' or 'demonstrative' reference and what some cognitive scientists have referred to as 'deictic pointers'. I explain why such a mechanism is needed and suggest that many workers have, in fact, been studying precisely this under the term 'visual index'. The visual index hypothesis is illustrated with the results of some relevant experiments, including multiple object tracking, visual routines and subset-selected visual searches. Indexing theory provides a synthesis that has profound implications for explaining a wide range of psychophysical findings, certain results in infant cognitive development and also some ancient problems in the philosophy of mind.