A sensorimotor account of vision and visual consciousness

Many current neurophysiological, psychophysical, and psychological approaches to vision rest on the idea that when we see, the brain produces an internal representation of the world. The activation of this internal representation is assumed to give rise to the experience of seeing. The problem with this kind of approach is that it leaves unexplained how the existence of such a detailed internal representation might produce visual consciousness. An alternative proposal is made here. We propose that seeing is a way of acting. It is a particular way of exploring the environment. Activity in internal representations does not generate the experience of seeing. The outside world serves as its own, external, representation. The experience of seeing occurs when the organism masters what we call the governing laws of sensorimotor contingency. The advantage of this approach is that it provides a natural and principled way of accounting for visual consciousness, and for the differences in the perceived quality of sensory experience in the different sensory modalities. Several lines of empirical evidence are brought forward in support of the theory, in particular: evidence from experiments in sensorimotor adaptation, visual "filling in," visual stability despite eye movements, change blindness, sensory substitution, and color perception.     

Characterisation of the misalignment and misangulation components in the Poggendorff and corner-Poggendorff illusions

In the Poggendorff illusion, two colinear segments abutting obliquely on an intervening configuration (often consisting of two long parallel lines) appear misaligned. We report here the results of a component analysis of the illusion and several of its variants, including in particular the 'corner-Poggendorff' illusion, and variants with a single arm. Using a nulling method, we determined an 'orientation profile' of each configuration, that is, how the illusions varied as the configuration was rotated in the plane of the display. We were able to characterise a pure-misalignment component (having peaks and dips around the +/- 22.5 degrees and +/- 67.5 degrees orientations of the arms) and a pure misangulation component of constant sign, having peaks at the +/- 45 degrees orientations of the arms. Both these components were present in both the classic and the corner-Poggendorff configurations. Thus, the misangulation component appears clearly in the classic Poggendorff illusion, once the misalignment component is partitioned out. Similarly, the corner-Poggendorff configuration, which essentially estimates a misangulation component, contains a misalignment component which becomes apparent once the misangulation is nulled. While our analysis accounts for much of the variability in the shapes of the profiles, additional assumptions must be made to explain the relatively small misangulation measured in the corner-Poggendorff configuration (1.5 degrees, on average, at peak value), and the relatively large illusion measured in the configurations with a single arm (above 6 degrees, on average, at peak values). We invoke the notion that parallelism and colinearity detectors provide counteracting cues, the first class reducing misangulation in the corner-Poggendorff configuration, and the second class reducing the illusion in the Poggendorff configurations with two arms.     

Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with 'the vOICe'

We investigated to what extent participants can acquire the mastery of an auditory-substitution-of-vision device ('The vOICe') using dynamic tasks in a three-dimensional environment. After extensive training, participants took part in four experiments. In the first experiment we explored locomotion and localisation abilities. Participants, blindfolded and equipped with the device, had to localise a target by moving a hand-held camera, walk towards the target, and point at it. In the second experiment, we studied the localisation ability in a constrained pointing task. In the third experiment we explored participants' ability to recognise natural objects via their auditory rendering. In the fourth experiment we tested the ability of participants to discriminate objects belonging to the same category. We analysed participants' performance from both an objective and a subjective point of view. The results showed that participants, through sensorimotor interactions with the perceptual scene while using the hand-held camera, were able to make use of the auditory stimulation to obtain the information necessary for locomotor guidance, localisation, and pointing, as well as for object recognition. Furthermore, analysis from a subjective perspective yielded insights into participants' qualitative experience and into the strategies they used to master the device, and thus to pass from a kind of deductive reasoning to a form of immediate apprehension of what is being perceived.     

The perception of space and form recognition in a simulated environment: the case of minimalist sensory-substitution devices

Whenever we explore a simulated environment, the sensorimotor interactions that underlie our perception of space may be modified. We investigated the conditions under which it is possible to acquire the mastery of new sensorimotor laws and thereby to infer new perceptual spaces. A computer interface, based on the principles of minimalist sensory-substitution devices, was designed to enable different possible links between a user's actions (manipulation of a mouse and/or keys of a keyboard) and the resulting pattern of sensory stimulation (visual or auditory) to be established. The interface generated an all-or-none stimulus whose activation varied as a function of the participant's exploration of a hidden form. In this study we addressed the following questions: What are the conditions necessary for participants to understand their actions as constituting a displacement in a simulated space? What are the conditions required for participants to conceive of sensations as originating from the encounter with an object situated in this space? Finally, what are the conditions required for participants to recognise forms within this space? The results of the two experiments reported here show that, under certain conditions, participants can interpret the new sensorimotor laws as movements in a new perceptual space and can recognise simple geometric forms, and that this occurs no matter whether the sensory stimulation is presented in the visual or auditory modality.     

Adaptation and communicative design.

Two theoretical frameworks that examine the nature of adaptability and mutual influence in interaction, interpersonal deception theory and interaction adaptation theory, were used to derive hypotheses concerning patterns of interaction that occur across time in truthful and deceptive conversations. Two studies were conducted in which senders were either truthful or deceptive in their interactions with a partner who increased or decreased involvement during the latter half of the conversation. Results revealed that deceivers felt more anxious and were more concerned about self‐presentation than truthtellers prior to the interaction and displayed less initial involvement than truthtellers. Patterns of interaction were also moderated by deception. Deceivers increased involvement over time but also reciprocated increases or decreases in receiver involvement. However, deceivers were less responsive than truthtellers to changes in receiver behavior. Finally, partner involvement served as feedback to senders regarding their own performance.     

Neighborhood frequency effects and letter visibility in visual word recognition.

Two experiments are described that measured lexical decision latencies and errors to five-letter French words with a single higher frequency orthographic neighbor and control words with no higher frequency neighbors. The higher frequency neighbor differed from the stimulus word by either the second letter (e.g., ASTRE-AUTRE) or the fourth letter (CHOPE-CHOSE). Neighborhood frequency effects were found to interact with this factor, and significant interference was observed only to CHOPE-type words. The effects of neighborhood frequency were also found to interact with the position of initial fixation in the stimulus word (either the second letter or the fourth letter). Interference was greatly reduced when the initial fixation was on the critical disambiguating letter (i.e., the letter P in CHOPE). Moreover, word recognition was improved when subjects initially fixated the second letter relative to when they initially fixated the fourth letter of a five-letter word, but this second-letter advantage practically disappeared when the stimulus differed from a more frequent word by its fourth letter. The results are interpreted in terms of the interaction between visual and lexical factors in visual work recognition.     

TO SEE OR NOT TO SEE:

Abstract— When looking at a scene, observers feel that they see its entire structure in great detail and can immediately notice any changes in it. However, when brief blank fields are placed between alternating displays of an original and a modified scene, a striking failure of perception is induced Identification of changes becomes extremely difficult, even when changes are large and made repeatedly. Identification is much faster when a verbal cue is provided showing that poor visibility is not the cause of this difficulty. Identification is also faster for objects considered to be important in the scene. These results support the idea that observers never form a complete, detailed representation of their surroundings. In addition, the results indicate that attention is required to perceive change, and the basis of high-level interest.