Assessment of sensory substitution prosthesis potentialities in minimalist conditions of learning

Pattern recognition with a prosthesis substituting vision by audition was investigated. During 15 1‐hour sessions, nine blindfolded sighted subjects were trained to recognise 2D patterns by trial and error. In addition to a global assessment, recognition of pattern element nature (vertical bars, horizontal bars…), element size and element spatial arrangement were independently assessed for each pattern. Influence of experimental parameters (complexity level of patterns, exploration number of a pattern) on recognition was studied. Performances improved over sessions. As a rule, patterns element nature was less well recognised than element size and spatial arrangement. Experimental parameters influenced pattern recognition performance. Results are discussed in relation with auditory and visual perception as well as in the perspective to implement a learning protocol for future users of prosthesis. Copyright © 2006 John Wiley & Sons, Ltd.     

Tactile memory in sighted and blind observers: the influence of orientation and rate of presentation

Sighted, early blind, and late blind subjects attempted to identify numerals or number sequences printed on their palms. The numerals were either upright, or inverted, or rotated perpendicular to the arm axis. Stimulus rotation degraded recognition in the early blind subjects, suggesting the influence of experience with visual frames of reference. Slower rates of presentation with upright number sequences improved recall in both sighted and blind observers. An experiment on tactual-visual braille recognition in the sighted observers showed that tilt degraded pattern identification, but visual guidance of the fingertip and ballpoint minimized this loss. A further experiment was performed to distinguish between visual imagery and visual frame of reference explanations of the visual guidance effect on recognition of rotated braille. Subjects explored upright or tilted braille characters while viewing only a light emitting diode on the exploratory fingertip. Sight of scanning movements did not aid pattern recognition with tilt. The results indicate that the benefits of visual guidance on recognition of tilted patterns were probably due to frame of reference information. It is concluded that spatial reference information may aid tactile memory in the sighted and late blind, since the early blind performed at a lower level in the retention task. It is proposed that visual imagery may only explain the superiority of the sighted and late blind when familiar stimuli are studied.     

盲生错误记忆和真实记忆研究

错误记忆指的是人们会回忆或再认那些没有出现过的事件。目前,针对错误记忆的研究多集中于视力正常的人群。国外近年来有针对视障人群错误记忆的研究认为,在听觉学习情境中视障成人有更好的真实记忆和更低的错误记忆。目前对于盲生的错误记忆特征尚缺乏研究。而盲文摸读作为视障人群重要的学习方式,其对视障人群的真实记忆和错误记忆的影响尚未有研究者关注。本研究以初中盲生和视力正常学生作为研究对象,采用DRM范式,发现被试采用阅读学习(盲生摸读盲文,视力正常学生阅读屏幕文字)和听觉学习两种学习方式下,都发现被试身上存在稳定的错误记忆;盲生的错误记忆显著低于视力正常学生。在摸读学习方式下,盲生真实记忆好于听觉学习方式。     

Tactual perception of embossed Morse code and braille: the alliance of vision and touch

In three experiments observers made visual matches to tangible embossed patterns. Stained glass was used to blur vision and thus allow the effect of visual guidance of tactual exploration on the accuracy of symbol recognition to be evaluated. Stained glass rendered the embossed code invisible, but allowed sight of the hand. In the first experiment subjects identified patterns made up of dots and dashes drawn from Morse code; in the second and third experiments they studied braille. The results show that subjects are more accurate in 'reading' tangible codes when provided with visual guidance. Performance was higher for braille than for Morse code. Vision aided touch through the provision of a frame of reference and through sight of scanning movements. Naive sighted observers were able to identify invisible braille dots by watching other individuals touch the symbols, suggesting the importance of vision of kinesthetic patterns.     

Spatial representation by blind and sighted children

Spatial representation by 72 blind and blindfolded sighted children between the ages of 6 and 11 was tested in two experiments by mental rotation of a raised line under conditions of clockwise varied directions.Experiment 1 showed that the two groups were well matched on tactual recognition and scored equally badly on matching displays to their own mentally rotated position.Experiment 2 found the sighted superior in recall tests. There was a highly significant interaction between sighted status and degree of rotation. Degree of rotation affected only the blind. Their scores were significantly lower for rotating to oblique and to the far orthogonal directions than to near orthogonal test positions. On near orthogonals the blind did not differ from the sighted.Age was a main effect, but it did not interact with any other variable. Older blind children whose visual experience dated from before the age of 6 were superior to congenitally blind subjects, but not differentially more so on oblique directions.The results were discussed in relation to hypotheses about the nature of spatial representation and strategies by children whose prior experience derived from vision or from touch and movement.     

Imagery in the congenitally blind: how visual are visual images?

Three experiments compared congenitally blind and sighted adults and children on tasks presumed to involve visual imagery in memory. In all three, the blind subjects' performances were remarkably similar to the sighted. The first two experiments examined Paivio's (1971) modality-specific imagery hypothesis. Experiment 1 used a paired-associate task with words whose referents were high in either visual or auditory imagery. The blind, like the sighted, recalled more high-visual-imagery pairs than any others. Experiment 2 used a free-recall task for words grouped according to modality-specific attributes, such as color and sound. The blind performed as well as the sighted on words grouped by color. In fact, the only consistent deficit in both experiments occurred for the sighted in recall of words whose referents are primarily auditory. These results challenge Paivio's theory and suggest either (a) that the visual imagery used by the sighted is no more facilitating than the abstract semantic representations used by the blind or (b) that the sighted are not using visual imagery. Experiment 3 used Neisser and Kerr's (1973) imaging task. Subjects formed images of scenes in which target objects were described as either visible in the picture plane or concealed by another object and thus not visible. On an incidental recall test for the target objects, the blind, like the sighted, recalled more pictorial than concealed targets. This finding suggests that the haptic images of the blind maintain occlusion just as the visual images of the sighted do.     

Perspective taking, pictures, and the blind

Congenitally blind, late blind, and blindfolded sighted controls attempted a Piagetian perspective-taking (three-mountain) task. Piaget used the term perspective to mean point of view (Piaget & Inhelder, 1967, p. 210), and the present usage does not imply linear perspective. Subjects used raised-line drawings to depict alternative points of view of an array of three geometric solid forms (cube, cone, and ball). They then identified the point of view of raised-line drawings. The effect of visual status on accuracy was nonsignificant for both response measures. Using alternating vision of the array and drawings, sighted subjects in a control condition performed like the congenitally blind. However, congenitally blind individuals did require more time than the other subjects for the perspective-taking task. In an additional experiment, no difference was found between the three groups in the accuracy or speed of tactile shape matching. The results suggest that visual imagery and visual experience are not necessary for tactile perspective taking.     

Late‐ but not early‐onset blindness impairs the development of audio‐haptic multisensory integration

Integrating different senses to reduce sensory uncertainty and increase perceptual precision can have an important compensatory function for individuals with visual impairment and blindness. However, how visual impairment and blindness impact the development of optimal multisensory integration in the remaining senses is currently unknown. Here we first examined how audio‐haptic integration develops and changes across the life span in 92 sighted (blindfolded) individuals between 7 and 70 years of age. We used a child‐friendly task in which participants had to discriminate different object sizes by touching them and/or listening to them. We assessed whether audio‐haptic performance resulted in a reduction of perceptual uncertainty compared to auditory‐only and haptic‐only performance as predicted by maximum‐likelihood estimation model. We then compared how this ability develops in 28 children and adults with different levels of visual experience, focussing on low‐vision individuals and blind individuals that lost their sight at different ages during development. Our results show that in sighted individuals, adult‐like audio‐haptic integration develops around 13–15 years of age, and remains stable until late adulthood. While early‐blind individuals, even at the youngest ages, integrate audio‐haptic information in an optimal fashion, late‐blind individuals do not. Optimal integration in low‐vision individuals follows a similar developmental trajectory as that of sighted individuals. These findings demonstrate that visual experience is not necessary for optimal audio‐haptic integration to emerge, but that consistency of sensory information across development is key for the functional outcome of optimal multisensory integration.     

Abilities and strategies of blind and sighted subjects in visuo-spatial imagery

An active visuo-spatial memory task was used in order to determine the characteristics of mental imagery in subjects with and without visual experience. Subjects were instructed to generate a mental representation of verbally presented 2D patterns that were placed in a grid and to indicate how many pattern elements were in corresponding positions in the two halves of the grid according to a specific grid axis (vertical or horizontal). Unexpectedly, results showed a similar performance in early blind, late blind and sighted subjects. However, subjects' debriefing showed that the three groups used different strategies. The sighted and the late blind subjects took advantage of a visuo-spatial strategy. They generated a mental image of the matrix and they simplified this image to maintain only the relevant information in memory. In contrast, the early blind subjects encoded each pattern element by its location in a (X,Y) coordinate system without visual representation. This indicates that both early and late blind subjects are able to perform an active visuo-spatial imagery task as well as sighted subjects although they use different strategies.     

Vertical-horizontal illusion present for sighted but not early blind humans using auditory substitution of vision

This experiment was undertaken to investigate the effect of sensory modality (vision vs. audition) and of visual status (early blind vs. sighted) on susceptibility to the vertical-horizontal illusion. Early blind volunteers and blindfolded sighted subjects explored variants of the vertical-horizontal illusion using a device that substituted audition for vision, whereas sighted subjects from an independent group inspected the same stimuli visually. Sensitivity to the vertical-horizontal illusion, including an illusion of moderate strength when using the sensory substitution device, was observed only in the two sighted groups. The existence of an illusion effect when using such a device supports the idea of a visual perception provided by sensory substitution, whereas the attenuation of the vertical-horizontal illusion strength is consistent with the visual field shape theory (Künnapas, 1955a). The absence of the illusion effect in early blind subjects suggests that the sensory experience influences the nature of perception and that the visual experience plays a crucial role in the vertical-horizontal illusion, in accordance with the size-constancy scaling theory (Gregory, 1963).     

Does visual experience influence the spatial distribution of auditory attention?

Sighted individuals are less accurate and slower to localize sounds coming from the peripheral space than sounds coming from the frontal space. This specific bias in favour of the frontal auditory space seems reduced in early blind individuals, who are particularly better than sighted individuals at localizing sounds coming from the peripheral space. Currently, it is not clear to what extent this bias in the auditory space is a general phenomenon or if it applies only to spatial processing (i.e. sound localization). In our approach we compared the performance of early blind participants with that of sighted subjects during a frequency discrimination task with sounds originating either from frontal or peripheral locations. Results showed that early blind participants discriminated faster than sighted subjects both peripheral and frontal sounds. In addition, sighted subjects were faster at discriminating frontal sounds than peripheral ones, whereas early blind participants showed equal discrimination speed for frontal and peripheral sounds. We conclude that the spatial bias observed in sighted subjects reflects an unbalance in the spatial distribution of auditory attention resources that is induced by visual experience.     

Spatial perception in the blind.

It is often that the spatial senses (vision, hearing and the tactual senses) operate as distinct and independent modalities and, moreover, that vision is crucial to the development of spatial abilities. However, well controlled studies of blind persons with adequate experience show that they can function usefully in space. In other words, vision is not a necessary condition for spatial awareness. On the other hand, thought the blind may be equal or even superior to the sighted when performing spatial tasks within the body space, they may be deficient, either developmentally or absolutely, in tasks which involve events at a distance from the body, principally in auditory localization. One possible explanation of the differences between blind and sighted (McKinney, 1964; Attneave & Benson, 1969, Warren, 1970) is that vision is the primary spatial reference, and inputs from other modalities are fitted to a visual map. Several criticisms of this theory are adduced and an alternative theory derived from Sherrington (1947), in which all sensory inputs map on to efferent patterns, is sketched.     

The role of visual experience on the representation and updating of novel haptic scenes

We investigated the role of visual experience on the spatial representation and updating of haptic scenes by comparing recognition performance across sighted, congenitally and late blind participants. We first established that spatial updating occurs in sighted individuals to haptic scenes of novel objects. All participants were required to recognise a previously learned haptic scene of novel objects presented across the same or different orientation as learning whilst they either remained in the same position to moved to a new position relative to the scene. Scene rotation incurred a cost in recognition performance in all groups. However, overall haptic scene recognition performance was worse in the congenitally blind group. Moreover, unlike the late blind or sighted groups, the congenitally blind group were unable to compensate for the cost in scene rotation with observer motion. Our results suggest that vision plays an important role in representing and updating spatial information encoded through touch and have important implications for the role of vision in the development of neuronal areas involved in spatial cognition.     

Auditory coding of visual patterns for the blind

Recognition tasks of simple visual patterns have been used to assess an early visual--auditory sensory-substitution system, consisting of the coupling of a rough model of the human retina with an inverse model of the cochlea, by means of a pixel-frequency relationship. The potential advantage of the device, compared with previous ones, is to give the blind the ability to both localise and recognise visual patterns. Four evaluation sessions assessed the performance of twenty-four blindfolded sighted subjects using the device. Subjects had to recognise twenty-five visual patterns, one at a time, using a head-mounted small camera and interpreting the corresponding sounds given by the device. Half the subjects were trained by means of a correction feedback procedure during ten one-hour training sessions embedded in between the evaluation sessions. Results revealed extremely successful training effects. Performance of trained subjects significantly increased with practice compared with the untrained control group. The improvement was also observed for new patterns, demonstrating a learning-process generalisation. The negative correlation observed between scores and processing time showed that the subjects' response accuracy was related to their speed. In conclusion, simple pattern recognition is possible with a fairly natural vision-to-audition coding scheme, given the possibility for the subjects to have sensory--motor interactions while using the device.     

Early- and late-onset blindness both curb audiotactile integration on the parchment-skin illusion

It has been shown that congenital blindness can lead to anomalies in the integration of auditory and tactile information, at least under certain conditions. In the present study, we used the parchment-skin illusion, a robust illustration of sound-biased perception of touch based on changes in frequency, to investigate the specificities of audiotactile interactions in early- and late-onset blind individuals. Blind individuals in both groups did not experience any illusory change in tactile perception when the frequency of the auditory signal was modified, whereas sighted individuals consistently experienced the illusion. This demonstration that blind individuals had reduced susceptibility to an auditory-tactile illusion suggests either that vision is necessary for the establishment of audiotactile interactions or that auditory and tactile information can be processed more independently in blind individuals than in sighted individuals. In addition, the results obtained in late-onset blind participants suggest that visual input may play a role in the maintenance of audiotactile integration.     

Reduced visual acuity is mirrored in low vision imagery

Research has examined the nature of visual imagery in normally sighted and blind subjects, but not in those with low vision. Findings with normally sighted subjects suggest that imagery involves primary visual areas of the brain. Since the plasticity of visual cortex appears to be limited in adulthood, we might expect imagery of those with adult-onset low vision to be relatively unaffected by these losses. But if visual imagery is based on recent and current experience, we would expect images of those with low vision to share some properties of impaired visual perception. We examined key parameters of mental images reported by normally sighted subjects, compared to those with early- and late-onset low vision, and with a group of subjects with restricted visual fields using an imagery questionnaire. We found evidence that those with reduced visual acuity report the imagery distances of objects to be closer than those with normal acuity and also depict objects in imagery with lower resolution than those with normal visual acuity. We also found that all low vision groups, like the normally sighted, image objects at a substantially greater distance than when asked to place them at a distance that ‘just fits’ their imagery field (overflow distance). All low vision groups, like the normally sighted, showed evidence of a limited visual field for imagery, but our group with restricted visual fields did not differ from the other groups in this respect. We conclude that imagery of those with low vision is similar to that of those with normal vision in being dependent on the size of objects or features being imaged, but that it also reflects their reduced visual acuity. We found no evidence for a dependence on imagery of age of onset or number of years of vision impairment.     

Haptic perception of the horizontal by blind and low-vision individuals.

We examined haptic perception of the horizontal in visually impaired people. Blind people (late blind and congenitally blind), persons with very low vision, and blindfolded sighted individuals felt raised-line drawings of jars at four angles. They had to demonstrate their understanding that water remains horizontal, despite jar tilt, by selecting the correct raised-line drawing given four choices. Low-vision subjects, with near perfect scores, performed significantly better than the other groups of subjects. While the late-blind and blindfolded sighted subjects performed slightly better than the congenitally blind participants, the difference between the late-blind and congenitally blind groups was nonsignificant. The performance of the congenitally blind subjects indicates that visual experience is not necessary for the development of an understanding that water level stays horizontal, given container tilt.     

Representing auditory space

Ontario Institute for Studtes tn Education, University of Toronto, Toronto, Canada M5S 1 V6 Auditory space has been characterized as an entity without bound or dimension, as opposed to visual space which is limited in three dimensions. While there is evidence that visual space may be represented mentally in terms of contrastive values on these dimensions, no evidence exists concerning the representation of auditory space. Two experiments used an auditory Stroop-like task to investigate (1) whether the linguistic code for auditory space is comprised of component dimensions as in vision or whether it is unitary, and (2) whether auditory space can also be encoded in a nonlinguistic fashion. Subjects were required to respond to the spatial location of a locative term whose meaning could be congruent, incongruent, or neutral with respect to its location. The findings pointed to the conclusion that when subjects are encouraged to code auditory space linguistically, the code is an undifferentiated symbol or name. Furthermore, some tentative evidence existed that auditory space may also be encoded in a nonlinguistic manner.     

Mental rotation and the frame of reference in blind and sighted individuals

Mental rotation in the congenitally blind was investigated with a haptic letter-judgment task. Blind subjects and blindfolded, sighted subjects were presented a letter in some orientation between 0° to 300° from upright and timed while they judged whether it was a normal or mirror-image letter. Both groups showed an increasing response time with the stimulus’s departure from upright; this result was interpreted as reflecting the process of mental rotation. The results for the blind subjects suggest that mental rotation can operate on a spatial representation that does not have any specifically visual components. Further research showed that for the sighted subjects in the haptic task, the orientation of a letter is coded with respect to the position of the hand. Sighted subjects may code the orientation of the letter and then translate this code into a visual representation, or they may use a spatial representation that is not specifically visual.     

Memory processing of spatial order as transmitted by auditory information in the absence of visual cues

Two experiments were conducted concerning spatial order recall when spatial information is transmitted by auditory stimuli. Temporal order either was congruent with spatial order or was independent of spatial order. In Experiment 1, the comparisons were among normally or partially sighted subjects allowed to look, normally sighted subjects who were blindfolded, and blind children. The main findings were a superiority of the sighted subjects allowed to look (that is, to support auditory information with visual cues) and a smaller advantage for the sighted-but-blindfolded subjects, relative to the blind group. In Experiment 2, normally sighted adults (either seeing or blindfolded) and blind adults were tested. Surprisingly, the blind were not worse than the sighted in this study. Subsequent interviews and detailed analysis of errors suggested that the blind coded spatial information kinesthetically. These indirect analyses also suggested that whereas spatial order was coded temporally in the sighted, it was controlled by both temporal and spatial factors in the blind and blindfolded subjects.