Learning representation hierarchies by sharing visual features: a computational investigation of Persian character recognition with unsupervised deep learning

In humans, efficient recognition of written symbols is thought to rely on a hierarchical processing system, where simple features are progressively combined into more abstract, high-level representations. Here, we present a computational model of Persian character recognition based on deep belief networks, where increasingly more complex visual features emerge in a completely unsupervised manner by fitting a hierarchical generative model to the sensory data. Crucially, high-level internal representations emerging from unsupervised deep learning can be easily read out by a linear classifier, achieving state-of-the-art recognition accuracy. Furthermore, we tested the hypothesis that handwritten digits and letters share many common visual features: A generative model that captures the statistical structure of the letters distribution should therefore also support the recognition of written digits. To this aim, deep networks trained on Persian letters were used to build high-level representations of Persian digits, which were indeed read out with high accuracy. Our simulations show that complex visual features, such as those mediating the identification of Persian symbols, can emerge from unsupervised learning in multilayered neural networks and can support knowledge transfer across related domains.     

Do locally defined feature discontinuities capture attention?

Results from previous visual search studies have suggested that abrupt onsets produce involuntary shifts of attention (i.e., attentional capture), but discontinuities in simple features such as color and brightness do not (Jonides & Yantis, 1988). In the present study we tested whether feature discontinuities (i.e., “singletons”) can produce attentional capture in a visual search task if defined “locally” or over a small spatial range. On each trial, a variable number of letters appeared, one of which differed from the others in color or intensity. The location of this singleton was uncorrelated with target location. Local discontinuities were created by embedding the letters in a dot texture. In Experiment 1, display size effects for singleton targets were not reduced with the addition of a background dot texture. Similar results were obtained in Experiment 2, regardless of variations in texture density. Experiment 3 confirmed that when targets are defined by a color or intensity singleton, they are detected preattentively, and that increasing texture density yields faster detection. We conclude that the spatial range over which feature discontinuities are defined may influence the guidance of spatial attention, but it has no influence on their ability to capture attention.     

An alternative for judging confusability of visual letters.

Numerical values for shared distinctive features were derived from Gibson's (1) feature analysis of the 26 uppercase alphabet letters. Due to the lack of agreement among the empirical matrices, it is proposed that this more general table is a useful and practical approach for judging confusability of visual letters for uppercase items. Reaction time data from a Posner-type of letter-matching task support the effectiveness of the table for judging visual confusability among uppercase letters.     

Hemispheric differences for the integration of stimulus levels and their contents: evidence from bilateral presentations

Several studies have demonstrated that hemispheric differences for the processing of hierarchical letter stimuli are more likely to occur when the letters at the levels are associated with conflicting responses. Typically, a single stimulus is presented, so that the conflict occurs between the global and the local levels of the same stimulus. Our hypothesis is that in this situation, conflict resolution requires integration of the letters and their respective levels and that the hemispheres differ in this integration process. According to this integration theory, the favorable effect of response conflict on hemispheric differences should vanish if other features, such as location, can also serve for conflict resolution. This prediction was tested in the present study by simultaneously presenting an individual hierarchical stimulus to each visual field. Conflicting letters either were arranged within one stimulus or were placed in different stimuli. In the latter case, a response conflict could also be resolved by integrating letters and locations. As was expected, there were no visual field effects in these conditions. On the other hand, visual field effects showed up when the conflicting letters were located within the same stimulus. These results support the idea that the hemispheres differ in their capacity for integrating level and form.     

Changes in letter processing in beginning readers

Letter pairs were tachistoscopically presented to children from grades 1, 2 and 6 (85.6, 99.4, and 147.8 months of age, respectively). They were required to determine whether the letters had a same name by pressing one of two response keys as fast as possible. Also a letter detection task was presented where letter matching was either based on physical or name characteristics. The name match in both tasks was slower than the match of letters which shared the same visual form. The name-physical match differences changed significantly as a function of grade level. Shifts in latency differences over grades can be considered as a fundamental correlate of reading ability. Children increasingly employ strategies of processing based on nominal cues and become efficient in extracting the invariant features of letters amongst irrelevant variations such as type face.     

Priming of abstract letter representations may be universal: The case of Arabic

Recent research on the Roman alphabet has demonstrated that the magnitudes of masked repetition priming are equivalent for letter pairs that have similar visual features across cases (e.g., c-C) and for letter pairs with dissimilar features (e.g., g-G). Here, we examined whether priming of abstract letter representations occurs in an orthographic system, Arabic, in which the letters show an intricate number of contextual forms. Arabic does not have a lowercase/uppercase distinction, but the letters exhibit different forms that depend on their position (initial, medial, final, or isolated) and their connectivity. Importantly, some letters look quite different across positions (e.g., (symbol in text) and (symbol in text), which correspond to the letter 'ayn), whereas others look very similar (e.g. (symbol in text), and (symbol in text), which correspond to the letter fā'). We employed a masked priming same-different task, in which native speakers of Arabic decided whether a target letter was the same as or different from a reference letter presented in a different position (middle vs. isolated). The results showed masked repetition priming effects of the same magnitude for letter pairs with similar and with dissimilar visual features across letter positions. These data support the view that priming of abstract letter representations is a universal phenomenon.     

Attentional weights in vision as products of spatial and nonspatial components

The relationship between visual attentional selection of items in particular spatial locations and selection by nonspatial criteria was investigated in a partial report experiment with report of letters (as many as possible) from brief postmasked exposures of circular arrays of letters and digits. The data were fitted by mathematical models based on Bundesen’s (Psychological Review, 97, 523-547, 1990) theory of visual attention (TVA). Both attentional weights of targets (letters) and attentional weights of distractors (digits) showed strong variations across the eight possible target locations, but for each of the ten participants, the ratio of the weight of a distractor at a given location to the weight of a target at the same location was approximately constant. The results were accommodated by revising the weight equation of TVA such that the attentional weight of an object equals a product of a spatial weight component (weight due to being at a particular location) and a nonspatial weight component (weight due to having particular features other than locations), the two components scaling the effects of each other multiplicatively.     

Inverse discrimination time as a perceptual distance for alphabetic characters

Reaction times to discriminate lower‐case letters were collected in an experiment. The inverse discrimination times were used to build metrics on the space of letters. These metrics were found to be significantly correlated with various well‐known letter confusability measures, and a meaningful dimensional analysis of the alphabet was performed. This methodology is mathematically well founded, it requires fewer data than common methods, and it appears to be highly sensitive to visual similarity between letters, which allows visual letter features to be effectively analysed.     

Letter recognition at low contrast levels: effects of letter size

Contrast variation was used to measure recognition thresholds for lowercase letters, with the aim of obtaining a better understanding of the role that early stages of visual processing play in letter recognition. Frequency-of-recognition curves were measured for alphabets of different letter size. Since variation of the adaptational state of the eye changes the characteristics of primary visual processing in a quantifiable way, recognition thresholds were measured both at a high (150 cd m-2) and at a low (0.9 cd m-2) adaptation level. Thresholds decreased as letter size increased, in a way comparable with data on visual acuity. At the lower adaptation level, recognition thresholds became higher, which is also in accordance with visual acuity data. Furthermore, the slopes of the frequency-of-recognition curves for alphabets as a function of log contrast decreased with decreasing letter size. It is argued that this is mainly caused by an increasing dispersion of internal representations of individual letters on the internal psychological scale as letter size decreases.     

Visual image analysis by square wavelets: empirical evidence supporting a theoretical agreement between wavelet analysis and receptive field organization of visual cortical neurons

It was proposed that the human visual system analyzes images into square wavelets. To test this view, comparisons were made between the perceived similarity-dissimilarity of alphabet letters and the wavelet analyses of those same letters. For the proposal to be considered tenable, the coefficients of the wavelet analysis of similar letters must be similar, and the coefficients of the wavelet analysis of dissimilar letters must be dissimilar. From a selection of 12 letters, four pairs of letters had been reported by Van der Heijden, Mathas, and Van den Roovaart as very similar, and four other pairs of letters dissimilar. Each of the 12 letters was separately depicted in 8 x 8 matrices, and the signal represented by each of the matrices was analyzed into square wavelets using a new and original procedure which yielded a single set of coefficients for each matrix. Correlations between sets of coefficients were high (r ranged from .88 to .58) for those letter pairs judged high in similarity; correlations were low (r ranged from -.02 to .29) for those letter pairs judged low in similarity. When the correlations between the coefficients of wavelets of all eight-letter pairs were compared with the judged similarity-dissimilarity of all eight-letter pairs, the linear agreement was statistically significant. Agreement was found between the neurophysiological mapping of receptive fields of visual cortical neurons and the vectors or the pattern of pluses and minuses which characterized the wavelet analysis. Furthermore, regeneration of the visual image, or the pattern of neural activity representing the image, could be described by a tree-like flow of information among visual cortical neurons which received response data from visual receptive fields, the response data being wavelet coefficients. Results indicate the analysis accurately produces reliable transformations of visual patterns and may be a process used by the visual system.     

Reducing the effects of adjacent distractors by narrowing attention.

Three experiments explored the gradual narrowing of visual attention to a letter target when other letters were positioned close by. The method by which attention was narrowed involved presenting a digit target immediately prior to the latter target and in the same location for progressively shorter durations and requiring the subject to identify both the digit target and the letter target before responding. The response time data from the first 2 experiments indicated that shorter durations of the digit reduced the amount of information processed from noise letters positioned on either side of the letter target. In the third experiment, in which separation of letters was increased slightly, the response times indicated that the information from flanking noise letters may have been virtually eliminated.     

Illusory conjunctions inside and outside the focus of attention

This article addresses 2 questions that arise from the finding that visual scenes are first parsed into visual features: (a) the accumulation of location information about objects during their recognition and (b) the mechanism for the binding of the visual features. The first 2 experiments demonstrated that when 2 colored letters were presented outside the initial focus of attention, illusory conjunctions between the color of one letter and the shape of the other were formed only if the letters were less than 1 degree apart. Separation greater than 2 degrees resulted in fewer conjunction errors than expected by chance. Experiments 3 and 4 showed that inside the spread of attention, illusory conjunctions between the 2 letters can occur regardless of the distance between them. In addition, these experiments demonstrated that the span of attention can expand or shrink like a spotlight. The results suggest that features inside the focus of attention are integrated by an expandable focal attention mechanism that conjoins all features that appear inside its focus. Visual features outside the focus of attention may be registered with coarse location information prior to their integration. Alternatively, a quick and imprecise shift of attention to the periphery may lead to illusory conjunctions among adjacent stimuli.     

Graphemic analysis underlying literacy

Three experiments are described that studied the role of detailed graphemic analysis upon the ability to read text. College students named letters in various approximations to English, with frequency of individual letters constant. Findings were that later skill at reading varied with the order of approximation to English of the letters that were named, that the spacing of the letter sequences was important to this result, and, finally, that the influence of specific visual practice extended to the typeface on which the naming and reading were carried out. Hence, rather than a letter-by-letter process or its opposite, a wholly semantic analysis, reading is shown to be intimately dependent upon details of visual analysis of patterns or letter sequences.     

Letter processing in the visual system: different activation patterns for single letters and strings

One would expect that a lifetime of experience recognizing letters would have an important influence on the visual system. Surprisingly, there is limited evidence of a specific neural response to letters over visual control stimuli. We measured brain activation during a sequential matching task using isolated characters (Roman letters, digits, and Chinese characters) and strings of characters. We localized the visual word form area (VWFA) by contrasting the response to pseudowords against that for letter strings, but this region did not show any other sign of visual specialization for letters. In addition, a left fusiform area posterior to the VWFA was selective for letter strings, whereas a more anterior left fusiform region showed selectivity for single letters. The results of different analyses using both large regions of interest and inspections of individual patterns of response reveal a dissociation between selectivity for letter strings and selectivity for single letters. The results suggest that reading experience fine-tunes visual representations at different levels of processing. An important conclusion is that the processing of nonpronounceable letter strings cannot be assumed to be equivalent to single-letter perception.     

Cognitive influences on perceptual processing

These experiments investigate the influence of frequency of occurrence of a visual stimulus (stimulus probability) on encoding processes, in an attempt to discover what sorts of mechanisms allow cognitive processes to modify perceptual processes. Experiments 1 and 2 show that frequently occurring visual letters do not facilitate encoding of visually similar letters. This implies that stimulus probability does not directly affect the feature detectors used in encoding the letters. Four more experiments provide evidence that stimulus probability has its effect on the availability of an abstract code that is generated by the encoding process from the visual input. Results from the experiments with letter stimuli could be interpreted using a model similar to the logogen model of Morton. Experiments with nonsense forms suggest that subjects use abstract codes in dealing with the forms only when the stimuli are constructed from a set of orthogonal features. A secondary finding was that visual quality has an effect that extends past the feature analysis stage and into a stage in which the visual input activities an abstract code. This result calls into question the common practice of interpreting the interaction of a factor with visual quality as evidence that the factor affects visual feature analysis.     

Is visual image segmentation a bottom-up or an interactive process?

Visualimage segmentation is the process by which the visual system groups features that are part of a single shape. Is image segmentation a bottom-up or an interactive process? In Experiments 1 and 2, we presented subjects with two overlapping shapes and asked them to determine whether two probed locations were on the same shape or on different shapes. The availability of top-down support was manipulated by presenting either upright or rotated letters. Subjects were fastest to respond when the shapes corresponded to familiar shapes—the upright letters. In Experiment 3, we used a variant of this segmentation task to rule out the possibility that subjects performed same/different judgments after segmentation and recognition of both letters. Finally, in Experiment 4,we ruled out the possibility that the advantage for upright letters was merely due to faster recognition of upright letters relative to rotated letters. The results suggested that the previous effects were not due to faster recognition of upright letters; stimulus familiarity influenced segmentation per se. The results are discussed in terms of an interactive model of visual image segmentation.     

Detecting distortions of peripherally presented letter stimuli under crowded conditions

When visual features in the periphery are close together they become difficult to recognize: something is present but it is unclear what. This is called “crowding”. Here we investigated sensitivity to features in highly familiar shapes (letters) by applying spatial distortions. In Experiment 1, observers detected which of four peripherally presented (8 deg of retinal eccentricity) target letters was distorted (spatial 4AFC). The letters were presented either isolated or surrounded by four undistorted flanking letters, and distorted with one of two types of distortion at a range of distortion frequencies and amplitudes. The bandpass noise distortion (“BPN”) technique causes spatial distortions in Cartesian space, whereas radial frequency distortion (“RF”) causes shifts in polar coordinates. Detecting distortions in target letters was more difficult in the presence of flanking letters, consistent with the effect of crowding. The BPN distortion type showed evidence of tuning, with sensitivity to distortions peaking at approximately 6.5 c/deg for unflanked letters. The presence of flanking letters causes this peak to rise to approximately 8.5 c/deg. In contrast to the tuning observed for BPN distortions, RF distortion sensitivity increased as the radial frequency of distortion increased. In a series of follow-up experiments, we found that sensitivity to distortions is reduced when flanking letters were also distorted, that this held when observers were required to report which target letter was undistorted, and that this held when flanker distortions were always detectable. The perception of geometric distortions in letter stimuli is impaired by visual crowding.     

Visual processing speed in old age

Mental speed is a common concept in theories of cognitive aging, but it is difficult to get measures of the speed of a particular psychological process that are not confounded by the speed of other processes. We used Bundesen’s (1990) Theory of Visual Attention (TVA) to obtain specific estimates of processing speed in the visual system controlled for the influence of response latency and individual variations of the perception threshold. A total of 33 non‐demented old people (69–87 years) were tested for the ability to recognize briefly presented letters. Performance was analyzed by the TVA model. Visual processing speed decreased approximately linearly with age and was on average halved from 70 to 85 years. Less dramatic aging effects were found for the perception threshold and the visual apprehension span. In the visual domain, cognitive aging seems to be most clearly related to reductions in processing speed.     

Are individual or consecutive letters the unit affected by repetition blindness?

When 2 similar words (e.g., react reach) are briefly sequentially displayed, the 2nd word may be omitted from the report, a phenomenon known as repetition blindness (RB). Previous researchers have suggested that consecutive letters are the unit affected by RB. Six experiments provided new data on orthographic RB. Two letters at the beginning or end of words resulted in RB, as did alternating interior letters (tactile earthly) and 3 letters with different relative positions (arid bird). However, no RB was found with a single final letter (show view). Observed RB may reflect pattern completion because RB for pairs like throat theory was reduced when the nonrepeated letters (eory) were consistent with only a single word. The experiments point to a model of orthographic RB in which both individual letters and letter sequences of length 2 or more play a role.     

Letter Dyslexia in a Letter-by-Letter Reader

We describe a letter-by-letter patient who produced misreading errors in both letters in isolation and in words. All errors were visual in nature. We hypothesized an access deficit to the abstract visual representation of letters that prevents letter identification. This deficit could account for the patient's letter-by-letter behavior, since each letter constituted a potential identification problem. An access deficit, moreover, could also explain the patient's letter visual errors. In access processing, in fact, the letters sharing common structural features in their abstract representations were the ones more frequently mismatched.