Perceptual interactions of facial dimensions in speeded classification and identification

The representation underlying the identification and classification of semirealistic line drawings taken from a computer model of the face was investigated by using a speeded classification task and an identification task. These data were analyzed by using a multidimensional extension of signal detection theory, within which varieties of perceptual interactions between dimensions within and across stimuli can be characterized. The dimensions of interest here were eye separation, nose length, and mouth width. The response time and accuracy data from the speeded classification task suggest that processing of a given feature did depend on whether other features were present or absent, but given that other features were present, the results strongly support separability (a macrolevel, across-stimulus form of invariance) for all pairs of facial dimensions used. This separability was confirmed by the subsequent identification task. Owing to its greater resolution, the identification task can reveal interactions that might exist at more microlevels of processing. In fact, the identification data did indicate the presence of perceptual dependence between facial dimensionswithin a stimulus when the dimensions that were varied were close in spatial proximity (i.e., eye separation and nose length). Within the theoretical framework, perceptual dependence can be interpreted as correlated noise between otherwise separate channels (and hence, is logically distinct from separability). This dependence was greatly reduced for dimensions that were more distant (eyes and mouth). The relation between these results and the configural effects that have been observed with faces as stimuli in other studies is discussed.     

Perceptual interactions of facial dimensions in speeded classification and identification.

The representation underlying the identification and classification of semirealistic line drawings taken from a computer model of the face was investigated by using a speeded classification task and an identification task. These data were analyzed by using a multidimensional extension of signal detection theory, within which varieties of perceptual interactions between dimensions within and across stimuli can be characterized. The dimensions of interest here were eye separation, nose length, and mouth width. The response time and accuracy data from the speeded classification task suggest that processing of a given feature did depend on whether other features were present or absent, but given that other features were present, the results strongly support separability (a macrolevel, across-stimulus form of invariance) for all pairs of facial dimensions used. This separability was confirmed by the subsequent identification task. Owing to its greater resolution, the identification task can reveal interactions that might exist at more microlevels of processing. In fact, the identification data did indicate the presence of perceptual dependence between facial dimensions within a stimulus when the dimensions that were varied were close in spatial proximity (i.e., eye separation and nose length). Within the theoretical framework, perceptual dependence can be interpreted as correlated noise between otherwise separate channels (and hence, is logically distinct from separability). This dependence was greatly reduced for dimensions that were more distant (eyes and mouth). The relation between these results and the configural effects that have been observed with faces as stimuli in other studies is discussed.     

Processing linguistic and perceptual dimensions of speech: interactions in speeded classification.

Performance on selective-attention and divided-attention tasks shows strong and consistent interactions when participants rapidly classify auditory stimuli whose linguistic and perceptual dimensions (the words low vs. high, low and high pitch, low and high position in space) share common labels. Compared with baseline performance, response times were greater when one or two irrelevant dimensions varied (Garner interference) and when combinations of attributes were incongruent rather than congruent (congruence effects). Performance depended only on the congruence relationships between the relevant dimension and each of the irrelevant dimensions and not on the congruence relationships between the irrelevant dimensions themselves. In selective attention, an additive multidimensional model accounts well for the patterns of both Garner interference and congruence effects.     

The semantic deficit hypothesis: Perceptual parsing and object classification by aphasic patients

Many aphasic patients are impaired in their ability to provide or to recognize the names of objects, but little is known about the processing deficits that underlie these difficulties. In this report, a model of object naming/name recognition is proposed, and a prediction is tested concerning one possible functional locus of impairment in name-recognition and object-naming disorders. A subgroup of aphasic patients is found to be impaired in the ability to perform perceptual similarity judgments for pairs of stimulus objects, and to be unable to classify the objects into one of two lexical categories. It is concluded that the classification disorder suffered by these patients results from an impairment at the level of the semantically guided perceptual parsing of objects.     

Perceptual and response interactions in semantic priming

In two experiments, subjects made pairs of lexical decisions verbally. In Experiment 1, masked stimuli appeared concurrently to the left and right of fixation; in Experiment 2, nonmasked stimuli appeared sequentially at fixation. The left-hand letter strings were judged more accurately in in Experiment 1, and the second letter strings were judged more accurately in Experiment 2. Each string in the pair could be either a word (e.g., fork) or a nonword anagram (e.g., frok). Consequently, the two strings in the pair could be related (e.g., fork-spoon, frok-spoon, etc.) or unrelated (e.g., fork-door, frok-door, etc.), independently of whether neither, either, or both strings were words. Semantically related stimuli induced consistent biases to respond "word," as noted in other studies. These biases were typically stronger for the event reported second. Minimal evidence was found for perceptual priming effects. The asymmetrical effects were consistent with spreading-activation-type mechanisms, but other considerations support a multiple-process view.     

Multisensory synesthetic interactions in the speeded classification of visual size

In the present study, we attempted to demonstrate a synesthetic relationship between auditory frequency and visual size. In Experiment 1, participants performed a speeded visual size discrimination task in which they had to judge whether a variable-sized disk was bigger or smaller than a standard reference disk. A task-irrelevant sound that was either synesthetically congruent with the relative size of the disk (e.g., a low-frequency sound presented with a bigger disk) or synesthetically incongruent with it (e.g., a low-frequency sound presented with a smaller disk) was sometimes presented together with the variable disk. Reaction times were shorter in the synesthetically congruent condition than in the incongruent condition. Verbal labeling and semantic mediation interpretations of this interaction were explored in Experiment 2, in which high- and low-frequency sounds were presented in separate blocks of trials, and in Experiment 3, in which the tones were replaced by the spoken words "high" and "low." Response priming/bias explanations were ruled out in Experiment 4, in which a synesthetic congruency effect was still reported even when participants made same-versus-different discrimination responses regarding the relative sizes of the two disks. Taken together, these results provide the first empirical demonstration that the relative frequency of an irrelevant sound can influence the speed with which participants judge the size of visual stimuli when the sound varies on a trial-by-trial basis along a synesthetically compatible dimension. The possible cognitive bases for this synesthetic association are also discussed.     

Perceptual simulations and linguistic representations have differential effects on speeded relatedness judgements and recognition memory

We examined the effect of spatial iconicity (a perceptual simulation of canonical locations of objects) and word-order frequency on language processing and episodic memory of orientation. Participants made speeded relatedness judgements to pairs of words presented in locations typical of their real-world arrangements (e.g., ceiling on top and floor on bottom). They then engaged in a surprise orientation recognition task for the word pairs. We replicated Louwerse's (2008) Louwerse, M. M. 2008. Embodied representations are encoded in language. Psychonomic Bulletin & Review, 15: 838–844. [Crossref], [PubMed], [Web of Science ®] , [Google Scholar] finding that word-order frequency has a stronger effect on semantic relatedness judgements than does spatial iconicity. This is consistent with recent suggestions that linguistic representations have a stronger impact on immediate decisions about verbal materials than do perceptual simulations. In contrast, spatial iconicity enhanced episodic memory of orientation to a greater extent than word-order frequency did. This new finding indicates that perceptual simulations have an important role in episodic memory. Results are discussed with respect to theories of perceptual representation and linguistic processing.     

The effect of correlated linguistic dimensions on speeded classification of visually presented trigrams

The influence of two types of linguistic dimensions, word-nonword and consonant pronunciation, on classification speed of trigrams in card-sorting tasks of two levels of complexity was examined. In complex grouping tasks, which required the evaluation of more than one letter to classify each stimulus, sorting times were faster when the linguistic dimension was correlated with, rather than orthogonal to, the response categories. For tasks in which each stimulus could be classified on the basis of a single letter, no effect of the correlated vs orthogonal linguistic dimension was observed, even when performance was degraded by visual noise. These results provide further evidence that, while linguistic properties of visual stimuli may influence classification time in complex tasks, they are of little importance in the performance of tasks only requiring the discriminalion of a single visual feature.     

The semantic congruity effect: Some tests of the expectancy hypothesis

Subjects comparing items in memory along some dimension are usually quicker to specify the lesser (than the greater) of two low magnitude items and the greater (than the lesser) of two high magnitude ones. One account explains this congruity effect as due to subjects instructed to specify the higher as expecting high magnitude items to follow and the reverse being true for subjects specifying the lesser. Three experiments tested this expectancy hypothesis. In experiment 1, subjects were set to the actual size range of each pair before the pair was shown but the congruity effect still occurred. In experiments 2 and 3, subjects compared critical pairs from a narrow size range plus more from either the same or much broader ranges. Times to compare the critical pairs were the same regardless of the range of the other pairs that subjects were exposed to. These results are strong evidence against the expectancy hypothesis.     

Locus of semantic priming effects in speeded naming

The locus of semantic priming effects was examined by measuring onset and rime durations as well as response latencies of words with consistent and inconsistent pronunciations, using the postvocalic naming task. We found that the effect of a semantic prime on naming duration was localized rather than spread across the entire word; onset durations were shorter in the related condition than in the unrelated condition, but rime durations were equal in the two prime conditions. Moreover, the priming effect on onset durations was larger for words with inconsistent than for those with consistent pronunciations. These duration results cannot be accounted for by previous proposals, but they can be accounted for by models in which phonemes are activated in parallel rather than serially from left to right and in which motor programs are based on phonemes rather than syllables. Contrary to previous reports of an interaction of prime and regularity (a factor closely related to consistency) on naming latency, we found no interaction of prime and consistency on response latency. We argue that this conflict is only apparent and arises because naming latency conflates response latency and initial phoneme duration for some targets.     

Perceptual priming is not a necessary consequence of semantic classification of pictures

Four experiments investigated how repetition priming of object recognition is affected by the task performed in the prime and test phases. In Experiment 1 object recognition was tested using both vocal naming and two different semantic decision tasks (whether or not objects were manufactured, and whether or not they would be found inside the house). Some aspects of the data were inconsistent with contemporary models of object recognition. Specifically, object priming was eliminated with some combinations of prime and test tasks, and there was no evidence of perceptual (as opposed to conceptual or response) priming in either semantic classification task, even though perceptual identification of the objects is required for at least one of these tasks. Experiment 2 showed that even when perceptual demands were increased by brief presentation, the inside task showed no perceptual priming. Experiment 3 showed that the inside task did not appear to be based on conceptual priming either, as it was not primed significantly when the prime decisions were made to object labels. Experiment 4 showed that visual sensitivity could be restored to the inside task following practice on the task, supporting the suggestion that a critical factor is whether the semantic category is preformed or must be computed. The results show that the visual representational processes revealed by object priming depend crucially on the task chosen.     

On cross-modal similarity: auditory-visual interactions in speeded discrimination

A series of four experiments explored how cross-modal similarities between sensory attributes in vision and hearing reveal themselves in speeded, two-stimulus discrimination. When subjects responded differentially to stimuli on one modality, speed and accuracy of response were greater on trials accompanied by informationally irrelevant "matching" versus "mismatching" stimuli from the other modality. Cross-modal interactions appeared in (a) responses to dim/bright lights and to dark/light colors accompanied by low-pitched/high-pitched tones; (b) responses to low-pitched/high-pitched tones accompanied by dim/bright lights or by dark/light colors; (c) responses to dim/bright lights, but not to dark/light colors, accompanied by soft/loud sounds; and (d) responses to rounded/sharp forms accompanied by low-pitched/high-pitched tones. These results concur with findings on cross-modal perception, synesthesia, and synesthetic metaphor, which reveal similarities between pitch and brightness, pitch and lightness, loudness and brightness, and pitch and form. The cross-modal interactions in response speed and accuracy may take place at a sensory/perceptual level of processing or after sensory stimuli are encoded semantically.     

Visual-auditory interaction in speeded classification: Role of stimulus difference

An experiment examined cross-modal interference and congruence in speeded classification: Subjects had to identify compound (visual-auditory) stimuli as either low or high in spatial position (visual judgment) or low or high in pitch (auditory judgment), in 16 conditions, each of which combined one of four possible pairs of tones, varying in frequency difference, with one of four possible pairs of dots, varying in positional difference. Both classification by position and classification by pitch revealed Garner interference (poorer performance than baseline, with orthogonal variation in the irrelevant dimension) and congruence effects (better performance with congruent than with incongruent stimulus combinations), but pitch classification showed more. Furthermore, the size of the pitch difference strongly affected classification by pitch and less strongly affected classification by position, but the size of the position difference affected neither. The findings are consistent with the view that Garner interference and congruence effects are closely related, perhaps arising from a common source, and suggest that the asymmetries could depend in part on the degree of dimensional overlap between stimuli and responses.     

Effects of similarity and practice on speeded classification response times and accuracies: Further tests of an exemplar-retrieval model

Observers were tested in a perceptual category-learning experiment in which they were instructed to make classification decisions as rapidly as possible without making errors. Nosofsky and Palmeri's (1997b) exemplar-based random walk (EBRW) model of speeded classification was tested for its ability to fit the classification response times and accuracies. The authors demonstrated that the EBRW model provided good quantitative fits to the mean response times and accuracies associated with individual objects as a function of their locations in a multidimensional similarity space and as a function of practice in the task. Preliminary evidence was also obtained that stimulus-specific adjustments in the random walk response criteria may have occurred during the course of learning.     

Selective attention in the speeded classification and comparison of multidimensional stimuli

Selective attention was studied when subjects were required to make either speeded classifications of single stimuli or comparisons of pairs of multidimensional stimuli. Experiment 1 established that subjects were able to attend selectively to form when the irrelevant dimension of size or shading varied in a speeded-classification (card-sorting) task. Experiment 2 confirmed this finding in a discrete-trials task. However, subjects werenot able to filter out irrelevant dimensional disparity in a comparably designed simultaneous-comparison (i.e., “same”-“different”) task. Mean “same” reaction time increased monotonically with increases in disparity between the two stimuli on the irrelevant dimension. Experiment 3 also revealed a monotonie increase in “same” RT as a function of irrelevant disparity in a successive-comparison task. These results were discussed in terms of a normalization model proposed by Dixon and Just (1978) in which it is assumed that a subject equates the two stimuli on the irrelevant dimension before deciding that they are the same along the relevant dimension. It was concluded that: (1) although subjects can efficiently filter out irrelevant disparity in a speeded-classification task, interference due to irrelevant disparity is obtained in the comparison tasks, (2) a common process such as normalization does not necessarily underlie performance in the speeded-classification and comparison tasks, (3) the ability to attend selectively to a stimulus dimension may be task determined as well as stimulus determined, and (4) contrary to the Dixon and Just proposal, normalization of irrelevant disparity occurs in a comparison task, even when the relevant dimension is represented as a separate encoding feature.     

Garner and congruence effects in the speeded classification of bimodal signals

The role of attention in speeded Garner classification of concurrently presented auditory and visual signals was examined in 4 experiments. Within-trial interference (i.e., congruence effects) occurred regardless of the attentional demands of the task. Between-trials interference (i.e., Garner interference) occurred only under conditions of divided attention when making judgments about auditory signals. Of importance, the data show congruence effects in the absence of Garner interference. Such a pattern has been rarely reported in studies of the classification of purely visual stimuli and contradicts theoretical accounts asserting that the effects share a common locus. The data question the notion that Garner classification reveals fundamental insights about the nature of the perceptual processing of bimodal stimuli.     

Assessing automaticity in audiovisual speech integration: evidence from the speeded classification task

The McGurk effect is usually presented as an example of fast, automatic, multisensory integration. We report a series of experiments designed to directly assess these claims. We used a syllabic version of the speeded classification paradigm, whereby response latencies to the first (target) syllable of spoken word-like stimuli are slowed down when the second (irrelevant) syllable varies from trial to trial. This interference effect is interpreted as a failure of selective attention to filter out the irrelevant syllable. In Experiment 1 we reproduced the syllabic interference effect with bimodal stimuli containing auditory as well as visual lip movement information, thus confirming the generalizability of the phenomenon. In subsequent experiments we were able to produce (Experiment 2) and to eliminate (Experiment 3) syllabic interference by introducing 'illusory' (McGurk) audiovisual stimuli in the irrelevant syllable, suggesting that audiovisual integration occurs prior to attentional selection in this paradigm.