视听跨通道信息的整合与冲突控制

多通道信息交互是指来自某个感觉通道的信息与另一感觉通道的信息相互作用、相互影响的一系列加工过程。主要包括两个方面：一是不同感觉通道的输入如何整合;二是跨通道信息的冲突控制。本文综述了视听跨通道信息整合与冲突控制的行为心理机制和神经机制,探讨了注意对视听信息整合与冲突控制的影响。未来需探究视听跨通道信息加工的脑网络机制,考察特殊群体的跨通道整合和冲突控制以帮助揭示其认知和社会功能障碍的机制。     

The role of audiovisual asynchrony in person recognition

Audiovisual integration (AVI) has been demonstrated to play a major role in speech comprehension. Previous research suggests that AVI in speech comprehension tolerates a temporal window of audiovisual asynchrony. However, few studies have employed audiovisual presentation to investigate AVI in person recognition. Here, participants completed an audiovisual voice familiarity task in which the synchrony of the auditory and visual stimuli was manipulated, and in which visual speaker identity could be corresponding or noncorresponding to the voice. Recognition of personally familiar voices systematically improved when corresponding visual speakers were presented near synchrony or with slight auditory lag. Moreover, when faces of different familiarity were presented with a voice, recognition accuracy suffered at near synchrony to slight auditory lag only. These results provide the first evidence for a temporal window for AVI in person recognition between approximately 100 ms auditory lead and 300 ms auditory lag.     

The role of motivation in the conceptual model of motor development in childhood

The aim of this study was twofold: first, to investigate whether perceived motor competence (PMC) mediates the relation between actual motor competence (AMC) and physical activity (PA) according to the conceptual model of motor development, and second to examine the role of different motivational regulations (i.e., intrinsic, identified, introjected, and external regulation) in the relationship between PMC and PA. A sample of 504 Spanish students (46.2% girls, 8–12 years old) voluntarily participated in this study. In relation to the first aim, structural equation modeling revealed that PMC indeed mediates the association between AMC and PA. In relation to the second aim, positive associations between AMC and PMC (β = 0.32, p < .001), which in turn was positively related to intrinsic and identified regulations (β = 0.46 and β = 0.43 respectively, p < .001), were found. The model showed direct paths from intrinsic and introjected regulation to PA (β = 0.27 and β = 0.22, p < .05) and indirect paths from PMC through intrinsic motivation to PA (β = 0.13, p < .05). This study confirms that intrinsic motivation mediates the association between PMC and PA. Strategies targeting to build and develop children’s AMC and PMC, and fostering children’s intrinsic motivation should be targeted to promote children’s PA engagement and motor development.     

语言加工过程中的视听跨通道整合

日常生活中, 语言的使用往往出现在某个视觉情境里。大量认知科学研究表明, 视觉信息与语言信息加工模块并不是独立工作, 而是存在复杂的交互作用。本文以视觉信息对语言加工的影响为主线, 首先对视觉信息影响言语理解, 言语产生以及言语交流的相关研究进展进行了综述。其次, 重点对视觉信息影响语言加工的机制进行了探讨。最后介绍了关于视觉信息影响语言加工的计算模型, 并对未来的研究方向提出了展望。     

Perceptual processing affects conceptual processing

According to the Perceptual Symbols Theory of cognition ( Barsalou, 1999 ), modality-specific simulations underlie the representation of concepts. A strong prediction of this view is that perceptual processing affects conceptual processing. In this study, participants performed a perceptual detection task and a conceptual property-verification task in alternation. Responses on the property-verification task were slower for those trials that were preceded by a perceptual trial in a different modality than for those that were preceded by a perceptual trial in the same modality. This finding of a modality-switch effect across perceptual processing and conceptual processing supports the hypothesis that perceptual and conceptual representations are partially based on the same systems.     

An acquired deficit of audiovisual speech processing

We report a 53-year-old patient (AWF) who has an acquired deficit of audiovisual speech integration, characterized by a perceived temporal mismatch between speech sounds and the sight of moving lips. AWF was less accurate on an auditory digit span task with vision of a speaker's face as compared to a condition in which no visual information from the lower face was available. He was slower in matching words to pictures when he saw congruent lip movements compared to no lip movements or non-speech lip movements. Unlike normal controls, he showed no McGurk effect. We propose that multisensory binding of audiovisual language cues can be selectively disrupted.     

The role of salience in conceptual combination

The present study investigated three factors that affect the interpretation of novel noun-noun (N-N) combinations: simple feature salience, ontological category, and assessed similarity. Participants read and defined a series of novel N-N combinations in which the feature salience of N1 and N2 was manipulated. Participants also rated the combinations for similarity. The combinations were constrained to be within ontological category. All interpretations were scored in terms of the strategies (property mapping vs. relation linking) used to produce the given interpretations. Highly salient features drove property-mapping interpretations based on those features. Natural kinds produced more property-mapping interpretations than did artifacts. There was no correlation between the proportion of property-mapping interpretations and the assessed similarity of the N-N combinations. These results are discussed as an extension of Estes and Glucksberg's (2000) interactive theory of conceptual combination and argue for the importance of feature salience as a factor in conceptual combination.     

The role of the motor system in conceptual processing: Effects of object affordances beyond response interference

Numerous behavioral and neuro-imaging studies have demonstrated that the motor system is activated when people are presented with manipulable objects. However it remains a matter of debate whether these results should be interpreted as evidence that certain conceptual processes employ motor programs. In order to address this issue, we conducted two experiments which required participants to assess the functions of tool-like objects and respond verbally. The results demonstrate that action affordances may constrain performance in tasks which are not based on the stimulus–response compatibility paradigm. We argue that this finding supports the causal role of the motor system in conceptual processing and that it cannot be explained by spreading of activation and response interference.     

The development of interpretations for novel noun-noun conceptual combinations during the early school years

This research investigates issues surrounding early school children's use of the similarity between head and modifier terms in deriving interpretations for novel noun-noun conceptual combinations. In these experiments, 6- and 9-year-olds and adults were asked to formulate interpretations of similar and dissimilar conceptual combinations. Both children and adults were sensitive to the similarity aspect of conceptual combinations, although the children had some difficulty with the property interpretations that high-similarity combinations require. Next, we examined 40 popular children's books for the presence of noun-noun conceptual combinations. Adult participants provided interpretations for these combinations and rated the similarity of the head and modifier nouns. Results indicated that there were few high-similarity combinations and few combinations requiring property interpretations, suggesting that children have limited exposure to highly similar combinations and property interpretations. Further analysis of children's interpretations indicates that they may have difficulty in selecting and integrating properties of the modifier onto the head, a process required by property interpretations.     

The role of diverse instruction in conceptual change

This study examined how a fundamental principle of induction and scientific reasoning, information diversity, could be used to promote change in children's mental models of the earth's shape. Six-year-old children (N=132) were randomly allocated to a control or to one of two training conditions. Some training groups received instruction that simultaneously challenged children's beliefs concerning (a) why the earth appears flat to a surface observer and (b) the role of gravity. Others received instruction that repeatedly challenged only one of these beliefs. An adaptation of the Vosniadou and Brewer (1992, Cognitive Psychology 24, 535-585) protocol for identifying mental models of the earth was administered before and after instruction. Both instruction methods produced increases in factual knowledge. Only children receiving instruction about two core beliefs, however, showed an increased rate of acceptance of a spherical earth model at posttest. The findings show that instruction that challenges diverse aspects of children's na?ve scientific beliefs is more likely to produce conceptual change.     

The linguistic and embodied nature of conceptual processing

Recent theories of cognition have argued that embodied experience is important for conceptual processing. Embodiment can be contrasted with linguistic factors such as the typical order in which words appear in language. Here, we report four experiments that investigated the conditions under which embodiment and linguistic factors determine performance. Participants made speeded judgments about whether pairs of words or pictures were semantically related or had an iconic relationship. The embodiment factor was operationalized as the degree to which stimulus pairs were presented in the spatial configurations in which they usually occur (i.e., an iconic configuration, e.g., attic presented above basement). The linguistic factor was operationalized as the frequency of the stimulus pairs in language. The embodiment factor predicted error rates and response time better for pictures, whereas the linguistic factor predicted error rates and response time better for words. These findings were modified by task, with the embodiment factor being strongest in iconicity judgments for pictures and the linguistic factor being strongest in semantic judgments for words. Both factors predicted error rates and response time for both semantic and iconicity judgments. These findings support the view that conceptual processing is both linguistic and embodied, with a bias for the embodiment or the linguistic factor depending on the nature of the task and the stimuli.     

The role of exposure to isolated words in early vocabulary development

Fluent speech contains no known acoustic analog of the blank spaces between printed words. Early research presumed that word learning is driven primarily by exposure to isolated words. In the last decade there has been a shift to the view that exposure to isolated words is unreliable and plays little if any role in early word learning. This study revisits the role of isolated words. The results show (a) that isolated words are a reliable feature of speech to infants, (b) that they include a variety of word types, many of which are repeated in close temporal proximity, (c) that a substantial fraction of the words infants produce are words that mothers speak in isolation, and (d) that the frequency with which a child hears a word in isolation predicts whether that word will be learned better than the child's total frequency of exposure to that word. Thus, exposure to isolated words may significantly facilitate vocabulary development at its earliest stages.     

The development of conceptual interests in young children

We conducted a longitudinal analysis of the relative intensity and duration of interests associated with conceptual domains between the ages of 4 and 6 years, respectively. Results indicated a significant portion of preschool children do sustain an interest in conceptual domains during some portion of their childhood. Expected gender differences were found, with boys more likely to express an interest in a conceptual domain than girls. A latent growth curve analysis revealed that the probability of exhibiting a conceptual interest declined as school began, though the rate of that decline was similar for both boys and girls. Potential explanations for the decrease in conceptual interests as school begins are considered.     

Continuous information processing in conceptual identifications

Summary This investigation was aimed at the analysis of processes of the conceptual object identification. Three experiments were performed which utilized a response preparation paradigm developed by Miller (1982). The results confirm the psychological validity of a distinction between sensorily and categorially represented concepts proposed formerly by the authors. The process of identification of objects as belonging to sensory concepts was analysed in more detail. The experimental results suggest that it may be a sequential feature matching process which is characterized by continuous information processing. Early in the process the figurative features allowing an identification of an object as belonging to a more general sensory concept are processed. In following processing stages this first conceptual identification is more and more specified by an additional checking of more detailed figurative features. More importantly, the results indicate that the information about the general conceptual identity of an object is not only available early in the identification process but can also be used for the preparation of following processes or processing stages even before the object is fully identified. Possible influences of the early information on the control of the conceptual identification itself and on the selection and control of motor behavior are discussed.We wish to thank the following psychology students at the Humboldt University in Berlin for their help in performing the experiments: Kathrin Hänel, Erika Küchler and Christina Matthé. Furthermore, we wish to thank the two anaonymous reviewers and Johannes Engelkamp for their helpful comments on a first draft of our paper     

The effect of early visual deprivation on the development of face processing

We evaluated the importance of early visual input for the later development of expertise in face processing by studying 17 patients, aged 10 to 38 years, treated for bilateral congenital cataracts that deprived them of patterned visual input for the first 7 weeks or more after birth. We administered five computerized tasks that required matching faces on the basis of identity (with changed facial expression or head orientation), facial expression, gaze direction and lip reading. Compared to an age–matched control group, patients’ recognition of facial identity was impaired significantly when there was a change in head orientation (e.g. from frontal to tilted up), and tended to be impaired when there was a change in facial expression (e.g. from happy to surprised). Patients performed normally when matching facial expression and direction of gaze (e.g. looking left or right), and in reading lips (e.g. pronouncing ‘u’ or ‘a’). The results indicate that visual input during early infancy is necessary for the normal development of some aspects of face processing, and are consistent with theories postulating the importance of early visual experience (de Schonen & Mathivet, 1989; Johnson & Morton, 1991) and separate neural mediation of different components of face processing (Bruce & Young, 1986).     

The role of facial colour and luminance in visual and audiovisual speech perception

We conducted four experiments to investigate the role of colour and luminance information in visual and audiovisual speech perception. In experiments 1a (stimuli presented in quiet conditions) and 1b (stimuli presented in auditory noise), face display types comprised naturalistic colour (NC), grey-scale (GS), and luminance inverted (LI) faces. In experiments 2a (quiet) and 2b (noise), face display types comprised NC, colour inverted (CI), LI, and colour and luminance inverted (CLI) faces. Six syllables and twenty-two words were used to produce auditory and visual speech stimuli. Auditory and visual signals were combined to produce congruent and incongruent audiovisual speech stimuli. Experiments 1a and 1b showed that perception of visual speech, and its influence on identifying the auditory components of congruent and incongruent audiovisual speech, was less for LI than for either NC or GS faces, which produced identical results. Experiments 2a and 2b showed that perception of visual speech, and influences on perception of incongruent auditory speech, was less for LI and CLI faces than for NC and CI faces (which produced identical patterns of performance). Our findings for NC and CI faces suggest that colour is not critical for perception of visual and audiovisual speech. The effect of luminance inversion on performance accuracy was relatively small (5%), which suggests that the luminance information preserved in LI faces is important for the processing of visual and audiovisual speech.     

Changes in thinking about early sex role development

How do we learn to recognize ourselves and to live as beings endowed with gender? This paper discusses changes in our answer to this question over the last 15 years. As our methods of study have changed, we have been forced to see the development of sex role as an increasingly complicated process. This paper documents two studies that were attempts to bring together two methodologies: cognitive development and social learning. In the first study, 180 children were tested using the R. G. Slaby and K. S. Frey (1975, Child Development, 46, 849–856) gender identification interview. The findings documented that children's gender understanding followed the sequence predicted by L. Kohlberg (1966, in E. Maccoby (Ed.), The development of sex differences. Stanford, CA: Stanford Univ. Press): identity, stability, and constancy. However, the child's level of gender understanding was unrelated to the adoption of sex-typed behaviors. In the second study, a second group of 64 children, 20 to 30 months of age, were tested for understanding of gender labels, gender identity, and sex-typed behaviors. Sex of playmates and boys' play with feminine toys were related to understanding of verbal gender labels. Reasons for continuing problems of interpretation in the sex role area are discussed.     

Cultural and linguistic factors in audiovisual speech processing: The McGurk effect in Chinese subjects

The “McGurk effect” demonstrates that visual (lip-read) information is used during speech perception even when it is discrepant with auditory information. While this has been established as a robust effect in subjects from Western cultures, our own earlier results had suggested that Japanese subjects use visual information much less than American subjects do (Sekiyama & Tohkura, 1993). The present study examined whether Chinese subjects would also show a reduced McGurk effect due to their cultural similarities with the Japanese. The subjects were 14 native speakers of Chinese living in Japan. Stimuli consisted of 10 syllables (/ba/, /pa/, /ma/, /wa/, /da/, /ta/, /na/, /ga/, /ka/, /ra/ ) pronounced by two speakers, one Japanese and one American. Each auditory syllable was dubbed onto every visual syllable within one speaker, resulting in 100 audiovisual stimuli in each language. The subjects’ main task was to report what they thought they had heard while looking at and listening to the speaker while the stimuli were being uttered. Compared with previous results obtained with American subjects, the Chinese subjects showed a weaker McGurk effect. The results also showed that the magnitude of the McGurk effect depends on the length of time the Chinese subjects had lived in Japan. Factors that foster and alter the Chinese subjects’ reliance on auditory information are discussed.