Semantic effects on sensory-motor modality switching

Language processing always involves a combination of sensory (auditory or visual) and motor modalities (vocal or manual). In line with embodied cognition theories, we additionally assume a semantically implied modality (SIM) due to modality references of the underlying concept. Understanding ear-related words (e.g. “noise”), for example, should activate the auditory SIM. In the present study, we investigated the influence of the SIM on sensory-motor modality switching (e.g. switching between the auditory-vocal and visual-manual combination). During modality switching, participants categorised words with regard to their SIM (e.g. ear- versus eye-related words). Overall performance was improved and switch costs were reduced whenever there was concordance between SIMs and sensory-motor modalities (e.g. an auditory presentation of ear-related words). Thus, the present study provides first evidence for semantic effects during sensory-motor modality switching in terms of facilitation effects whenever the SIM was in concordance with sensory-motor modalities.