The development of intersensory temporal perception: an epigenetic systems/limitations view

Several theories have stressed the importance of intersensory integration for development but have not identified specific underlying integration mechanisms. The author reviews and synthesizes current knowledge about the development of intersensory temporal perception and offers a theoretical model based on epigenetic systems theory, proposing that responsiveness to 4 basic features of multimodal temporal experience--temporal synchrony, duration, temporal rate, and rhythm--emerges in a sequential, hierarchical fashion. The model postulates that initial developmental limitations make intersensory synchrony the basis for the integration of intersensory temporal relations and that the emergence of responsiveness to the other, increasingly more complex, temporal relations occurs in a hierarchical, sequential fashion by building on the previously acquired intersensory temporal processing skills.     

Intersensory redundancy guides attentional selectivity and perceptual learning in infancy

This study assessed an intersensory redundancy hypothesis, which holds that in early infancy information presented redundantly and in temporal synchrony across two sense modalities selectively recruits attention and facilitates perceptual differentiation more effectively than does the same information presented unimodally. Five-month-old infants' sensitivity to the amodal property of rhythm was examined in 3 experiments. Results revealed that habituation to a bimodal (auditory and visual) rhythm resulted in discrimination of a novel rhythm, whereas habituation to the same rhythm presented unimodally (auditory or visual) resulted in no evidence of discrimination. Also, temporal synchrony between the bimodal auditory and visual information was necessary for rhythm discrimination. These findings support an intersensory redundancy hypothesis and provide further evidence for the importance of redundancy for guiding and constraining early perceptual learning.     

The development of infant intersensory perception: advantages of a comparative convergent-operations approach

Despite impressive demonstrations of human infants' intersensory capabilities over the past several decades, there has been little focus on the contributions of prenatal and postnatal experience or the specific developmental processes underlying the emergence of intersensory functioning. Research with nonhuman animals has, however, provided a number of advances in understanding early intersensory perception. The authors explore the value of a comparative, convergent-operations approach to the study of early intersensory perception and examine how this approach has highlighted the study of (a) prenatal factors, (b) brain-behavior relations, and (c) context and experience variables contributing to infants' intersensory responsiveness. Examples of how human and animal research programs can cross-fertilize one another in their attempts to understand developmental processes underlying intersensory perception are considered.     

Intersensory redundancy facilitates prenatal perceptual learning in bobwhite quail (Colinus virginianus) embryos.

Information presented redundantly and in temporal synchrony across sensory modalities (intersensory redundancy) selectively recruits attention and facilitates perceptual learning in human infants. This comparative study examined whether intersensory redundancy also facilitates perceptual learning prenatally. The authors assessed quail (Colinus virginianus) embryos' ability to learn a maternal call when it was (a) unimodal, (b) concurrent but asynchronous with patterned light, or (c) redundant and synchronous with patterned light. Chicks' preference for the familiar over a novel maternal call was assessed 24 hr following hatching. Chicks receiving redundant, synchronous stimulation as embryos learned the call 4 times faster than those who received unimodal exposure. Chicks who received asynchronous bimodal stimulation showed no evidence of learning. These results provide the first evidence that embryos are sensitive to redundant, bimodal information and that it can facilitate learning during the prenatal period.     

Learning and discrimination of audiovisual events in human infants: the hierarchical relation between intersensory temporal synchrony and rhythmic pattern cues

This study examined 4- to 10-month-old infants' perception of audio-visual (A-V) temporal synchrony cues in the presence or absence of rhythmic pattern cues. Experiment 1 established that infants of all ages could successfully discriminate between two different audiovisual rhythmic events. Experiment 2 showed that only 10-month-old infants detected a desynchronization of the auditory and visual components of a rhythmical event. Experiment 3 showed that 4- to 8-month-old infants could detect A-V desynchronization but only when the audiovisual event was nonrhythmic. These results show that initially in development infants attend to the overall temporal structure of rhythmic audiovisual events but that later in development they become capable of perceiving the embedded intersensory temporal synchrony relations as well.     

Infants' learning,memory, and generalization of learning for bimodal events

Study 1 investigated whether infants 3 and 7 months of age show differential learning of and memory for sight-sound pairs depending on whether or not temporal synchrony was present; memory was assessed after a 10-min and 1-week interval. Study 2 examined whether 7-month-olds show generalization of learning when they encounter novel bimodal events that are similar (changes in size, orientation, or color, and spectral sound properties) to the sight-sound pairs learned 1 week earlier based on temporal synchrony. For Study 1, infants received a familiarization phase followed by a paired-comparison preference procedure to assess for learning of the sight-sound pairs. One week later a memory test was given. Results confirmed that 7-month-olds had no difficulty learning auditory-visual pairings regardless of whether or not events were temporally synchronous, and they remembered these 10 min and 1 week later. In contrast, 3-month-olds showed poorer learning of sight-sound associations in the no-synchrony than synchrony conditions, and memory for sight-sound pairs 1 week later was shown only for the synchrony conditions. Results for Study 2 revealed generalization of learning of bimodal pairings under all stimulus conditions after a 1-week interval at 7 months of age. Implications of these findings for development of intersensory knowledge are discussed.     

Dynamic organization of intersensory function

Issues related to infant intersensory functioning are reviewed. The argument is offered that limitations of sensory inputs during early stages of development are necessary and provide structure and organization which determine behavioural characteristics at later stages. It is also suggested that infant organisms respond to the intensity of stimulation rather than organizational characteristics and that this is responsible for the form of sensory equivalence apparent during early development. A scheme for the investigation of sources of intersensory organization is presented which involves a prospective analysis of how consideration of the times of onset of sensory functions helps us to understand intra- and intersensory development. The advantages of this experimental approach are outlined.     

Judgments of synchrony between auditory and moving or still visual stimuli.

The flash-lag effect is a visual illusion wherein intermittently flashed, stationary stimuli seem to trail after a moving visual stimulus despite being flashed synchronously. We tested hypotheses that the flash-lag effect is due to spatial extrapolation, shortened perceptual lags, or accelerated acquisition of moving stimuli, all of which call for an earlier awareness of moving visual stimuli over stationary ones. Participants judged synchrony of a click either to a stationary flash of light or to a series of adjacent flashes that seemingly bounced off or bumped into the edge of the visual display. To be judged synchronous with a stationary flash, audio clicks had to be presented earlier--not later--than clicks that went with events, like a simulated bounce (Experiment 1) or crash (Experiments 2-4), of a moving visual target. Click synchrony to the initial appearance of a moving stimulus was no different than to a flash, but clicks had to be delayed by 30-40 ms to seem synchronous with the final (crash) positions (Experiment 2). The temporal difference was constant over a wide range of motion velocity (Experiment 3). Interrupting the apparent motion by omitting two illumination positions before the last one did not alter subjective synchrony, nor did their occlusion, so the shift in subjective synchrony seems not to be due to brightness contrast (Experiment 4). Click synchrony to the offset of a long duration stationary illumination was also delayed relative to its onset (Experiment 5). Visual stimuli in motion enter awareness no sooner than do stationary flashes, so motion extrapolation, latency difference, and motion acceleration cannot explain the flash-lag effect.     

The effects of socioeconomic status on working memory in childhood are partially mediated by intersensory processing of audiovisual events in infancy

Socioeconomic status (SES) is a well-established predictor of individual differences in childhood language and cognitive functioning, including executive functions such as working memory. In infancy, intersensory processing—selectively attending to properties of events that are redundantly specified across the senses at the expense of non-redundant, irrelevant properties—also predicts language development. Our recent research demonstrates that individual differences in intersensory processing in infancy predict a variety of language outcomes in childhood, even after controlling for SES. However, relations among intersensory processing and cognitive outcomes such as working memory have not yet been investigated. Thus, the present study examines relations between intersensory processing in infancy and working memory in early childhood, and the role of SES in this relation. Children (N = 101) received the Multisensory Attention Assessment Protocol at 12-months to assess intersensory processing (face-voice and object-sound matching) and received the WPPSI at 36-months to assess working memory. SES was indexed by maternal education, paternal education, and income. A variety of novel findings emerged. 1) Individual differences in intersensory processing at 12-months predicted working memory at 36-months of age even after controlling for SES. 2) Individual differences in SES predicted intersensory processing at 12-months of age. 3) The well-established relation between SES and working memory was partially mediated by intersensory processing. Children from families of higher-SES have better intersensory processing skills at 12-months and this combination of factors predicts greater working memory two years later at 36-months. Together these findings reveal the role of intersensory processing in cognitive functioning.     

On Measuring and Modeling Physiological Synchrony in Dyads

Physiological synchrony within a dyad, or the degree of temporal correspondence between two individuals' physiological systems, has become a focal area of psychological research. Multiple methods have been used for measuring and modeling physiological synchrony. Each method extracts and analyzes different types of physiological synchrony, where ‘type’ refers to a specific manner through which two different physiological signals may correlate. Yet, to our knowledge, there is no documentation of the different methods, how each method corresponds to a specific type of synchrony, and the statistical assumptions embedded within each method. Hence, this article outlines several approaches for measuring and modeling physiological synchrony, connects each type of synchrony to a specific method, and identifies the assumptions that need to be satisfied for each method to appropriately extract each type of synchrony. Furthermore, this article demonstrates how to test for between-dyad differences of synchrony via inclusion of dyad-level (i.e., time-invariant) covariates. Finally, we complement each method with an empirical demonstration, as well as online supplemental material that contains Mplus code.     

The development of infant discrimination of affect in multimodal and unimodal stimulation: The role of intersensory redundancy

This research examined the developmental course of infants' ability to perceive affect in bimodal (audiovisual) and unimodal (auditory and visual) displays of a woman speaking. According to the intersensory redundancy hypothesis (L. E. Bahrick, R. Lickliter, & R. Flom, 2004), detection of amodal properties is facilitated in multimodal stimulation and attenuated in unimodal stimulation. Later in development, however, attention becomes more flexible, and amodal properties can be perceived in both multimodal and unimodal stimulation. The authors tested these predictions by assessing 3-, 4-, 5-, and 7-month-olds' discrimination of affect. Results demonstrated that in bimodal stimulation, discrimination of affect emerged by 4 months and remained stable across age. However, in unimodal stimulation, detection of affect emerged gradually, with sensitivity to auditory stimulation emerging at 5 months and visual stimulation at 7 months. Further temporal synchrony between faces and voices was necessary for younger infants' discrimination of affect. Across development, infants first perceive affect in multimodal stimulation through detecting amodal properties, and later their perception of affect is extended to unimodal auditory and visual stimulation. Implications for social development, including joint attention and social referencing, are considered.     

Increasing task difficulty enhances effects of intersensory redundancy: testing a new prediction of the Intersensory Redundancy Hypothesis

Prior research has demonstrated intersensory facilitation for perception of amodal properties of events such as tempo and rhythm in early development, supporting predictions of the Intersensory Redundancy Hypothesis (IRH). Specifically, infants discriminate amodal properties in bimodal, redundant stimulation but not in unimodal, nonredundant stimulation in early development, whereas later in development infants can detect amodal properties in both redundant and nonredundant stimulation. The present study tested a new prediction of the IRH: that effects of intersensory redundancy on attention and perceptual processing are most apparent in tasks of high difficulty relative to the skills of the perceiver. We assessed whether by increasing task difficulty, older infants would revert to patterns of intersensory facilitation shown by younger infants. Results confirmed our prediction and demonstrated that in difficult tempo discrimination tasks, 5‐month‐olds perform like 3‐month‐olds, showing intersensory facilitation for tempo discrimination. In contrast, in tasks of low and moderate difficulty, 5‐month‐olds discriminate tempo changes in both redundant audiovisual and nonredundant unimodal visual stimulation. These findings indicate that intersensory facilitation is most apparent for tasks of relatively high difficulty and may therefore persist across the lifespan.     

Frame-differencing methods for measuring bodily synchrony in conversation

The study of interpersonal synchrony examines how interacting individuals grow to have similar behavior, cognition, and emotion in time. Many of the established methods of analyzing interpersonal synchrony are costly and time-consuming; the study of bodily synchrony has been especially laborious, traditionally requiring researchers to hand-code movement frame by frame. Because of this, researchers have been searching for more efficient alternatives for decades. Recently, some researchers (e.g., Nagaoka & Komori (IEICE Transactions on Information and Systems, 91(6), 1634–1640, 2008); Ramseyer & Tschacher, 2008) have applied computer science and computer vision techniques to create frame-differencing methods (FDMs) to simplify analyses. In this article, we provide a detailed presentation of one such FDM, created by modifying and adding to existing FDMs. The FDM that we present requires little programming experience or specialized equipment: Only a few lines of MATLAB code are required to execute an automated analysis of interpersonal synchrony. We provide sample code and demonstrate its use with an analysis of brief, friendly conversations; using linear mixed-effects models, the measure of interpersonal synchrony was found to be significantly predicted by time lag (p < .001) and by the interaction between time lag and measures of interpersonal liking (p < .001). This pattern of results fits with existing literature on synchrony. We discuss the current limitations and future directions for FDMs, including their use as part of a larger methodology for capturing and analyzing multimodal interaction.     

A test battery to assess intrasensory and intersensory development of young children.

The purpose of the study was to develop a battery of tests for use in evaluation of intra- and intersensory development of young children. A battery of 15 tests (4 visual, 4 auditory, 4 tactile-kinesthetic, and 3 intersensory) was administered to 109 normally developing 6- and 8-year-old and 32 slowly developing or learning disabled children. Interdependence of test items within each intrasensory and the intersensory category was determined; intercorrelations ranged from .00 to .78. Reliability estimates were also determined. Face validity was claimed for each item. The effects of age or developmental level on test performance were established. Based upon the interdependence of the tests, reliability estimates, and the capacity of the tests to discriminate among groups classified according to age or developmental level, a battery of 10 intra- and intersensory tests was proposed. The battery has 3 tests of visual perception-visual memory, dynamic depth perception, and size discrimination; 3 tests of auditory perception-auditory discrimination, auditory memory of related syllables, and auditory sequential memory of numbers; 2 tests of tactile-kinesthetic perception-tactile integration and movement awareness; and 2 tests of intersensory integration-auditory-tactile intergration and auditory-visual integration.     

Mother and infant coordinate heart rhythms through episodes of interaction synchrony

Animal studies demonstrated the powerful impact of maternal-infant social contact on the infant's physiological systems, yet the online effects of social interactions on the human infant's physiology remain poorly understood. Mothers and their 3-month old infants were observed during face-to-face interactions while cardiac output was collected from mother and child. Micro-analysis of the partners’ behavior marked episodes of gaze, affect, and vocal synchrony. Time-series analysis showed that mother and infant coordinate heart rhythms within lags of less than 1 s. Bootstrapping analysis indicated that the concordance between maternal and infant biological rhythms increased significantly during episodes of affect and vocal synchrony compared to non-synchronous moments. Humans, like other mammals, can impact the physiological processes of the attachment partner through the coordination of visuo-affective social signals.     

Entrainment and Modulation of Gesture–Speech Synchrony Under Delayed Auditory Feedback

Gesture–speech synchrony re‐stabilizes when hand movement or speech is disrupted by a delayed feedback manipulation, suggesting strong bidirectional coupling between gesture and speech. Yet it has also been argued from case studies in perceptual–motor pathology that hand gestures are a special kind of action that does not require closed‐loop re‐afferent feedback to maintain synchrony with speech. In the current pre‐registered within‐subject study, we used motion tracking to conceptually replicate McNeill's ( 1992 ) classic study on gesture–speech synchrony under normal and 150 ms delayed auditory feedback of speech conditions (NO DAF vs. DAF). Consistent with, and extending McNeill's original results, we obtain evidence that (a) gesture‐speech synchrony is more stable under DAF versus NO DAF (i.e., increased coupling effect), (b) that gesture and speech variably entrain to the external auditory delay as indicated by a consistent shift in gesture‐speech synchrony offsets (i.e., entrainment effect), and (c) that the coupling effect and the entrainment effect are co‐dependent. We suggest, therefore, that gesture–speech synchrony provides a way for the cognitive system to stabilize rhythmic activity under interfering conditions.     

Synchrony Preference: Why Some People Go With the Flow and Some Don't

As work in organizations becomes more fluid and fast paced the effective execution of tasks often requires people to be temporally adaptable in working with others. This paper brings to light the importance of understanding how people relate to time in the context of social interactions. By integrating the research on time and social motives, we develop a relational perspective about time. We introduce the construct of synchrony preference, an individual difference that describes a willingness to adapt one's pace and rhythm within social interactions for the purpose of creating synchrony with others. We develop a measure of synchrony preference that we validate using multiple methods across 4 studies and 7 samples, which include over 1,400 individuals. In particular, we establish a nomological network and show that synchrony preference predicts flexible pacing behaviors, interpersonal facilitation, contribution to team synchrony, contribution to team performance, and job dedication. Our results reveal that both scholars and practitioners can benefit from considering people's preference for synchrony when working with others.     

Temporal order judgment criteria are affected by synchrony judgment sensitivity

Synchrony perception for audio-visual stimulus pairs is typically studied by using temporal order judgment (TOJ) or synchrony judgment (SJ) tasks. Research has shown that estimates of the point of subjective simultaneity (PSS) obtained using these two methods do not necessarily correspond. Here, we investigate the hypothesis that the PSS estimate obtained in a TOJ task is shifted in the direction of the most sensitive part of the synchrony judgment curve, as obtained in an SJ task. The results confirm that criterion shifts in the TOJ task move the PSS toward regions of the audio-visual temporal interval continuum where discriminations are most sensitive.     

Professionalism: A Competency Cluster Whose Time Has Come

Despite the burgeoning literature on professionalism in other health professions, psychology lags behind in the level of attention given to this core competency. In this article, we review definitions from other health professions and how they address professionalism. Next, we review how this competency evolved within health service psychology (HSP), and we propose a definition. We offer an approach for assessing professionalism within HSP. Consideration is given to strategies and methods for providing effective education and training in this multifaceted competency. Finally, recommendations are made for creating a culture of professionalism within HSP and honoring psychology’s social contract with multiple publics.     

The role of visual-auditory “compellingness” in the ventriloquism effect: Implications for transitivity among the spatial senses

A magnitude estimation response procedure was used to evaluate the strength of visualauditory intersensory bias effects under conditions of spatial discrepancy. Maj or variables were the cognitive compellingness of the stimulus situation and instructions as to the unity or duality of the perceptual event. With a highly compelling stimulus situation and single-event instructions, subjects showed a very high visual bias of audition, a significant auditory bias of vision, and a sum of bias effects that indicated that their perception was fully consonant with the assumption of a single perceptual event. This finding reopens the possibility that the spatial modalities function as a transitive system, an outcome that Pick, Warren, and Hay (1969) had expected but did not obtain. Furthermore, the results support the model for intersensory interaction proposed by Welch and Warren (1980) with respect to the susceptibility of intersensory bias effects to several independent variables. Finally, a new means of assessing intersensory bias effects by the use of spatial separation threshold was demonstrated.