Knowing when to respond: the role of visual information in conversational turn exchanges

When engaging in conversation, we efficiently go back and forth with our partner, organizing our contributions in reciprocal turn-taking behavior. Using multiple auditory and visual cues, we make online decisions about when it is the appropriate time to take our turn. In two experiments, we demonstrated, for the first time, that auditory and visual information serve complementary roles when making such turn-taking decisions. We presented clips of single utterances spoken by individuals engaged in conversations in audiovisual, auditory-only or visual-only modalities. These utterances occurred either right before a turn exchange (i.e., ‘Turn-Ends’) or right before the next sentence spoken by the same talker (i.e., ‘Turn-Continuations’). In Experiment 1, participants discriminated between Turn-Ends and Turn-Continuations in order to synchronize a button-press response to the moment the talker would stop speaking. We showed that participants were best at discriminating between Turn-Ends and Turn-Continuations in the audiovisual condition. However, in terms of response synchronization, participants were equally precise at timing their responses to a Turn-End in the audiovisual and auditory-only conditions, showing no advantage of visual information. In Experiment 2, we used a gating paradigm, where increasing segments of Turns-Ends and Turn-Continuations were presented, and participants predicted if a turn exchange would occur at the end of the sentence. We found an audiovisual advantage in detecting an upcoming turn early in the perception of a turn exchange. Together, these results suggest that visual information functions as an early signal indicating an upcoming turn exchange while auditory information is used to precisely time a response to the turn end.     

The location-, word-, and arrow-based Simon effects: An ex-Gaussian analysis

Task-irrelevant spatial information, conveyed by stimulus location, location word, or arrow direction, can influence the response to task-relevant attributes, generating the location-, word-, and arrow-based Simon effects. We examined whether different mechanisms are involved in the generation of these Simon effects by fitting a mathematical ex-Gaussian function to empirical response time (RT) distributions. Specifically, we tested whether which ex-Gaussian parameters (μ, σ, and τ) show Simon effects and whether the location-, word, and arrow-based effects are on different parameters. Results show that the location-based Simon effect occurred on mean RT and μ but not on τ, and a reverse Simon effect occurred on σ. In contrast, a positive word-based Simon effect was obtained on all these measures (including σ), and a positive arrow-based Simon effect was evident on mean RT, σ, and τ but not μ. The arrow-based Simon effect was not different from the word-based Simon effect on τ or σ but was on μ and mean RT. These distinct results on mean RT and ex-Gaussian parameters provide evidence that spatial information conveyed by the various location modes are different in the time-course of activation.     

Visually induced gains in pitch discrimination: Linking audio-visual processing with auditory abilities

Perception is fundamentally a multisensory experience. The principle of inverse effectiveness (PoIE) states how the multisensory gain is maximal when responses to the unisensory constituents of the stimuli are weak. It is one of the basic principles underlying multisensory processing of spatiotemporally corresponding crossmodal stimuli that are well established at behavioral as well as neural levels. It is not yet clear, however, how modality-specific stimulus features influence discrimination of subtle changes in a crossmodally corresponding feature belonging to another modality. Here, we tested the hypothesis that reliance on visual cues to pitch discrimination follow the PoIE at the interindividual level (i.e., varies with varying levels of auditory-only pitch discrimination abilities). Using an oddball pitch discrimination task, we measured the effect of varying visually perceived vertical position in participants exhibiting a wide range of pitch discrimination abilities (i.e., musicians and nonmusicians). Visual cues significantly enhanced pitch discrimination as measured by the sensitivity index d’, and more so in the crossmodally congruent than incongruent condition. The magnitude of gain caused by compatible visual cues was associated with individual pitch discrimination thresholds, as predicted by the PoIE. This was not the case for the magnitude of the congruence effect, which was unrelated to individual pitch discrimination thresholds, indicating that the pitch-height association is robust to variations in auditory skills. Our findings shed light on individual differences in multisensory processing by suggesting that relevant multisensory information that crucially aids some perceivers’ performance may be of less importance to others, depending on their unisensory abilities.     

Recognition during recall failure: Semantic feature matching as a mechanism for recognition of semantic cues when recall fails

Research suggests that a feature-matching process underlies cue familiarity-detection when cued recall with graphemic cues fails. When a test cue (e.g., potchbork) overlaps in graphemic features with multiple unrecalled studied items (e.g., patchwork, pitchfork, pocketbook, pullcork), higher cue familiarity ratings are given during recall failure of all of the targets than when the cue overlaps in graphemic features with only one studied target and that target fails to be recalled (e.g., patchwork). The present study used semantic feature production norms (McRae et al., Behavior Research Methods, Instruments, & Computers, 37, 547–559, 2005) to examine whether the same holds true when the cues are semantic in nature (e.g., jaguar is used to cue cheetah). Indeed, test cues (e.g., cedar) that overlapped in semantic features (e.g., a_tree, has_bark, etc.) with four unretrieved studied items (e.g., birch, oak, pine, willow) received higher cue familiarity ratings during recall failure than test cues that overlapped in semantic features with only two (also unretrieved) studied items (e.g., birch, oak), which in turn received higher familiarity ratings during recall failure than cues that did not overlap in semantic features with any studied items. These findings suggest that the feature-matching theory of recognition during recall failure can accommodate recognition of semantic cues during recall failure, providing a potential mechanism for conceptually-based forms of cue recognition during target retrieval failure. They also provide converging evidence for the existence of the semantic features envisaged in feature-based models of semantic knowledge representation and for those more concretely specified by the production norms of McRae et al. (Behavior Research Methods, Instruments, & Computers, 37, 547–559, 2005).     

Interference Control in Children with Attention Deficit/Hyperactivity Disorder

The view that Attention Deficit/Hyperactivity Disorder (ADHD) is associated with a diminished ability to control interfference is controversial and based exclusively on results of (verbal)-visual interference tasks, primarily the Stroop Color Word task. The present study compares medication-naïve children with ADHD (n?=?35 and n?=?51 in Experiments 1 and 2, respectively) with normal controls (n?=?26 and n?=?32, respectively) on two interference tasks to assess interference control in both the auditory and the visual modality: an Auditory Stroop task and a Simon task. Both groups showed reliable but equal degrees of interference on both tasks, suggesting that children with ADHD do not differ from normal controls in their ability to control interference in either modality.     

Agonistic character displacement in social cognition of advertisement signals

Interspecific aggression between sibling species may enhance discrimination of competitors when recognition errors are costly, but proximate mechanisms mediating increased discriminative ability are unclear. We studied behavioral and neural mechanisms underlying responses to conspecific and heterospecific vocalizations in Alston’s singing mouse (Scotinomys teguina), a species in which males sing to repel rivals. We performed playback experiments using males in allopatry and sympatry with a dominant heterospecific (Scotinomys xerampelinus) and examined song-evoked induction of egr-1 in the auditory system to examine how neural tuning modulates species-specific responses. Heterospecific songs elicited stronger neural responses in sympatry than in allopatry, despite eliciting less singing in sympatry. Our results refute the traditional neuroethological concept of a matched filter and instead suggest expansion of sensory sensitivity to mediate competitor recognition in sympatry.     

Spatial term apprehension with a reference object’s rotation in three-dimensional space

It has been reported that the overall shapes of spatial categorical patterns of projective spatial terms such as above and below are not influenced by the rotation of a reference object on a two-dimensional (2D) upright plane. However, is this also true in three-dimensional (3D) space? This study shows the dynamic aspects of the apprehension of projective spatial terms in 3D space by detailing how the rotation of a reference object with an inherent front influences the apprehension of projective spatial terms on a level plane by mapping their spatial categorical patterns. The experiment was designed to examine how spatial categorical patterns on a level plane changed with the rotation of a reference object with an inherent front in 3D computer graphics space. We manipulated the rotation of a reference object with an inherent front at three levels (0°, 90°, and 180° rotations) and examined how such manipulation changed the overall spatial categorical patterns of four basic Japanese projective spatial terms: mae, ushiro, hidari, and migi (similar to in front of, behind, to the left of, and to the right of in English, respectively). The results show that spatial term apprehension was affected by the rotation of the reference object in 3D space. In particular, rotation influenced the mae–ushiro and hidari–migi systems differently. The results also imply that our understanding of projective spatial terms on a level plane in 3D space is affected dynamically by visual information from 3D cues.     

The Multilevel Modality-Switch Effect: What Happens When We See the Bees Buzzing and Hear the Diamonds Glistening

Previous studies demonstrated that the sequential verification of different sensory modality properties for concepts (e.g., BLENDER-loud; BANANA-yellow) brings about a processing cost, known as the modality-switch effect. We report an experiment designed to assess the influence of the mode of presentation (i.e., visual, aural) of stimuli on the modality-switch effect in a property verification and lexical decision priming paradigm. Participants were required to perform a property verification or a lexical decision task on a target sentence (e.g., “a BEE buzzes”, “a DIAMOND glistens”) presented either visually or aurally after having been presented with a prime sentence (e.g., “the LIGHT is flickering”, “the SOUND is echoing”) that could either share both, one or none of the target’s mode of presentation and content modality. Results show that the mode of presentation of stimuli affects the conceptual modality-switch effect. Furthermore, the depth of processing required by the task modulates the complex interplay of perceptual and semantic information. We conclude that the MSE is a task-related, multilevel effect which can occur on two different levels of information processing (i.e., perceptual and semantic).     

Use of the familiarity difference cue in inferential judgments

The familiarity difference cue has been regarded as a general cue for making inferential judgments (Honda, Abe, Matsuks, & Yamagishi in Memory and Cognition, 39(5), 851–863, 2011; Schwikert & Curran in Journal of Experimental Psychology: General, 143(6), 2341–2365, 2014). The current study tests a model of inference based on familiarity differences that encompasses the recognition heuristic (Goldstein & Gigerenzer, 1999, Goldstein & Gigerenzer in Psychological Review, 109(1), 75–90, 2002). In two studies, using a large pool of stimuli, participants rated their familiarity of cities and made choices on a typical city-size task. The data were fitted with the r-s model (Hilbig, Erdfelder, & Pohl in, Journal of Experimental Psychology: Learning Memory and Cognition, 37(4), 827–839, 2011), which was adapted to include familiarity differences. The results indicated that people used the familiarity difference cue because the participants ignored further knowledge in a substantial number of cases when the familiarity difference cue was available. An analysis of reaction-time data further indicated that the response times were shorter for heuristic judgments than for knowledge-only-based judgments. Furthermore, when knowledge was available, the response times were shorter when knowledge was congruent with a heuristic cue than when it was in conflict with it. Differences between the familiarity difference cue and the fluency heuristic (Schooler & Hertwig, 2005, Psychological Review, 112, 610–628) are discussed.     

Grammatical Aspect in Early Child Mandarin: Evidence from a Preferential Looking Experiment

The study assessed 30-month-old Mandarin-speaking children’s awareness of aspectual distinctions involving the perfective marker le and the imperfective marker zhe in a preferential looking experiment. In the experiment, we presented our child subjects with a choice between two video clips (one depicting a closed event and the other depicting an on-going event), in the presence of an auditory stimulus (either the le sentence, the zhe sentence or the control sentence without any aspect marker). Children’s looking behavior in the task was recorded and analyzed. The results revealed 30-month-old children’s emerging sensitivity to the aspectual contrast between le and zhe. This was manifest by an increase in looking to the closed event when hearing the le sentence and an increase in looking to the on-going event when hearing the zhe sentence. The absence of le or zhe in the control sentence did not result in any increase or decrease in looking to either event. We also found that the effect of le on children’s looking behavior was immediate whereas the effect of zhe was late. We attributed this difference to the facilitative role of le in children’s sentence processing as well as their preference for the event boundary. The results lend support to the continuity view that functional morphemes like aspect markers are available to children early in language development.     

ExpTime Tableau Decision Procedures for Regular Grammar Logics with Converse

Grammar logics were introduced by Fariñas del Cerro and Penttonen in 1988 and have been widely studied. In this paper we consider regular grammar logics with converse (REG ^c logics) and present sound and complete tableau calculi for the general satisfiability problem of REG ^c logics and the problem of checking consistency of an ABox w.r.t. a TBox in a REG ^c logic. Using our calculi we develop ExpTime (optimal) tableau decision procedures for the mentioned problems, to which various optimization techniques can be applied. We also prove a new result that the data complexity of the instance checking problem in REG ^c logics is coNP-complete.     

English Orthographic Learning in Chinese-L1 Young EFL Beginners

English orthographic learning, among Chinese-L1 children who were beginning to learn English as a foreign language, was documented when: (1) only visual memory was at their disposal, (2) visual memory and either some letter-sound knowledge or some semantic information was available, and (3) visual memory, some letter-sound knowledge and some semantic information were all available. When only visual memory was available, orthographic learning (measured via an orthographic choice test) was meagre. Orthographic learning was significant when either semantic information or letter-sound knowledge supplemented visual memory, with letter-sound knowledge generating greater significance. Although the results suggest that letter-sound knowledge plays a more important role than semantic information, letter-sound knowledge alone does not suffice to achieve perfect orthographic learning, as orthographic learning was greatest when letter-sound knowledge and semantic information were both available. The present findings are congruent with a view that the orthography of a foreign language drives its orthographic learning more than L1 orthographic learning experience, thus extending Share’s (Cognition 55:151–218, 1995) self-teaching hypothesis to include non-alphabetic L1 children’s orthographic learning of an alphabetic foreign language. The little letter-sound knowledge development observed in the experiment-I control group indicates that very little letter-sound knowledge develops in the absence of dedicated letter-sound training. Given the important role of letter-sound knowledge in English orthographic learning, dedicated letter-sound instruction is highly recommended.     

Statistical properties of four effect-size measures for mediation models

This project examined the performance of classical and Bayesian estimators of four effect size measures for the indirect effect in a single-mediator model and a two-mediator model. Compared to the proportion and ratio mediation effect sizes, standardized mediation effect-size measures were relatively unbiased and efficient in the single-mediator model and the two-mediator model. Percentile and bias-corrected bootstrap interval estimates of ab/s _Y , and ab(s _X )/s _Y in the single-mediator model outperformed interval estimates of the proportion and ratio effect sizes in terms of power, Type I error rate, coverage, imbalance, and interval width. For the two-mediator model, standardized effect-size measures were superior to the proportion and ratio effect-size measures. Furthermore, it was found that Bayesian point and interval summaries of posterior distributions of standardized effect-size measures reduced excessive relative bias for certain parameter combinations. The standardized effect-size measures are the best effect-size measures for quantifying mediated effects.     

Parent-Offspring Transmission of Internalizing and Sensory over-Responsivity Symptoms in Adolescence

Reactions to sensory experiences are an overlooked correlate of affective regulation, despite the importance of bodily states on psychological processes. Children who display sensory over-responsivity (i.e., adverse reactions to typical sensations) are at greater risk for developing affective disorders. We extended this literature to adolescents and their middle-aged parents. Participants in a birth record-based study of families of adolescent twins (N = 506 families; 1012 adolescents; 53% female) completed a subset of items from the Adult Sensory Profile. We derived adolescent self-reported internalizing disorder symptoms and parent affective diagnoses from structured diagnostic interviews. Structural equation models tested the relationship between parent sensory over-responsivity symptoms and affective diagnoses and their adolescent offspring’s sensory over-responsivity and internalizing symptoms. Adolescent sensory over-responsivity symptoms were correlated with internalizing disorder symptoms. Parents with a diagnosis of anxiety or depression (mothers only) reported more frequent SOR symptoms than parents without a diagnosis. Parent depression was significantly related to adolescent sensory over-responsivity symptoms, over and above parent sensory over-responsivity symptoms (β = 0.26, p < 0.001 for mothers; β = 0.13, p = 0.004 for fathers). Father alcohol abuse/dependency also predicted offspring sensory over-responsivity symptoms. Offspring of parents with affective disorders were at additional risk for sensory dysregulation via parents’ influence on offspring internalizing problems.     

Assessing the Approximate Number System: no relation between numerical comparison and estimation tasks

Whether our general numerical skills and the mathematical knowledge that we acquire at school are entwined is a debated issue, which many researchers are still striving to investigate. The findings reported in the literature are actually inconsistent; some studies emphasized the existence of a relationship between the acuity of the Approximate Number System (ANS) and arithmetic competence, while some others did not observe any significant correlation. One potential explanation of the discrepancy might stem from the evaluation of the ANS itself. In the present study, we correlated two measures used to index ANS acuity with arithmetic performance. These measures were the Weber fraction (w), computed from a numerical comparison task and the coefficient of variation (CV), computed from a numerical estimation task. Arithmetic performance correlated with estimation CV but not with comparison w. We further investigated the meaning of this result by taking the relationship between w and CV into account. We expected a tight relation as both these measures are believed to assess ANS acuity. Crucially, however, w and CV did not correlate with each other. Moreover, the value of w was modulated by the congruity of the relation between numerical magnitude and non-numerical visual cues, potentially accounting for the lack of correlation between the measures. Our findings thus challenge the overuse of w to assess ANS acuity and more generally put into question the relevance of correlating this measure with arithmetic without any deeper understanding of what they are really indexing.     

Color and intensity discrimination in <Emphasis Type="Italic">Xenopus laevis</Emphasis> tadpoles

Investigations into the physiology of Xenopus laevis have the potential to greatly accelerate biomedical research, especially concerning neural plasticity and sensory systems, but are limited by the lack of available information on behavioral learning in this species. Here, we attempt to lay the foundations for a behavioral assay in Xenopus that can be used in conjunction with biological manipulations. We tested cohorts of Xenopus tadpoles across four light-mediated active-avoidance experiments, using either wavelength or intensity as the salient discriminative cue. In the wavelength task, we determine a baseline learning rate and characterize retention of learning, identifying active extinction effects as far more potent than the passage of time in the loss of behavior. In the intensity task, we examine the effects of varying differences between the discriminative stimuli on acquisition and extinction and identify a critical range of intensity differences where learning changes. The results of our experiments demonstrate that Xenopus is a tractable model organism for cognitive research and can learn a variety of associative tasks in the laboratory settings. These data reveal new aspects of the Xenopus larval visual processing system and facilitate future research between cognitive methods and biological/chemical manipulations to study mechanisms of brain structure and function.     

Meta-Analysis of the Effects of Computerized Cognitive Training on Executive Functions: a Cross-Disciplinary Taxonomy for Classifying Outcome Cognitive Factors

The growing prevalence of neurodegenerative disorders associated with aging and cognitive decline has generated increasing cross-disciplinary interest in non-pharmacological interventions, such as computerized cognitive training (CCT), which may prevent or slow cognitive decline. However, inconsistent findings across meta-analytic reviews in the field suggest a lack of cross-disciplinary consensus and on-going debate regarding the benefits of CCT. We posit that a contributing factor is the lack of a theoretically-based taxonomy of constructs and representative tasks typically used. An integration of the Cattell-Horn-Carroll (CHC) taxonomy of broad and narrow cognitive factors and the Miyake unity-diversity theory of executive functions (EF) is proposed (CHC-M) as an attempt to clarify this issue through representing and integrating the disciplines contributing to CCT research. The present study assessed the utility of this taxonomy by reanalyzing the Lampit et al. (2014) meta-analysis of CCT in healthy older adults using the CHC-M framework. Results suggest that: 1) substantively different statistical effects are observed when CHC-M is applied to the Lampit et al. meta-analytic review, leading to importantly different interpretations of the data; 2) typically-used classification practices conflate Executive Function (EF) tasks with fluid reasoning (Gf) and retrieval fluency (Gr), and Attention with sensory perception; and 3) there is theoretical and practical advantage in differentiating attention and working-memory tasks into the narrow shifting, inhibition, and updating EF domains. Implications for clinical practice, particularly for our understanding of EF are discussed.     

Determination of the number of independent parameters of a score matrix from the examination of rank orders

Two ordinal consequences are drawn from the linear multiple-factor analysis model. First, the numberR(s, d) of distinct ways in whichs subjects can be ranked by linear functions ofd factors is limited by the recursive expressionR(s, d)=R(s?1,d)+(s?1)R(s?1,d?1). Second, every setS ofd+2 subjects can be separated into two subsetsS* andS ? S* such that no linear function ofd variables can rank allS* over allS ? S*, and vice versa. When these results are applied to the hypothetical data of Thurstone's “box problem,” three independent parameters are found. Relations to Thurstone's suggestion for a non-correlational factor analysis are discussed.     

Subjunctive Conditional Probability

There seem to be two ways of supposing a proposition: supposing “indicatively” that Shakespeare didn’t write Hamlet, it is likely that someone else did; supposing “subjunctively” that Shakespeare hadn’t written Hamlet, it is likely that nobody would have written the play. Let P(B//A) be the probability of B on the subjunctive supposition that A. Is P(B//A) equal to the probability of the corresponding counterfactual, A □→B? I review recent triviality arguments against this hypothesis and argue that they do not succeed. On the other hand, I argue that even if we can equate P(B//A) with P(A □→B), we still need an account of how subjunctive conditional probabilities are related to unconditional probabilities. The triviality arguments reveal that the connection is not as straightforward as one might have hoped.     

Reasoning defeasibly about probabilities

In concrete applications of probability, statistical investigation gives us knowledge of some probabilities, but we generally want to know many others that are not directly revealed by our data. For instance, we may know prob(P/Q) (the probability of P given Q) and prob(P/R), but what we really want is prob(P/Q&;R), and we may not have the data required to assess that directly. The probability calculus is of no help here. Given prob(P/Q) and prob(P/R), it is consistent with the probability calculus for prob(P/Q&;R) to have any value between 0 and 1. Is there any way to make a reasonable estimate of the value of prob(P/Q&;R)? A related problem occurs when probability practitioners adopt undefended assumptions of statistical independence simply on the basis of not seeing any connection between two propositions. This is common practice, but its justification has eluded probability theorists, and researchers are typically apologetic about making such assumptions. Is there any way to defend the practice? This paper shows that on a certain conception of probability—nomic probability—there are principles of “probable probabilities” that license inferences of the above sort. These are principles telling us that although certain inferences from probabilities to probabilities are not deductively valid, nevertheless the second-order probability of their yielding correct results is 1. This makes it defeasibly reasonable to make the inferences. Thus I argue that it is defeasibly reasonable to assume statistical independence when we have no information to the contrary. And I show that there is a function Y(r, s, a) such that if prob(P/Q) = r, prob(P/R) = s, and prob(P/U) = a (where U is our background knowledge) then it is defeasibly reasonable to expect that prob(P/Q&;R) = Y(r, s, a). Numerous other defeasible inferences are licensed by similar principles of probable probabilities. This has the potential to greatly enhance the usefulness of probabilities in practical application.