Problematic effects of aggregation in z ROC analysis and a hierarchical modeling solution

The confidence-rating recognition memory paradigm has been used extensively for testing theories of memory. Current methods for analyzing confidence-rating data require that data be aggregated over participants, items, or both. We show that this aggregation threatens claims of curvature in zROCs and distorts estimates of signal-distribution variance—the very findings that are used to make fine-grained distinctions between competing models. We develop a hierarchical signal detection model that accounts for variability from the sampling of participants and items in addition to variability from the underlying psychological processes. The model provides for accurate signal detection parameter estimates. With accurate estimates, the validity of benchmark findings in recognition memory may be assessed.     

Decision processes in temporal discrimination

The processing dynamics underlying temporal decisions and the response times they generate have received little attention in the study of interval timing. In contrast, models of other simple forms of decision making have been extensively investigated using response times, leading to a substantial disconnect between temporal and non-temporal decision theories. An overarching decision-theoretic framework that encompasses existing, non-temporal decision models may, however, account both for interval timing itself and for time-based decision-making. We sought evidence for this framework in the temporal discrimination performance of humans tested on the temporal bisection task. In this task, participants retrospectively categorized experienced stimulus durations as short or long based on their perceived similarity to two, remembered reference durations and were rewarded only for correct categorization of these references. Our analysis of choice proportions and response times suggests that a two-stage, sequential diffusion process, parameterized to maximize earned rewards, can account for salient patterns of bisection performance. The first diffusion stage times intervals by accumulating an endogenously noisy clock signal; the second stage makes decisions about the first-stage temporal representation by accumulating first-stage evidence corrupted by endogenous noise. Reward-maximization requires that the second-stage accumulation rate and starting point be based on the state of the first-stage timer at the end of the stimulus duration, and that estimates of non-decision-related delays should decrease as a function of stimulus duration. Results are in accord with these predictions and thus support an extension of the drift–diffusion model of static decision making to the domain of interval timing and temporal decisions.     

Time Pressure Heuristics Can Improve Performance Due to Increased Consistency

Our goal is to demonstrate that potential performance theory (PPT) provides a unique type of methodology for studying the use of heuristics under time pressure. While most theories tend to focus on different types of strategies, PPT distinguishes between random and nonrandom effects on performance. We argue that the use of a heuristic under time pressure actually can increase performance by decreasing randomness in responding. We conducted an experiment where participants performed a task under time pressure or not. In turn, PPT equations make it possible to parse the observed change in performance from the unspeeded to the speeded condition into that which is due to a change in the participant's randomness in responding versus that which is due to a change in systematic factors. We found that the change in randomness was slightly more important than the change in systematic factors.     

Negative valence can evoke a liberal response bias in syllogistic reasoning

Recently, studies have demonstrated that negative valence reduces the magnitude of the belief-bias effect in syllogistic reasoning. This effect has been localized in the reasoning stage, in the form of increased deliberation on trials where validity and conclusion believability are incongruent. Here, using signal detection theory, we show that the attenuation of belief bias observed when valence was negative can also be evoked by a liberal response bias at the decision stage. Indeed, when valence was negative participants adopted a more liberal criterion for judging syllogisms as “valid,” and were overconfident in their judgments. They also displayed less sensitivity in distinguishing between valid and invalid syllogisms. Our findings dovetail with recent evidence from memory research suggesting that negative valence can evoke a liberal response bias without improving performance. Our novel contribution is the demonstration that the attenuating effect of negative valence on belief bias can take multiples routes—by influencing the decision stage as was the case here, the reasoning stage as has been demonstrated elsewhere, and potentially both stages.     

Vibrotactile vigilance: The effects of costs and values on signals

Vibrotactile vigilance behavior was examined for the effects on signal detection performance of placing costs on misses and false detections and a positive value on the correct detection of a signal. Analyses over time showing that the frequencies of correct detection and false detection of signals decreased, while response times increased, indicated that increased costs for misses and false alarms lead to poorer correct detection performance, while the value placed on the correct detection of a signal had little effect. The d’ statistic of signal detection theory was invariant with both signal costs and time, while β varied with both factors.These results imply that the performance decrement during a vigil was due to an increased strictness in the criterion (β) the S set for deciding whether or not a signal was present. The cost factors were effective in manipulating performance during the watch by causing changes in the S’s decision criteria. Findings from this study clearly support those predicted by the “Decision Theory” of Swets, Tanner, and Birdsall (1961); this is suggested as perhaps being the best available basis for a theory of vigilance at this time.     

The independence of endogenous and exogenous temporal attention

Temporal attention is the focusing of perceptual resources at a particular point in time. Valid temporal cue information has the capability to improve performance by reducing reaction times, while invalid information has the possibility of impairing performance. The performance difference between valid and invalid conditions is called a temporal cueing effect (TCE). We explored how different alerting mechanisms interact with a participant’s ability to utilize temporal information cues, using the Kingstone (The Quarterly Journal of Experimental Psychology, 44(1), 69–104, 1992) temporal cueing paradigm. Extracting the alerting procedure from Lawrence and Klein (Journal of Experimental Psychology: General, 142(2), 560–572, 2013), one of two different temporally contingent warning signals were presented to participants during a trial. The “hi-intensity” warning signal increases intensity and elicits both exogenous and endogenous alerting mechanisms. The “no-intensity” warning signal is isointense relative to baseline and elicits only endogenous alerting mechanisms. Two experiments conducted previously using a discrimination task showed interference between the signal intensity and task difficulty, where the “no-intensity” signal failed to elicit TCEs. In the present study, we implemented a detection task, reducing the mental effort required for a response. The results showed equal TCEs in both signal conditions. We argue for independence of these alerting mechanisms, by way of Sternberg’s (Acta Psychologica, 30, 276–315, 1969) additive factor method. Arguments contrasting what mechanism is being impacted by this paradigm are further outlined.     

Can bottom-up processes of attention be a source of ‘interference’ in situations where top-down control of attention is crucial?

In this study, we investigate whether emotionally engaged bottom-up processes of attention can be a source of ‘interference’ in situations where top-down control of attention is necessary. Participants were asked to monitor and report on a video of a war scenario showing a developing battle in two conditions: emotionally positive and emotionally negative. Half of the participants (n = 15) were exposed to task-irrelevant pictures of positive emotional valence embedded within the scenario; the other half were exposed to task-irrelevant pictures of negative emotional valence. Sensitivity and Bias scores were calculated using signal detection theory. Overall, task accuracy scores were dependent upon the valence; negative pictures had an adverse effect on performance, whereas positive pictures improved performance. We concluded that negative emotional pictures interfered with top-down control of attention by attracting competing bottom-up processes of attention. We found the opposite effect for positive emotional stimuli.     

Associations between cortical thickness and general intelligence in children,adolescents and young adults

Neuroimaging research indicates that human intellectual ability is related to brain structure including the thickness of the cerebral cortex. Most studies indicate that general intelligence is positively associated with cortical thickness in areas of association cortex distributed throughout both brain hemispheres. In this study, we performed a cortical thickness mapping analysis on data from 182 healthy typically developing males and females ages 9 to 24 years to identify correlates of general intelligence (g) scores. To determine if these correlates also mediate associations of specific cognitive abilities with cortical thickness, we regressed specific cognitive test scores on g scores and analyzed the residuals with respect to cortical thickness. The effect of age on the association between cortical thickness and intelligence was examined. We found a widely distributed pattern of positive associations between cortical thickness and g scores, as derived from the first unrotated principal factor of a factor analysis of Wechsler Abbreviated Scale of Intelligence (WASI) subtest scores. After WASI specific cognitive subtest scores were regressed on g factor scores, the residual score variances did not correlate significantly with cortical thickness in the full sample with age covaried. When participants were grouped at the age median, significant positive associations of cortical thickness were obtained in the older group for g-residualized scores on Block Design (a measure of visual-motor integrative processing) while significant negative associations of cortical thickness were observed in the younger group for g-residualized Vocabulary scores. These results regarding correlates of general intelligence are concordant with the existing literature, while the findings from younger versus older subgroups have implications for future research on brain structural correlates of specific cognitive abilities, as well as the cognitive domain specificity of behavioral performance correlates of normative gray matter thinning during adolescence.     

Processing time predictions of current models of perception in the classic additive factors paradigm

This article explores the consequences for factorial additivity in a Sternberg [(1969). The discovery of processing stages: Extensions of donders method In: W.G. Koster (Ed.), Attention and performance II, Acta Psychologica, 30, 276-315] additive-factors paradigm of the assumptions adopted by models of perception that relate the representation of a stimulus to decision time. Three example models, signal detection theory with the latency-distance hypothesis, stochastic general recognition theory, and a random walk model of exemplar classification, are interrogated to determine what type of interaction they predict factors will yield in a hypothetical factorial (choice) reaction time experiment in which the ‘empirical’ factors’ effects are manifest as parameter changes. All frameworks make the critical assumption that decision time depends on the perceptual representation of the stimulus as well as the architecture. As a consequence, nonadditivity of factors thought to affect different “stages” in the classical approach emerges within the current modeling approach. The nature of this influence is revealed through analytic investigations and simulation. Earlier empirical findings of failures of selective influence that have defied adequate explanation are reinterpreted in light of the present findings.     

Lateralisation of language and emotion in schizotypal personality: Evidence from dichotic listening

Striking disturbances have been reported in language and emotional prosody processing by patients diagnosed with schizophrenia. In view of this and of research suggesting that schizotypal personality traits can also be expressed sub-clinically, the present study aimed to discover whether similar disturbances would be reflected in cognitive laterality patterns when symptoms of schizotypy are present yet at a non-clinical level. A dichotic listening task was used to examine the sensitivity and speed with which 132 right-handed participants (85 females and 47 males, mean age = 32.44, SD = 12.29) detected both words and emotional prosody, all of whom also completed the Schizotypal Personality Questionnaire. Findings indicated that both high (n = 64) and low (n = 68) schizotypy groups demonstrated the typical right ear advantage for the detection of words and left ear advantage for the detection of emotional prosody. Individuals with higher schizotypal personality scores also demonstrated poorer sensitivity in detecting emotional prosody. These results reveal that within the healthy population, higher levels of schizotypy are not associated with the atypical lateralisation of language and emotion. Nevertheless, the existence of these symptoms does signal the presence of shared characteristics with the clinical sphere, namely poorer emotion recognition performance.     

Effects of perceptual context on transsaccadic visual matching

Transsaccadic visual matching refers to the phenomenon in which presaccadic signals at the location of the saccade goal influence the visibility of postsaccadic test signals presented at the fovea (Jüttner & Röhler, 1993). The present study uses variations of the perceptual context, provided by the structure of the presaccadic stimulus display, to investigate the influence of spatial information, decision factors, and visual attention on this form of transsaccadic information processing. The experiments yielded the following results: First, analysis of the data in terms of signal detection theory revealed that transsaccadic visual matching manifests in a shift of decision criterion (ß) rather than in a change of sensory sensitivity (d′). The criterion shift leads to a suppression of postsaccadic stimulus information that is incompatible with presaccadic processed information. Second, the matching effect strongly depends on the structure of the presaccadic display, which suggests that mechanisms of visual attention provide the spatial binding of pre- and postsaccadic stimulus information. Third, transsaccadic matching is phenomenologically tied to the execution of the saccade and effective during a postsaccadic period extending up to 160 msec after termination of the eye movement. These findings indicate a form of context-sensitive evaluation process that could subsidize the maintenance of visual stability.     

Characterizing perceptual learning with external noise

Performance in perceptual tasks often improves with practice. This effect is known as ‘perceptual learning,’ and it has been the source of a great deal of interest and debate over the course of the last century. Here, we consider the effects of perceptual learning within the context of signal detection theory. According to signal detection theory, the improvements that take place with perceptual learning can be due to increases in internal signal strength or decreases in internal noise. We used a combination of psychophysical techniques (external noise masking and double-pass response consistency) that involve corrupting stimuli with externally added noise to discriminate between the effects of changes in signal and noise as observers learned to identify sets of unfamiliar visual patterns. Although practice reduced thresholds by as much as a factor of 14, internal noise remained virtually fixed throughout training, indicating learning served to predominantly increase the strength of the internal signal. We further examined the specific nature of the changes that took place in signal strength by correlating the externally added noise with observer’s decisions across trials (response classification). This technique allowed us to visualize some of the changes that took place in the linear templates used by the observers as learning occurred, as well as test the predictions of a linear template-matching model. Taken together, the results of our experiments offer important new theoretical constraints on models of perceptual learning.     

Decision processes in discrimination: fundamental misrepresentations of signal detection theory.

A new approach to studying decision making in discrimination tasks is described that does not depend on the technical assumptions of signal detection theory (e.g., normality of the encoding distributions). In 3 different experiments, results of these new distribution-free tests converge on a single, surprising conclusion: response biases had substantial effects on the encoding distributions but no effect on the decision rule, which was uniformly unbiased in equal and unequal base rate conditions and in symmetric and asymmetric payoff conditions. This seemingly paradoxical result is fundamentally inconsistent with the entire family of signal detection theory models, raising some important questions about the significance of many published results in the human performance literature.     

BOLD response to semantic and syntactic processing during hypoglycemia is load-dependent

This study investigates how syntactic and semantic load factors impact sentence comprehension and BOLD signal under moderate hypoglycemia. A dual session, whole brain fMRI study was conducted on 16 healthy participants using the glucose clamp technique. In one session, they experienced insulin-induced hypoglycemia (plasma glucose at ∼50 mg/dL); in the other, plasma glucose was maintained at euglycemic levels (∼100 mg/dL). During scans subjects were presented with sentences of contrasting syntactic (embedding vs. conjunction) and semantic (reversibility vs. irreversibility) load. Semantic factors dominated the overall load effects on both performance (p < 0.001) and BOLD response (p < 0.01, corrected). Differential BOLD signal was observed in frontal, temporal, temporo-parietal and medio-temporal regions. Hypoglycemia and syntactic factors significantly impacted performance (p = 0.002) and BOLD response (p < 0.01, corrected) in the reversible clause conditions, more extensively in reversible-embedded than in reversible-conjoined clauses. Hypoglycemia resulted in a robust decrease in performance on reversible clauses and exerted attenuating effects on BOLD unselectively across cortical circuits.The dominance of reversibility in all measures underscores the distinction between the syntactic and semantic contrasts. The syntactic is based in a quantitative difference in algorithms interpreting embedded and conjoined structures. We suggest that the semantic is based in a qualitative difference between algorithmic mapping of arguments in reversible clauses and heuristic linking in irreversible clauses. Because heuristics drastically reduce resource demand, the operations they support would resist the load-dependent cognitive consequences of hypoglycemia.     

Bisensory order judgement and the prior entry hypothesis.

Non-parametric signal detection analysis of the results of three order judgement experiments are reported in which Ss judged the order of visual and auditory signal pairs. Under one condition they only made the order judgement, under others they made both the order judgement and a decision about the visual or the auditory signal. No tendency to report the attended signal as coming first was evident under the latter conditions, contradicting the predictions of the prior entry hypothesis, although a significant decrease in sensitivity was observed under these conditions.     

Assessment of calibration for reconstructed eye-witness memories

Two experiments examined confidence judgments associated with eyewitness memory for events to misinformation [Loftus, E. F. (1975). Leading questions and the eyewitness report. Cognitive Psychology, 7, 560-572]. Participants in each experiment watched videos depicting crimes and either responded to questionnaires (Experiment 1) or read narratives (Experiment 2) containing veridical or non-veridical information. Upon testing for memory performance 24 h later, participants exposed to misinformation accepted the misinformation as part of their memory of the events in the videos. In both experiments, confidence judgments of participants subjected to misinformation were high and comparable to those of participants not subjected to misinformation. The relationship between accurate responding and confidence was assessed via calibration analysis [Yates, J. F. (1990). Judgments and decision making. Englewood Cliffs, NJ: Prentice Hall]. Performance on various measures of calibration was generally better for participants who were not exposed to misinformation. We conclude that confidence judgments can depend on factors other than those that improve the accuracy of memory, such as the retrieval fluency of one’s memories.     

A signal detection analysis of gist-based discrimination of genetic breast cancer risk

Pervasive biases in probability judgment render the probability scale a poor response mode for assessing risk judgments. By applying fuzzy trace theory, we used ordinal gist categories as a response mode, coupled with a signal detection model to assess risk judgments. The signal detection model is an extension of the familiar model used in binary choice paradigms. It provides three measures of discriminability—low versus medium risk, medium versus high risk, and low versus high risk—and two measures of response bias. We used the model to assess the effectiveness of BRCA Gist, an intelligent tutoring system designed to improve women’s judgments and understanding of genetic risk for breast cancer. Participants were randomly assigned to the BRCA Gist intelligent tutoring system, the National Cancer Institute (NCI) Web pages, or a control group, and then they rated cases that had been developed using the Pedigree Assessment Tool and also vetted by medical experts. BRCA Gist participants demonstrated increased discriminability for all three risk categories, relative to the control group; the NCI group showed increased discriminability for two of the three levels. This result suggests that BRCA Gist best improved discriminability among genetic risk categories, and both BRCA Gist and the NCI website improved participants’ ability to discriminate, rather than simply shifting their decision criterion. A spreadsheet that fits the model and compares parameters across the conditions can be downloaded from the Behavior Research Methods website and used in any research involving categorical responses.     

Short-term proactive inhibition in the pigeon

Proactive inhibition in the pigeon was studied using a delayed-color-matching-to-sample task, with and without presentation of stimuli prior to the sample. Interference theory predicts that, relative to control performance, the disruptive effects of a prior stimulus should increase over a retention interval, while decay theory predicts that such disruptive effects should decrease over time. Results supported decay theory when performance was at a low level and interference theory when performance was at a high level. With performance at an intermediate level, parallel functions were obtained, an outcome supporting neither theory. It was concluded that the results supporting interference and decay theories were artifactually produced by floor and ceilling effects, respectively, and it was suggested that the results of earlier experiments using the present paradigm may well have been influenced by similar artifacts. Inserting a nondisruptive stimulus between a prior disruptive stimulus and the sample virtually eliminated the disruptive effects of the prior stimulus when tested immediately, but not when tested after 1 sec. The results of the present experiments and related findings can best be explained by either a modified decay theory (Grant, D. S. Proactive interference in pigeon short-term memory. Journal of Experimental Psychology: Animal Behavior Processes 1975, 104, 207–220) or a modified temporal discrimination theory.     

Table of criterion values (β) used in signal detection theory

A table of criterion values for use with the theory of signal detection (TSD) is presented. Criterion values (β) are presented for varying proportions of hits and false alarms. A brief explanation of signal detection theory is given, and the measures for sensory sensitivity (d′) and response criterion (β) are described.