High working memory capacity does not always attenuate distraction: Bayesian evidence in support of the null hypothesis

Individual differences in working memory capacity (WMC) predict individual differences in basically all tasks that demand some form of cognitive labor, especially if the persons conducting the task are exposed to distraction. As such, tasks that measure WMC are very useful tools in individual-differences research. However, the predictive power of those tasks, combined with conventional statistical tools that cannot support the null hypothesis, also makes it difficult to study the limits of that power. In this article, we review studies that have failed to find a relationship between WMC and effects of auditory distraction on visual–verbal cognitive performance, and use meta-analytic Bayesian statistics to test the null hypothesis. The results favor the assumption that individual differences in WMC are, in fact, not (always) related to the magnitude of distraction. Implications for the nature of WMC are discussed.     

EVALUATION OF TASK AND JOB SKILL LINKAGE JUDGMENTS USED TO DEVELOP TEST SPECIFICATIONS

The research on job analysis judgments, such as "time spent," has been relatively limited, particularly with reference to external criteria remote from the job analysis operation. The more complex job analysis judgment linking a job skill to specific tasks or duties has not been systematically examined. While it would appear that a simple scaling of importance of a skill for a task or duty or a retranslation judgment would suffice, the fact is that a single job skill may be a prerequisite for performance in a variety of tasks, and any one task may require multiple skills of varying levels for effective performance. With a multiple assignment of tasks to job skills, the evaluation becomes considerably more difficult. In the present study, a sequence of statistical evaluations was conducted to examine, first, the reliability of the subject matter expert (SME) panel's association of tasks and job skills and, second, the factor structure of the task by job skill relationship. The results are discussed with reference to developing selection test specifications and test budgets.     

One-year test–retest reliability of the Maryland Resource for the Behavioral Utilization of the Reinforcement of Negative Stimuli (MRBURNS)

The Maryland Resource for the Behavioral Utilization of the Reinforcement of Negative Stimuli (MRBURNS) is a novel behavioral task designed to measure individual differences in negative reinforcement-based risk taking propensity. Performance on the MRBURNS has been linked with alcohol-related problems and negative reinforcement-based drinking motives, as well as symptoms of anxiety and depression; however, it is unclear if performance on the task represents a stable measure of negative reinforcement-based risk taking over time. As such, the current study aimed to examine the test–retest reliability of the MRBURNS over a period of one year. Results indicate that the correlation between year 1 and year 2 risk behavior (average number of pumps) on the MRBURNS was .43 across all trials. With the one year test–retest reliability of the MRBURNS established, the MRBURNS may be a useful approach to measuring the relative contribution of negative reinforcement-based risk taking in the development of risky behaviors over time, and may be used to monitor the effects of novel interventions that aim to reduce negative reinforcement based risk taking in the real world.     

Statistically Induced Chunking Recall: A Memory-Based Approach to Statistical Learning

The computations involved in statistical learning have long been debated. Here, we build on work suggesting that a basic memory process, chunking, may account for the processing of statistical regularities into larger units. Drawing on methods from the memory literature, we developed a novel paradigm to test statistical learning by leveraging a robust phenomenon observed in serial recall tasks: that short-term memory is fundamentally shaped by long-term distributional learning. In the statistically induced chunking recall (SICR) task, participants are exposed to an artificial language, using a standard statistical learning exposure phase. Afterward, they recall strings of syllables that either follow the statistics of the artificial language or comprise the same syllables presented in a random order. We hypothesized that if individuals had chunked the artificial language into word-like units, then the statistically structured items would be more accurately recalled relative to the random controls. Our results demonstrate that SICR effectively captures learning in both the auditory and visual modalities, with participants displaying significantly improved recall of the statistically structured items, and even recall specific trigram chunks from the input. SICR also exhibits greater test–retest reliability in the auditory modality and sensitivity to individual differences in both modalities than the standard two-alternative forced-choice task. These results thereby provide key empirical support to the chunking account of statistical learning and contribute a valuable new tool to the literature.     

The reliability and stability of the turner and Engle working memory task

The present study explored the psychometric properties of Turner and Engle’s (1989) operation span task, a widely used measure of working memory capacity. We administered the task three times to 33 college students, using equivalent test materials. The interval between the first and second administrations was 3 weeks, with 6–7 weeks between the second and third administrations. Alpha coefficients were all .75 or more. Recall accuracy decreased as operation set size increased. Raw test-retest correlations ranged from .67 to .81, the corrected reliability was .88, and stability scores ranged from .76 to .92. Performance improved from the first to the second test. Relative to reported reliabilities of other tasks used to assess individual differences in working memory capacity, the operation span task appears to have several statistical advantages.     

The test–retest reliability of the latent construct of executive function depends on whether tasks are represented as formative or reflective indicators

This study investigates the test–retest reliability of a battery of executive function (EF) tasks with a specific interest in testing whether the method that is used to create a battery-wide score would result in differences in the apparent test–retest reliability of children’s performance. A total of 188 4-year-olds completed a battery of computerized EF tasks twice across a period of approximately two weeks. Two different approaches were used to create a score that indexed children’s overall performance on the battery—i.e., (1) the mean score of all completed tasks and (2) a factor score estimate which used confirmatory factor analysis (CFA). Pearson and intra-class correlations were used to investigate the test–retest reliability of individual EF tasks, as well as an overall battery score. Consistent with previous studies, the test–retest reliability of individual tasks was modest (rs ≈ .60). The test–retest reliability of the overall battery scores differed depending on the scoring approach (r_mean = .72; r_{factor_score} = .99). It is concluded that the children’s performance on individual EF tasks exhibit modest levels of test–retest reliability. This underscores the importance of administering multiple tasks and aggregating performance across these tasks in order to improve precision of measurement. However, the specific strategy that is used has a large impact on the apparent test–retest reliability of the overall score. These results replicate our earlier findings and provide additional cautionary evidence against the routine use of factor analytic approaches for representing individual performance across a battery of EF tasks.     

fMRI reliability: Influences of task and experimental design

As scientists, it is imperative that we understand not only the power of our research tools to yield results, but also their ability to obtain similar results over time. This study is an investigation into how common decisions made during the design and analysis of a functional magnetic resonance imaging (fMRI) study can influence the reliability of the statistical results. To that end, we gathered back-to-back test–retest fMRI data during an experiment involving multiple cognitive tasks (episodic recognition and two-back working memory) and multiple fMRI experimental designs (block, event-related genetic sequence, and event-related m-sequence). Using these data, we were able to investigate the relative influences of task, design, statistical contrast (task vs. rest, target vs. nontarget), and statistical thresholding (unthresholded, thresholded) on fMRI reliability, as measured by the intraclass correlation (ICC) coefficient. We also utilized data from a second study to investigate test–retest reliability after an extended, six-month interval. We found that all of the factors above were statistically significant, but that they had varying levels of influence on the observed ICC values. We also found that these factors could interact, increasing or decreasing the relative reliability of certain Task × Design combinations. The results suggest that fMRI reliability is a complex construct whose value may be increased or decreased by specific combinations of factors.     

The reliability of eyetracking to assess attentional bias to threatening words in healthy individuals

Eyetracking is commonly used to investigate attentional bias. Although some studies have investigated the internal consistency of eyetracking, data are scarce on the test–retest reliability and agreement of eyetracking to investigate attentional bias. This study reports the test–retest reliability, measurement error, and internal consistency of 12 commonly used outcome measures thought to reflect the different components of attentional bias: overall attention, early attention, and late attention. Healthy participants completed a preferential-looking eyetracking task that involved the presentation of threatening (sensory words, general threat words, and affective words) and nonthreatening words. We used intraclass correlation coefficients (ICCs) to measure test–retest reliability (ICC > .70 indicates adequate reliability). The ICCs(2, 1) ranged from –.31 to .71. Reliability varied according to the outcome measure and threat word category. Sensory words had a lower mean ICC (.08) than either affective words (.32) or general threat words (.29). A longer exposure time was associated with higher test–retest reliability. All of the outcome measures, except second-run dwell time, demonstrated low measurement error (<6%). Most of the outcome measures reported high internal consistency (α > .93). Recommendations are discussed for improving the reliability of eyetracking tasks in future research.     

The Measurement of Verbal Working Memory Capacity and Its Relation to Reading Comprehension

Ninety-four subjects were tested on the Daneman and Carpenter (1980) reading span task, four versions of a related sentence span task in which reaction times and accuracy on sentence processing were measured along with sentence-final word recall, two number generation tasks designed to test working memory, digit span, and two shape-generation tasks designed to measure visual-spatial working memory. Forty-four subjects were retested on a subset of these measures at a 3-month interval. All subjects were tested on standard vocabulary and reading tests. Correlational analyses showed better internal consistency and test-retest reliability of the sentence span tasks than of the Daneman-Carpenter reading span task. Factor analysis showed no factor that could be related to a central verbal working memory; rotated factors suggested groupings of tests into factors that correspond to digitrelated tasks, spatial tasks, sentence processing in sentence span tasks, and recall in sentence span tasks. Correlational analyses and regression analyses showed that the sentence processing component of the sentence span tasks was the best predictor of performance on the reading test, with a small independent contribution of the recall component. The results suggest that sentence span tasks are unreliable unless measurements are made of both their sentence processing and recall components, and that the predictive value of these tasks for reading comprehension abilities lies in the overlap of operations rather than in limitations in verbal working memory that apply to both.     

A Monte Carlo simulation study of the reliability of intraindividual variability

Recent research has seen intraindividual variability become a useful technique to incorporate trial-to-trial variability into many types of psychological studies. Intraindividual variability, as measured by individual standard deviations (ISDs), has shown unique prediction to several types of positive and negative outcomes (Ram, Rabbit, Stollery, & Nesselroade, 2005). One unanswered question regarding measuring intraindividual variability is its reliability and the conditions under which optimal reliability is achieved. Monte Carlo simulation studies were conducted to determine the reliability of the ISD as compared with the intraindividual mean. The results indicate that ISDs generally have poor reliability and are sensitive to insufficient measurement occasions, poor test reliability, and unfavorable amounts and distributions of variability in the population. Secondary analysis of psychological data shows that use of individual standard deviations in unfavorable conditions leads to a marked reduction in statistical power, although careful adherence to underlying statistical assumptions allows their use as a basic research tool. (PsycINFO Database Record (c) 2012 APA, all rights reserved).     

回忆任务对合作抑制和错误修剪的调节——情绪效价和编码水平的影响

前人在项目回忆条件的合作记忆研究中记录到明显的合作抑制和错误修剪,背景提取条件的相关研究尤显不足;同时,情绪效价和编码水平对两类现象调节的研究尚未涉及。为此,本文在两个实验中采用经典合作记忆研究范式,以不同情绪效价的词汇为实验材料并以词汇在学习阶段的呈现颜色为背景展开研究。实验1和实验2分别在学习阶段采用了深编码和浅编码任务,回忆阶段则均含项目回忆(回忆已学词汇)和背景提取(回忆词汇在学习阶段的呈现颜色)两种任务。采用深编码条件的实验1的结果显示,项目回忆比背景提取条件的合作抑制更强、错误修剪更弱,回忆任务与词汇情绪效价交互影响合作抑制强度;采用浅编码条件的实验2则发现错误修剪在两种任务间的差异不显著。两实验的联合分析显示,回忆任务与编码水平交互影响合作抑制和错误修剪强度。上述结果表明:回忆任务对合作抑制和错误修剪的调节支持双重加工模型;回忆任务与情绪效价对合作抑制强度的交互影响支持权衡说,且与双重加工模型相吻合;回忆任务与编码水平交互影响合作抑制和错误修剪强度。     

Speech segmentation by statistical learning is supported by domain-general processes within working memory

The purpose of this study was to examine the extent to which working memory resources are recruited during statistical learning (SL). Participants were asked to identify novel words in an artificial speech stream where the transitional probabilities between syllables provided the only segmentation cue. Experiments 1 and 2 demonstrated that segmentation performance improved when the speech rate was slowed down, suggesting that SL is supported by some form of active processing or maintenance mechanism that operates more effectively under slower presentation rates. In Experiment 3 we investigated the nature of this mechanism by asking participants to perform a two-back task while listening to the speech stream. Half of the participants performed a two-back rhyme task designed to engage phonological processing, whereas the other half performed a comparable two-back task on un-nameable visual shapes. It was hypothesized that if SL is dependent only upon domain-specific processes (i.e., phonological rehearsal), the rhyme task should impair speech segmentation performance more than the shape task. However, the two loads were equally disruptive to learning, as they both eradicated the benefit provided by the slow rate. These results suggest that SL is supported by working-memory processes that rely on domain-general resources.     

Reproducibility of fMRI-determined language lateralization in individual subjects

This study investigated within-subject test-retest reproducibility (i.e., reliability) of language lateralization obtained with fMRI. Nine healthy subjects performed the same set of three different language tasks during two fMRI sessions on separate days (verb generation, antonym generation, and picture naming). A fourth task analysis was added in which the three tasks were analyzed conjointly (combined task analysis, CTA). The CTA targets brain areas that are commonly used in different language tasks, aiming more selectively at language-critical structures. The number of active voxels (i.e., robustness) and calculated lateralization index (LI) were compared across sessions, tasks, subjects, and two a priori defined volumes of interest (classical language regions versus whole hemisphere) for a wide range of statistical thresholds. Robustness and reliability strongly varied between task analyses. The CTA was a robust detector of language-related brain activity, in contrast to the single task approaches. The CTA and verb generation task allowed for reliable calculation of the LI. Higher thresholds yielded a clear increase in left lateralization, which was largest when calculated from active voxels in classical language regions.     

Judgements of relative direction: The effect of task instructions on spatial recall

One of the most widely used tasks in the spatial memory literature is the judgement of relative direction (JRD) test. The present investigation examined the hypothesis that standard JRD task demands bias spatial recall. In two experiments, participants' recall of small-scale layouts as measured by standard JRD tests (in which the relationship between objects was employed to establish imagined orientations within the learned scene) was compared with recall as measured by novel JRD tasks. The novel tasks emphasized either the internal front/back and left/right axes of individual objects (Experiment 1) or extrinsic spatial cues (Experiment 2). Spatial recall was found to reflect the reference cues emphasized by the JRD task in Experiment 1 and by the novel task in Experiment 2. The finding that directional judgements tended to reflect a frame of reference aligned with the set of cues emphasized by task demands suggests that the nature of the task employed to test knowledge can have an effect on spatial recall.     

Investigating the stability of and relationships among global/local processing measures

Global/local stimuli have been used to estimate global processing biases in individuals and groups, as well as in response to various manipulations. Throughout the literature, multiple different versions of global/local stimuli have been used, such as traditional hierarchical letters and numbers (i.e., Navon letters), abstract hierarchical shapes, and high- and low-spatial-frequency gratings and faces. However, currently it is unclear how reliable or stable performance is on these measures within individuals over time, and whether these seemingly different measures are tapping into the same underlying process. As such, the purpose of the present study was to examine the stability of individual performance on three distinct global/local measures over time and to examine the relationships among the measures. In two studies, we examined the reliability of the biases within, and the relationships among, standard Navon letters in a traditional interference task, hierarchical shapes in a forced choice task, and superimposed high- and low-pass spatial frequency faces in a forced choice task. In both studies, participants completed all three of the tasks, and then returned 7–10 days later to again complete the same tasks. The degree of global/local bias within an individual was found to be highly reliable in the hierarchical shape task and the spatial frequency face task, but less reliable in the traditional Navon letter task. Interestingly, in both studies we found that none of the three measures of global bias were related to each other. Therefore, while these measures do appear to be reliable over time, they may be tapping into distinct aspects of global/local processing.     

Development of Different Forms of Skill Learning Throughout the Lifespan

The acquisition of complex motor, cognitive, and social skills, like playing a musical instrument or mastering sports or a language, is generally associated with implicit skill learning (SL). Although it is a general view that SL is most effective in childhood, and such skills are best acquired if learning starts early, this idea has rarely been tested by systematic empirical studies on the developmental pathways of SL from childhood to old age. In this paper, we challenge the view that childhood and early school years are the prime time for skill learning by tracking age‐related changes in performance in three different paradigms of SL. We collected data from participants between 7 and 87 years for (1) a Serial Reaction Time Task (SRT) testing the learning of motor sequences, (2) an Artificial Grammar Learning (AGL) task testing the extraction of regularities from auditory sequences, and (3) Probabilistic Category Learning in the Weather Prediction task (WP), a non‐sequential categorization task. Results on all three tasks show that adolescence and adulthood are the most efficient periods for skill learning, since instead of becoming less and less effective with age, SL improves from childhood into adulthood and then later declines with aging.     

Confidence in Dyadic Decision Making: The Role of Individual Differences

Groups typically express more confidence than individuals, yet how individual‐level confidence combines during collaborative decision tasks is not well understood. We prescreened 686 community members using a novel confidence measure (a true/false trivia test) intentionally designed to be difficult (accuracy rates were not significantly better than chance) and randomly assigned 72 individuals to collaborate on a matched version of the same test in dyads composed of two low‐confidence individuals, two high‐confidence individuals, or one of each (“mixed”). Consistent with past research, we found that the confidence expressed by dyads was higher than the confidence expressed by individuals; importantly, however, this pattern varied markedly by dyad type, with low‐confidence dyads showing the largest increase, mixed dyads showing a moderate increase, and high‐confidence dyads showing no increase—despite the fact that all dyads showed similarly low accuracy (about 55%). These results highlight the conditions under which groups express greater confidence than individuals and offer insights for the composition of collaborative decision‐making teams. Copyright © 2015 John Wiley & Sons, Ltd.     

Older children's misunderstanding of uncertain belief after passing the false belief test

A four-location belief task was designed to examine children's understanding of another's uncertain belief after passing a false belief (FB) task. In Experiment 1, after passing the FB task, participants were asked what a puppet would do after he failed to find his toy at the falsely believed location. Most 4-year-olds and half of 6-year-olds children who passed the FB test showed difficulty in handling uncertain belief; answering that the puppet would then look for his toy at the current (moved-to) location. Eight-year-old children and adults all recognized that the puppet would look for the toy everywhere, or at random. In Experiment 2, 4- and 6-year-olds were presented two other search tasks; it was shown that preschoolers could use search strategies to solve a similar search problem when FB was not involved. This new aspect of post-FB understanding can be interpreted in terms of limited understanding of uncertainty in a less-knowledgeable individual and of limited ability to infer the consequences of belief-disconfirmation.     

Altering memory through recall: The effects of cue-guided retrieval processing

Three experiments were designed to investigate the influence of initial recall on memory by assessing delayed recall after different immediate cued-recall tests. In all experiments, subjects performed semantic and phonemic encoding tasks on a word list. The subjects then received a cued-recall test that cued the target using the same word as the context word in the encoding task, a test that cued the target with a word from the same level at which the target was encoded, a test that cued the target with a cue from a different level at which the target was encoded, or no immediate-recall test. One day later, the subjects performed a final cued-recall test in which the type of cue (semantic or phonemic) was varied. Consistently, delayed recall was facilitated primarily when the cue on the immediate test was from the same level as the cue on the delayed test. This pattern of facilitation suggests that immediate cued-recall produces an elaboration of an existing memory representation that is closely tied to the type of cue used on the immediate test.