fMRI reliability: Influences of task and experimental design

As scientists, it is imperative that we understand not only the power of our research tools to yield results, but also their ability to obtain similar results over time. This study is an investigation into how common decisions made during the design and analysis of a functional magnetic resonance imaging (fMRI) study can influence the reliability of the statistical results. To that end, we gathered back-to-back test–retest fMRI data during an experiment involving multiple cognitive tasks (episodic recognition and two-back working memory) and multiple fMRI experimental designs (block, event-related genetic sequence, and event-related m-sequence). Using these data, we were able to investigate the relative influences of task, design, statistical contrast (task vs. rest, target vs. nontarget), and statistical thresholding (unthresholded, thresholded) on fMRI reliability, as measured by the intraclass correlation (ICC) coefficient. We also utilized data from a second study to investigate test–retest reliability after an extended, six-month interval. We found that all of the factors above were statistically significant, but that they had varying levels of influence on the observed ICC values. We also found that these factors could interact, increasing or decreasing the relative reliability of certain Task × Design combinations. The results suggest that fMRI reliability is a complex construct whose value may be increased or decreased by specific combinations of factors.     

ROCs obtained with two signal intensities presented in random order,and a comparison between yes-no and rating ROCs

Two experiments are presented. The first experiment investigated the effect of mixing signals of two different intensities within the same blocks of trials upon receiver operating characteristics (ROCs) obtained with the rating procedure. The ROCs obtained when the signals of differing intensity were mixed in blocks of trials were found to be quite similar to those obtained when only one signal intensity was presented in any given block of trials. In the second experiment ROCs were determined by having subjects adopt different decision criteria on different blocks of trials in a yes-no experiment. The data points obtained in this experiment fell quite close to the ROCs obtained from the same subjects with the rating procedure in the first experiment.     

Intersection crossing assist system: Transition from a road-side to an in-vehicle system

Due to high vehicle velocities rural intersections have a disproportionately high rate of fatalities. The current study examines the transition from an infrastructure-based rural intersection crossing assist system to one located inside a vehicle. Moreover, we investigate the efficacy of the in-vehicle system. Three different designs of the assist system were examined regarding their impact on driving performance and applicability to varying age groups and visibility conditions. These designs differed in terms of their complexity based on the amount of information that drivers received about the intersection traffic. Seventy-two older and younger participants divided into the three design groups crossed a busy rural intersection in a simulated environment. Drivers completed four blocks of trials in which the presence of the assist system and visibility conditions (limited, clear) were counterbalanced. When presented with the assist system drivers were less likely to accept a smaller gap, especially under low-visibility conditions. The design of the assist system that resulted in the best overall intersection crossing performance was also the most informative about the traffic. Older drivers exhibited some benefits from the presence of the assist system, although not to the same extent as the younger drivers. The results suggest that some infrastructure-based assist or information display systems could successfully be transitioned to inside a vehicle.     

TOWARD A METHODOLOGY OF WITHDRAWAL DESIGNS FOR THE ASSESSMENT OF RESPONSE MAINTENANCE

Single-case experimental designs have advanced considerably in the evaluation of functional relationships between interventions and behavior change. The systematic investigation of response maintenance once intervention effects have been demonstrated has, however, received relatively little attention. The lack of research on maintenance may stem in part from the paucity of design options that systematically evaluate factors that contribute to maintenance. The present paper discusses three design options potentially useful for the investigation of response maintenance. These include: (a) the sequential-withdrawal, (b) the partial-withdrawal, and (c) the partial-sequential withdrawal designs. Each design is illustrated and potential limitations are discussed.     

Training Public School Special Educators to Implement Two Functional Analysis Models

The purpose of this study was to investigate the efficacy and efficiency of a training package to teach public school special educators to conduct functional analyses of challenging behavior. Six public school educators were divided into two cohorts of three and were taught two models of functional analysis of challenging behavior: traditional and trial-based functional analysis. The effect of the training package on functional analysis implementation was evaluated using multiple-baseline designs across participants for each functional analysis model. The sequence of functional analysis models taught was counterbalanced across cohorts. Following the training package, all participants reached 100 % implementation fidelity during role-plays and classroom sessions with students, and maintained high fidelity at follow-up role-play sessions. Data on training duration, trials to criterion, and social validity revealed that trial-based functional analysis training had a shorter duration and was rated as the more favorable functional analysis model.     

Maintenance versus manipulation of information held in working memory: an event-related fMRI study.

One model of the functional organization of lateral prefrontal cortex (PFC) in primates posits that this region is organized in a dorsal/ventral fashion subserving spatial and object working memory, respectively. Alternatively, it has been proposed that a dorsal/ventral subdivision of lateral PFC instead reflects the type of processing performed upon information held in working memory. We tested this hypothesis using an event-related fMRI method that can discriminate among functional changes occurring during temporally separated behavioral subcomponents of a single trial. Subjects performed a delayed-response task with two types of trials in which they were required to: (1) retain a sequence of letters across the delay period (maintenance) or (2) reorder the sequence into alphabetical order across the delay period (manipulation). In each subject, activity during the delay period was found in both dorsolateral and ventrolateral PFC in both types of trials. However, dorsolateral PFC activity was greater in manipulation trials. These findings are consistent with the processing model of the functional organization of working memory in PFC.     

Minimax d-optimal designs for item response theory models

Various different item response theory (IRT) models can be used in educational and psychological measurement to analyze test data. One of the major drawbacks of these models is that efficient parameter estimation can only be achieved with very large data sets. Therefore, it is often worthwhile to search for designs of the test data that in some way will optimize the parameter estimates. The results from the statistical theory on optimal design can be applied for efficient estimation of the parameters.A major problem in finding an optimal design for IRT models is that the designs are only optimal for a given set of parameters, that is, they are locally optimal. Locally optimal designs can be constructed with a sequential design procedure. In this paper minimax designs are proposed for IRT models to overcome the problem of local optimality. Minimax designs are compared to sequentially constructed designs for the two parameter logistic model and the results show that minimax design can be nearly as efficient as sequentially constructed designs.     

The posterior parietal paradox: Why do functional magnetic resonance imaging and lesion studies on episodic memory produce conflicting results?

Recent functional magnetic resonance imaging (fMRI) studies addressing healthy subjects point towards posterior parietal cortex (PPC) involvement in episodic memory tasks. This is noteworthy, since neuropsychological studies usually do not connect parietal lesions to episodic memory impairments. Therefore an inventory of the possible factors behind this apparent paradox is warranted. This review compared fMRI studies which demonstrated PPC activity in episodic memory tasks, with findings with studies of patients with PPC lesions. A systematic evaluation of possible explanations for the posterior parietal paradox indicates that PPC activation in fMRI studies does not appear to be attributable to confounding cognitive/psychomotor processes, such as button pressing or stimulus processing. What may be of more importance is the extent to which an episodic memory task loads on three closely related cognitive processes: effort and attention, self‐related activity, and scene and image construction. We discuss to what extent these cognitive processes can account for the paradox between lesion and fMRI results. They are strongly intertwined with the episodic memory and may critically determine in how far the PPC plays a role in a given memory task. Future patient studies might profit from specifically taking these cognitive factors into consideration in the task design.     

Brain activity during the encoding, retention, and retrieval of stimulus representations

Studies of delayed nonmatching-to-sample (DNMS) performance following lesions of the monkey cortex have revealed a critical circuit of brain regions involved in forming memories and retaining and retrieving stimulus representations. Using event-related functional magnetic resonance imaging (fMRI), we measured brain activity in 10 healthy human participants during performance of a trial-unique visual DNMS task using novel barcode stimuli. The event-related design enabled the identification of activity during the different phases of the task (encoding, retention, and retrieval). Several brain regions identified by monkey studies as being important for successful DNMS performance showed selective activity during the different phases, including the mediodorsal thalamic nucleus (encoding), ventrolateral prefrontal cortex (retention), and perirhinal cortex (retrieval). Regions showing sustained activity within trials included the ventromedial and dorsal prefrontal cortices and occipital cortex. The present study shows the utility of investigating performance on tasks derived from animal models to assist in the identification of brain regions involved in human recognition memory.     

A Comparison of Multilevel Mediation Modeling Methods: Recommendations for Applied Researchers

Multilevel structural equation modeling (MSEM) has been proposed as a valuable tool for estimating mediation in multilevel data and has known advantages over traditional multilevel modeling, including conflated and unconflated techniques (CMM & UMM). Recent methodological research has focused on comparing the three methods for 2-1-1 designs, but in regards to 1-1-1 mediation designs, there are significant gaps in the published literature that prevent applied researchers from making educated decisions regarding which model to employ in their own specific research design. A Monte Carlo study was performed to compare MSEM, UMM, and CMM on relative bias, confidence interval coverage, Type I Error, and power in a 1-1-1 model with random slopes under varying data conditions. Recommendations for applied researchers are discussed and an empirical example provides context for the three methods.     

Brain network interactions in auditory, visual and linguistic processing

In the paper, we discuss the importance of network interactions between brain regions in mediating performance of sensorimotor and cognitive tasks, including those associated with language processing. Functional neuroimaging, especially PET and fMRI, provide data that are obtained essentially simultaneously from much of the brain, and thus are ideal for enabling one to assess interregional functional interactions. Two ways to use these types of data to assess network interactions are presented. First, using PET, we demonstrate that anterior and posterior perisylvian language areas have stronger functional connectivity during spontaneous narrative production than during other less linguistically demanding production tasks. Second, we show how one can use large-scale neural network modeling to relate neural activity to the hemodynamically-based data generated by fMRI and PET. We review two versions of a model of object processing - one for visual and one for auditory objects. The regions comprising the models include primary and secondary sensory cortex, association cortex in the temporal lobe, and prefrontal cortex. Each model incorporates specific assumptions about how neurons in each of these areas function, and how neurons in the different areas are interconnected with each other. Each model is able to perform a delayed match-to-sample task for simple objects (simple shapes for the visual model; tonal contours for the auditory model). We find that the simulated electrical activities in each region are similar to those observed in nonhuman primates performing analogous tasks, and the absolute values of the simulated integrated synaptic activity in each brain region match human fMRI/PET data. Thus, this type of modeling provides a way to understand the neural bases for the sensorimotor and cognitive tasks of interest.     

Analysis of behaviors involved in performance of the block design subtest of the WISC-R

This study analyzed two of the component behaviors which might contribute to developmental differences on the Block Design Subtest of the WISC-R. The performance of approximately 30 boys and girls in each of the first, third, fifth and eleventh grades was videotaped as they completed the Block Design Subtest. The behaviors coded from the videotape included the order of placement of blocks within each design, and the degree to which children persisted in placing a given block correctly before moving to the next one. Results showed no developmental differences in either of these dependent variables. There were striking similarities in the solution strategies applied by older and younger children to most of the block designs.     

Prefrontal brain activity predicts temporally extended decision-making behavior

Although functional neuroimaging studies of human decision-making processes are increasingly common, most of the research in this area has relied on passive tasks that generate little individual variability. Relatively little attention has been paid to the ability of brain activity to predict overt behavior. Using functional magnetic resonance imaging (fMRI), we investigated the neural mechanisms underlying behavior during a dynamic decision task that required subjects to select smaller, short-term monetary payoffs in order to receive larger, long-term gains. The number of trials over which the longterm gains accrued was manipulated experimentally (2 versus 12). Event-related neural activity in right lateral prefrontal cortex, a region associated with high-level cognitive processing, selectively predicted choice behavior in both conditions, whereas insular cortex responded to fluctuations in amount of reward but did not predict choice behavior. These results demonstrate the utility of a functional neuroimaging approach in behavioral psychology, showing that (a) highly circumscribed brain regions are capable of predicting complex choice behavior, and (b) fMRI has the ability to dissociate the contributions of different neural mechanisms to particular behavioral tasks.     

Visual search for dimensionally redundant pop-out targets: evidence for parallel-coactive processing of dimensions

In two visual search experiments, the detection of singleton feature targets redundantly defined on multiple dimensions was investigated. Targets differed from the distractors in orientation, color, or both (redundant targets). In Experiment 1, the various target types were presented either in separate blocks or in random order within blocks. Reaction times to redundant targets significantly violated the race model inequality (Miller, 1982), but only when there was constancy of the target-defining dimension(s) within trial blocks. In Experiment 2, there was dimensional variability within blocks. Consistent with Experiment 1, constancy of the target-defining dimension(s), but this time across successive trials (rather than within blocks), was critical for observing violations of the race model inequality. These results provide evidence for parallel-coactive processing of multiple dimensions, consistent with the dimension-weighting account of Müller, Heller, and Ziegler (1995).     

Attentional capture with various distractor and target types

Effects of nonpredictive distractors that involved changes in luminance, size, or shape were examined in three experiments. In Experiment 1, with two types of distractors (onsets and offsets), accuracy was better on trials when the distractor was near the location of either an offset or an onset target than on trials when the distractor was in a different location from that of the target, demonstrating attentional capture. Capture occurred both when the type of target (onset or offset) was blocked and therefore predictable and also when the type of target was mixed within blocks and therefore not predictable. Further experiments indicated that distractors captured attention even when the change to distractor did not create a new perceptual object. Neither a singleton-detection mode, nor a contingent involuntary orienting hypothesis, nor creation of a new object seems to explain all of these data adequately. Rather, capture may depend on a number of factors in the task.     

The integration of social influence and reward: Computational approaches and neural evidence

Decades of research have established that decision-making is dramatically impacted by both the rewards an individual receives and the behavior of others. How do these distinct influences exert their influence on an individual’s actions, and can the resulting behavior be effectively captured in a computational model? To address this question, we employed a novel spatial foraging game in which groups of three participants sought to find the most rewarding location in an unfamiliar two-dimensional space. As the game transitioned from one block to the next, the availability of information regarding other group members was varied systematically, revealing the relative impacts of feedback from the environment and information from other group members on individual decision-making. Both reward-based and socially-based sources of information exerted a significant influence on behavior, and a computational model incorporating these effects was able to recapitulate several key trends in the behavioral data. In addition, our findings suggest how these sources were processed and combined during decision-making. Analysis of reaction time, location of gaze, and functional magnetic resonance imaging (fMRI) data indicated that these distinct sources of information were integrated simultaneously for each decision, rather than exerting their influence in a separate, all-or-none fashion across separate subsets of trials. These findings add to our understanding of how the separate influences of reward from the environment and information derived from other social agents are combined to produce decisions.     

Attentional capture with various distractor and target types

Effects of nonpredictive distractors that involved changes in luminance, size, or shape were examined in three experiments. In Experiment 1, with two types of distractors (onsets and offsets), accuracy was better on trials when the distractor was near the location of either an offset or an onset target than on trials when the distractor was in a different location from that of the target, demonstrating attentional capture. Capture occurred both when the type of target (onset or offset) was blocked and therefore predictable and also when the type of target was mixed within blocks and therefore not predictable. Further experiments indicated that distractors captured attention even when the change to distractor did not create a new perceptual object. Neither a singleton-detection mode, nor a contingent involuntary orienting hypothesis, nor creation of a new object seems to explain all of these data adequately. Rather, capture may depend on a number of factors in the task.     

Performance Criteria for Stabilometer Learning

A method for collecting multiple dependent variables (DVs) on stabilometer performance is described. The relative sensitivity of 6 measures of stabilometer performance was determined within a 2 × 2 × 2 × 6 (blocks of 5 trials) factorial design. Two levels each of Plane of Balance, KR, and Starting Position were independent variables. Ss (N=80) performed 30 trials with a 30-sec. intertrial rest. Univariate ANOVA revealed significant changes across blocks on four of the six DVs measured. Multivariate ANOVA of the three treatment effects revealed KR significant, p<.01. Discriminant function analysis indicated a measure of S’s variability about a chosen balance point as the best discriminator for the KR factor. Recording continuous analog information of S’s response revealed the stabilometer sensitive to S’s learning. Results were discussed in terms of the role of KR, task criterion manipulations, and the use of the stabilometer as a motor task within theoretical frameworks.     

Structural equation modeling of multitrait-multimethod data: different models for different types of methods

The question as to which structural equation model should be selected when multitrait-multimethod (MTMM) data are analyzed is of interest to many researchers. In the past, attempts to find a well-fitting model have often been data-driven and highly arbitrary. In the present article, the authors argue that the measurement design (type of methods used) should guide the choice of the statistical model to analyze the data. In this respect, the authors distinguish between (a) interchangeable methods, (b) structurally different methods, and (c) the combination of both kinds of methods. The authors present an appropriate model for each type of method. All models allow separating measurement error from trait influences and trait-specific method effects. With respect to interchangeable methods, a multilevel confirmatory factor model is presented. For structurally different methods, the correlated trait-correlated (method-1) model is recommended. Finally, the authors demonstrate how to appropriately analyze data from MTMM designs that simultaneously use interchangeable and structurally different methods. All models are applied to empirical data to illustrate their proper use. Some implications and guidelines for modeling MTMM data are discussed.