The Ecological Validity of Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills

Evaluating the ecological validity of neuropsychological tests has become an increasingly important topic over the past decade. In this paper, we provide a comprehensive review of the research on the ecological validity of neuropsychological tests, as it pertains to everyday cognitive skills. This review is presented in the context of several theoretical issues facing ecological validity research. Overall, the research suggests that many neuropsychological tests have a moderate level of ecological validity when predicting everyday cognitive functioning. The strongest relationships were noted when the outcome measure corresponded to the cognitive domain assessed by the neuropsychological tests. Several other factors that may moderate the degree of ecological validity established for neuropsychological tests are in need of further exploration. These factors include the effects of the population being tested, the approach utilized (verisimilitude vs. veridicality), the person completing the outcome measure (significant other vs. clinician), illness severity, and time from injury until evaluation. In addition, a standard measurement of outcome for each cognitive domain is greatly needed to allow for comparison across studies.     

Computerized Neurocognitive Testing in the Management of Sport-Related Concussion: An Update

Since the late nineties, computerized neurocognitive testing has become a central component of sport-related concussion (SRC) management at all levels of sport. In 2005, a review of the available evidence on the psychometric properties of four computerized neuropsychological test batteries concluded that the tests did not possess the necessary criteria to warrant clinical application. Since the publication of that review, several more computerized neurocognitive tests have entered the market place. The purpose of this review is to summarize the body of published studies on psychometric properties and clinical utility of computerized neurocognitive tests available for use in the assessment of SRC. A review of the literature from 2005 to 2013 was conducted to gather evidence of test-retest reliability and clinical validity of these instruments. Reviewed articles included both prospective and retrospective studies of primarily sport-based adult and pediatric samples. Summaries are provided regarding the available evidence of reliability and validity for the most commonly used computerized neurocognitive tests in sports settings.     

Heterogeneity of Neuropsychological Impairment in HIV Infection: Contributions from Mild Cognitive Impairment

Despite longstanding acknowledgement of the heterogeneity of HIV-associated neurocognitive disorders (HAND), existing HAND diagnostic methods classify according to the degree of impairment, without regard to the pattern of neuropsychological strengths and weaknesses. Research in mild cognitive impairment (MCI) has demonstrated that classifying individuals into subtypes by both their level and pattern of impairment, using either conventional or statistical methods, has etiologic and prognostic utility. Methods for characterizing the heterogeneity of MCI provide a framework that can be applied to other disorders and may be useful in clarifying some of the current challenges in the study of HAND. A small number of studies have applied these methods to examine the heterogeneity of neurocognitive function among individuals with HIV. Most have supported the existence of multiple subtypes of neurocognitive impairment, with some evidence for distinct clinicodemographic features of these subtypes, but a number of gaps exist. Following a review of diagnostic methods and challenges in the study of HAND, we summarize the literature regarding conventional and empirical subtypes of MCI and HAND and identify directions for future research regarding neurocognitive heterogeneity in HIV infection.     

The role of neuropsychology in UK pediatric HIV care: Relevance to clinical practice and research

There has been a dramatic improvement in the survival of children with perinatally-acquired HIV (PHIV) following the introduction of effective treatment in 1990s. The care for children living with PHIV is now focused on more accurately understanding the effects of both HIV and HIV treatment on the developing body and brain. An evaluation of current HIV neuroimaging, and neurocognitive research, when combined with clinical experience in the area of HIV, could help to inform United Kingdom (UK) PHIV service provision. This paper argues that an understanding from a neuropsychological perspective will help these young people to optimize their health, quality of life, and future functioning. The aim of the paper is to bring together research and clinical understanding of HIV and its treatment effects on the developing brain, together with an understanding of other potential neurological risk factors. It is argued here that there is a need for targeted neuropsychology assessment and preventative interventions, supported by clinical and preliminary research on the neurocognitive effects of HIV and its treatments.     

Cognitive Neurorehabilitation of HIV-associated Neurocognitive Disorders: A Qualitative Review and Call to Action

Despite significant advances in the virologic management of HIV infection over the last two decades, effective treatments for HIV-associated neurocognitive disorders (HAND) remain elusive. While pharmacological interventions have yielded some success in improving neurocognitive outcomes in HIV, there is a dearth of rigorous studies examining the efficacy of cognitive rehabilitation for remediating HIV-associated neurocognitive impairment. This qualitative review summarizes and critiques the emerging literature on cognitive and behavioral treatments for HAND, which provides many reasons for optimism, but also has major limitations that underscore the scope of the work that lies ahead. Considering the notable real-world consequences of HAND, the development, validation, and clinical deployment of cognitive neurorehabilitation interventions tailored to the needs of persons living with HIV infection is a priority for clinical neuroAIDS investigators. In describing potential future directions for this endeavor, particular attention was paid to the application of cognitive neuropsychological principles in developing theory-driven approaches to managing HAND, improving everyday functioning, and enhancing HIV health outcomes.     

Neurobiology of Cocaine-Induced Organic Brain Impairment: Contributions from Functional Neuroimaging

The present review is directed at imparting the current knowledge regarding functional neuroimaging as a tool for enhancing the understanding of cerebrophysiologic and neurobehavioral consequences of stimulant abuse. Stimulants like cocaine are capable of inducing clinically significant neurocognitive impairment through direct action on the brain, and indirectly through other organs that influence cerebral physiology. Neurochemical dysregulation including profound effects on the serotonergic and dopaminergic systems have substantial physiological and neurobehavioral consequences. Brain hemorrhages, transient ischemic attacks, strokes, and seizures frequently follow cocaine use. The residual cerebropathologic consequences of cocaine are seen only in significant or pronounced brain events when structural neuroimaging techniques such as computed tomography (CT) and magnetic resonance imaging (MRI) are employed. However, recent research with newer functional neuroimaging techniques such as single photon emission, positron emission tomography, and quantitative electroencephalography have revealed high rates of significant alteration in brain function among cocaine users, with negative structural imaging studies. These findings are often associated with impairment on neuropsychological evaluation, also in the absence of positive findings on CT and MRI. Both cerebral metabolic and hypoperfusion anomalies are seen, especially in anterior and temporal brain regions. Observed changes can persist for months, and for some patients, may represent a permanent change in brain functioning.     

The Relationship Between Parental Report on the BRIEF and Performance-Based Measures of Executive Function in Children with Moderate to Severe Traumatic Brain Injury

The Behavior Rating Inventory of Executive Function (BRIEF) is a questionnaire that assesses parental observations of behaviors associated with executive function in children in the home environment. The current investigation examines the relationship between the BRIEF and individually-administered neuropsychological tests in children with traumatic brain injury. Forty-eight children with moderate to severe traumatic brain injury were administered the WISC-III and several performance-based tests of executive function (the Wisconsin Card Sorting Test, Trail Making Test Part B, verbal fluency), and a parent completed the BRIEF. Results indicate that the Metacognition Index from the BRIEF correlates with Verbal IQ, but none of the index scores from the BRIEF correlate with any of the performance-based tests of executive function. Results are discussed with respect to the ecological validity of standardized clinical neuropsychological tests of executive function.     

The relationship between parental report on the BRIEF and performance-based measures of executive function in children with moderate to severe traumatic brain injury.

The Behavior Rating Inventory of Executive Function (BRIEF) is a questionnaire that assesses parental observations of behaviors associated with executive function in children in the home environment. The current investigation examines the relationship between the BRIEF and individually-administered neuropsychological tests in children with traumatic brain injury. Forty-eight children with moderate to severe traumatic brain injury were administered the WISC-III and several performance-based tests of executive function (the Wisconsin Card Sorting Test, Trail Making Test Part B, verbal fluency), and a parent completed the BRIEF. Results indicate that the Metacognition Index from the BRIEF correlates with Verbal IQ, but none of the index scores from the BRIEF correlate with any of the performance-based tests of executive function. Results are discussed with respect to the ecological validity of standardized clinical neuropsychological tests of executive function.     

Historical Perspectives on Ancient Greek Derived “a” Prefixed Nomenclature for Acquired Neurocognitive Impairment

Distinct forms of acquired neurocognitive impairment are often described by “a” prefixed terms that derive from ancient Greek (and in one case Latin). Two modern English language neurological and neuropsychological reference books were searched to identify 17 such terms in contemporary usage: amnesia, akinesia, ataxia, aphasia, agraphia, anosmia, apraxia, athetosis, ageusia, achromatopsia, agnosia, alexia, amusia, anomia, anarthria, anosognosia, and acalculia. These were traced to their initial association with acquired neurocognitive impairment in German, English, and French language medical publications from the late 18th, 19th, and early 20th centuries (1770 through 1920). Some of these terms (e.g., agnosia) were used in ancient Greek, although not associated with neurocognitive impairment. The remainder constitute novel semantically plausible (e.g., anosmia) and unclear (e.g., alexia) formulations. In the localizationist thinking of the time, neurocognition was conceived as being organized within specialized “centers” in specific locations connected by pathways within the brain.     

Introduction to the special section on theory and data in categorization: Integrating computational, behavioral, and cognitive neuroscience approaches

This special section brings together behavioral, computational, mathematical, and neuroimaging approaches to understand the processes underlying category learning. Over the past decade, there has been growing convergence in research on categorization, with computational-mathematical models influencing the interpretation of brain imaging and neuropsychological data, and with cognitive neuroscience findings influencing the development and refinement of models. Classic debates between single-system and multiple-memory-system theories have become more nuanced and focused. Multiple brain areas and cognitive processes contribute to categorization, but theories differ markedly in whether and when those neurocognitive components are recruited for different aspects of categorization. The articles in this special section approach this issue from several diverse angles.     

Cognitive mechanisms in numerical processing: evidence from acquired dyscalculia.

This article discusses cognitive neuropsychological research on acquired dyscalculia (i.e., impaired numerical processing resulting from brain damage), surveying issues of current interest, and illustrating the ways in which analyses of acquired deficits can contribute to an understanding of normal processing. I first review the logic whereby inferences concerning normal cognition are drawn from patterns of impaired performance. I then consider research exploring the general functional architecture of the cognitive numerical processing mechanisms, and finally turn to studies aimed at probing the internal structure and functioning of individual processing components.     

Neuropsychological Assessment of HIV-Infected Populations in International Settings

Resource-limited regions of the world represent the areas most affected by the global HIV epidemic. Currently, there are insufficient data on the neurocognitive effects of HIV in these areas and neuropsychological studies that have been carried out thus far are marked by inconsistent methods, test batteries, and rating systems for levels of cognitive impairment. These differences in methods, along with genetic variability of both virus and host, differences in co-infections and other co-morbidities, differences in language and culture, and infrastructural deficiencies in many international settings create challenges to the assessment of neurocognitive functioning and interpretation of neuropsychological data. Identifying neurocognitive impairment directly attributable to HIV, exploring relationships between HIV-associated neurocognitive impairment, disease variables, and everyday functioning, evaluating differences in HIV-1 subtype associated neuropathology, and determining implications for treatment remain complicated and challenging goals. Endeavors to establish a more standardized approach to neurocognitive assessments across international studies in addition to accumulating appropriate normative data that will allow more accurate rating of neuropsychological test performance will be crucial to future efforts attempting to achieve these goals.     

Neuropsychological Sequelae of PTSD and TBI Following War Deployment among OEF/OIF Veterans

Posttraumatic stress disorder (PTSD) and mild traumatic brain injury (mTBI) are highly prevalent among Veterans of the conflicts in Iraq and Afghanistan. These conditions are associated with common and unique neuropsychological and neuroanatomical changes. This review synthesizes neuropsychological and neuroimaging studies for both of these disorders and studies examining their co-occurrence. Recommendations for future research, including use of combined neuropsychological and advanced neuroimaging techniques to study these disorders alone and in concert, are presented. It is clear from the dearth of literature that addiitonal studies are required to examine and understand the impact of specific factors on neurocognitive outcome. Of particular relevance are temporal relationships between PTSD and mTBI, risk and resilience factors associated with both disorders and their co-occurrence, and mTBI-specific factors such as time since injury and severity of injury, utilizing comprehensive, yet targeted cognitive tasks.     

Route-finding: A measure of everyday executive functioning in the head-injured adult

This is an ecological test of adaptation in head-injured adults, reporting reliability and validity of the Executive Function Route-finding Task (EFRT; Boyd and Sautter, 1985). A Likert scale was used to rate executive aspects of route-finding such as task formulation, strategy of approach, detection and correction of errors, and dependence on cueing among 31 head-injured young adults within a large rehabilitation facility. The Task has high inter-rater reliability and acceptable concurrent validity with other neuropsychological constructs.     

Structural and Functional Meta-analytic Evidence for Fronto-subcortical System Deficit in Progressive Supranuclear Palsy

Meta-analytic methods were used to determine the most sensitive indexes to fronto-subcortical deficit in progressive supranuclear palsy (PSP) and to further characterize the neurocognitive and related features of PSP that can provide a basis of comparison to other disorders with prominent subcortical brain lesions. Studies dating back to 1984 were gathered and calibrated to compare the neuropsychological, neuroimaging, and neurophysiological test results from 229 patients with PSP, and 357 healthy controls. The tests most sensitive to fronto-subcortical deficit in PSP were mostly neuropsychological measures that include such tests as the Stroop Task, Trail Making Test Part A, and Purdue pegboard performance. We conclude that although neuropsychological measures may be most sensitive to deficits in PSP, they are also less specific and valid indicators of fronto-subcortical brain system integrity.     

The clinical utility of the Rey Auditory-Verbal Learning Test in medical rehabilitation

Recent research has reported the Rey Auditory-Verbal Learning Test Trial V (RAVLT-V) score to be a sensitive indicator of global cognitive functioning, not merely verbal learning skills. The concurrent validity of the RAVLT-V relative to other neuropsychological measures frequently utilized in medical rehabilitation was investigated in a sample of 64 outpatients diagnosed with traumatic brain injury (TBI). The sample was predominantly male (77%), white (97%), youthful (31.4±11.5 years), high school educated, and an average of 53 months postinjury. Using stepwise multiple regression analysis, a three-variable model consisting of gender, Wechsler Memory Scale-Revised Delayed Memory Index, and Trailmaking Part B (time) accounted for 61% of the variance in RAVLT-V scores. These findings suggest that in TBI patients, the RAVLT taps not only specific verbal learning and memory, but also global cognitive functions. Sensitivity to diffuse neuropsychological changes and ease/speed of administration make the RAVLT a valuable tool in medical rehabilitation settings.     

Beery-Buktenica Developmental Test of Visual-Motor Integration performance in children with traumatic brain injury and attention-deficit/hyperactivity disorder

Evaluation of visuoconstructional abilities is a common part of clinical neuropsychological assessment, and the Beery-Buktenica Developmental Test of Visual-Motor Integration (VMI; K. E. Beery & N. A. Beery, 2004) is often used for this purpose. However, few studies have examined its psychometric properties when used to assess children and adolescents with traumatic brain injury (TBI) or attention-deficit/hyperactivity disorder (ADHD), even though these are among the most common acquired and neurodevelopmental forms of brain dysfunction in children. This study examined the validity of VMI scores in 123 children with TBI and 65 with ADHD. The TBI and ADHD groups performed significantly worse than the standardization sample, obtaining VMI mean scores of 87.2 (SD = 13.7) and 93.5 (SD = 11.27). Previous research has noted decrements in visuoconstructional abilities in TBI but relative sparing in ADHD. To examine the criterion validity of VMI scores, the authors therefore compared these 2 groups. As anticipated, the TBI group performed significantly worse than the ADHD group, but receiver operator characteristic analysis indicated that VMI scores were poor at discriminating between groups. Nonetheless, convergent validity evidence supported interpretation of VMI scores as measuring perceptual organization in both groups. In particular, principal components analysis indicated that VMI total scores loaded with perceptual organization tests from the Wechsler Intelligence Scale for Children, 3rd ed. (WISC-III; D. Wechsler, 1997), and its highest correlation was with the WISC-III Perceptual Organization Index. Also, the VMI correlated significantly with the Grooved Pegboard test for the group with TBI. These findings suggest that VMI scores are sensitive to visuoconstructional and motor deficits in children with developmental and acquired brain dysfunction.     

Initial validation of a virtual environment for assessment of memory functioning: virtual reality cognitive performance assessment test.

The current project is an initial attempt at validating the Virtual Reality Cognitive Performance Assessment Test (VRCPAT), a virtual environment-based measure of learning and memory. To examine convergent and discriminant validity, a multitrait-multimethod matrix was used in which we hypothesized that the VRCPAT's total learning and memory scores would correlate with other neuropsychological measures involving learning and memory but not with measures involving potential confounds (i.e., executive functions; attention; processing speed; and verbal fluency). Using a sequential hierarchical strategy, each stage of test development did not proceed until specified criteria were met. The 15-minute VRCPAT battery and a 1.5-hour in-person neuropsychological assessment were conducted with a sample of 30 healthy adults, between the ages of 21 and 36, that included equivalent distributions of men and women from ethnically diverse populations. Results supported both convergent and discriminant validity. That is, findings suggest that the VRCPAT measures a capacity that is (a) consistent with that assessed by traditional paper-and-pencil measures involving learning and memory and (b) inconsistent with that assessed by traditional paper-and-pencil measures assessing neurocognitive domains traditionally assumed to be other than learning and memory. We conclude that the VRCPAT is a valid test that provides a unique opportunity to reliably and efficiently study memory function within an ecologically valid environment.     

Recent development of cognitive and neuropsychological assessment in Asian countries

There is a growing trend of research in neuropsychological assessment in Asia: The mean number of published studies increased from 1.8 in the 1980s to 16.0 in 2 recent years. About 40% of the studies reviewed concerned test development or adaptation, with the latter greatly outnumbering the former. To facilitate the test selection for clinical use, 36 clinical measures were evaluated based on some statistical criteria. The results showed that only 8 neuropsychological tests (5 Chinese, 2 Korean, and 1 Japanese) met the criteria. Given that Asian and Western populations differ on certain cognitive processes, future development of neuropsychological assessment measures in Asia should move from the adaptation of Western tests toward the construction of original tests with better ecological validity.