Validity of Interpretation: A user validity perspective beyond the test score

This paper introduces the concept of user validity and provides a new perspective on the validity of interpretations from tests. Test interpretation is based on outputs such as test scores, profiles, reports, spreadsheets of multiple candidates' scores, etc. The user validity perspective focuses on the interpretations a test user makes given the purpose of the test and the information provided in the test output. This innovative perspective focuses on how user validity can be extended to content, criterion, and to some extent construct‐related validity. It provides a basis for researching the validity of interpretations and an improved understanding of the appropriateness of different approaches to score interpretation, as well as how to design test outputs and assessments that are pragmatic and optimal.     

The predictive validity of individual psychological assessments in selecting UK public sector senior managers

Individual Psychological Assessment (IPA) is a very widely offered service in Organizational Psychology. It generally consists of a psychologist or HR practitioner using a combination of interview and psychometrics to arrive at a detailed assessment of an individual's capabilities in relation to a job they are being considered for. Although much used, this practice has limited supporting evidence of its validity—not least because of the methodological difficulties in conducting research on this subject—and has been criticized accordingly. The current study examines the use of IPAs with 115 middle and senior management level candidates in a civil service context. All candidates completed a set of psychometric measures and had an in‐depth interview with a psychologist as part of a standardized process. The ratings made by the assessors were correlated with a criterion measure of potential for promotion derived from multisource feedback ratings obtained on these candidates some months later. Analysis of the results indicated that three of the four attributes rated by assessors correlated significantly with the criterion measure. Further, assessors’ ratings were found to show incremental validity over that provided by psychometric test scores alone. These findings are discussed in terms of the use of IPAs in senior level assessment.     

High-Risk Palliative Care Patients’ Knowledge and Attitudes about Hereditary Cancer Testing and DNA Banking

Even at the end of life, testing cancer patients for inherited susceptibility may provide life-saving information to their relatives. Prior research suggests palliative care inpatients have suboptimal understanding of genetic importance, and testing may be underutilized in this clinical setting. These conclusions are based on limited research. This study aimed to estimate genetic testing prevalence among high-risk palliative care patients in a National Cancer Institute-designated comprehensive cancer center. We also aimed to understand these patients’ understanding of, and attitudes toward, hereditary cancer testing and DNA banking. Palliative care in-patients with cancer completed structured interviews, and their medical records were reviewed. Among patients at high risk for hereditary cancer, we assessed history of genetic testing/DNA banking; and related knowledge and attitudes. Among 24 high-risk patients, 14 (58.3%) said they/their relatives had genetic testing or they had been referred for a genetics consultation. Of the remaining 10 patients, seven (70%) said they would “probably” or “definitely” get tested. Patients who had not had testing were least concerned about the impact of future testing on their family relationships; two (20%) said they were “extremely concerned” about privacy related to genetic testing. Of patients without prior testing, five (50%) said they had heard or read “a fair amount” about genetic testing. No high-risk patients had banked DNA. Overall, 23 (95.8%) said they had heard or read “almost nothing” or “relatively little” about DNA banking. Written materials and clinician discussion were most preferred ways to learn about genetic testing and DNA banking. Overall, this study demonstrates underutilization of genetics services at the end of life continues to be problematic, despite high patient interest.