Test validity from a temporal perspective: Incorporating time in validation research

As psychological theories and methodologies are increasingly being permeated with time, the need for reviewing and updating test validation practices is becoming apparent. After a brief comparison of the differential (between-subjects) and temporal (within-subjects) approaches to psychological research, this article examines the role that time has played in traditional validation research, and identifies a number of conceptual and methodological shortcomings that complicate meta-analysis, hamper theory building, and limit practical applications. Starting from the assumption that human traits and behaviours unfold over time, and that stability is a special form of change, it argues that time should be an explicit part of any measurement, either test or criterion, and that validation should include an evaluation of dynamic aspects of the test–criterion relationship. It furthermore argues that when stability is assumed, its temporal boundaries should be marked and supporting evidence be presented, using instruments displaying measurement invariance over time. The article continues to present a framework based on a Variables × Subjects × Time data model, and distinguishes four conditions for validation, namely (1) stable predictor, stable criterion, (2) stable predictor, dynamic criterion, (3) dynamic predictor, stable criterion, and (4) dynamic predictor, dynamic criterion, which allow the definition of five new types of validity. It discusses implications for validation practice, meta-analysis, and theory development as well as for practical applications.     

The liebowitz social anxiety scale as a self-report instrument: a preliminary psychometric analysis

The Liebowitz social anxiety scale (LSAS) is a commonly used clinician-administered instrument. The present study reports on the properties of a self-report version of the LSAS (LSAS-SR). About 175 participants diagnosed with social phobia participated in the study. The LSAS-SR showed overall good psychometric properties as indicated by the results of test-retest reliability, internal consistency, and convergent and discriminant validity. Furthermore, the scale was sensitive to treatment change. The construct validity of the LSAS-SR, however, remains to be further explored. These findings support the utility of the LSAS-SR, which has the advantage of saving valuable clinician time compared to the clinician-administered version.     

A short form of the metacognitions questionnaire: properties of the MCQ-30

The metacognitions questionnaire (MCQ) measures individual differences in a selection of metacognitive beliefs, judgments and monitoring tendencies considered important in the metacognitive model of psychological disorders. The development and properties of a shortened 30-item version of the MCQ, the MCQ-30, are reported. Construct validity was evaluated by confirmatory and exploratory factor analysis. Overall, the fit indices suggested an acceptable fit to a five-factor model consistent with the original MCQ. Exploratory factor analysis supported a five-factor structure, which was almost identical to the original solution obtained in previous studies with the full MCQ. The five factors are cognitive confidence, positive beliefs about worry, cognitive self-consciousness, negative beliefs about uncontrollability of thoughts and danger, and beliefs about need to control thoughts. The MCQ-30 showed good internal consistency and convergent validity, and acceptable to good test-retest reliability. Positive relationships between metacognitions and measures of worry and obsessive-compulsive symptoms provided further support for the validity of the measure and the metacognitive theory of intrusive thoughts. The psychometric properties of MCQ-30 suggest that the instrument is a valuable addition to the assessment of metacognitions that has the advantage of being more economical to use compared with the original MCQ.     

Discriminant validation of measures of organizations commitment,job involvement,and job satisfaction among Swedish army officers

The discriminant validity of measures of job involvement, job satisfaction and organizational commitment was tested with data from 467 army officers in Sweden. Confirmatory-factor analysis showed a close fit between the proposed three-factor model and the data. Further, six of eleven job and health correlates related differently to the three attitude vairables. The results indicate that job involvement, job satisfaction and organizational commitment represent three empirically distinct constructs. Implications for future research discussed.     

重估测谎决策的Bayes推断试验

谎与非谎的情绪反应没有独立的客观标准,以Polygrapy构成技术的测谎缺乏稳定的效度。本研究尝试运用Bayes方法分析一项实测案例,计算两名被试的低显性测谎反应结果,使准确判决有谎的概率由机会水平50％分别调整到可明确分辩的74％和49％,从决策分析角度为改良测谎程序和提高测谎效率提供了一种新的操作思路。     

Psychometric properties of the Chinese version of the End-of-Life Decision-Making and Staff Stress Questionnaire

Background/objectiveThe goal of this study is to establish a Chinese version of the End-of-Life Decision Making and Associated Staff Stress Questionnaire to assess its reliability and validity.MethodA sample of 119 Intensive Care Unit physicians and 485 nurses in China completed the questionnaire, along with questionnaires assessing motional exhaustion subscale, Stress Overload Scale, and other variables associated with end-of-life decision.ResultsSeven factors obtained via exploratory factor analysis could explain 70.61% of the total variance. Confirmatory factor analysis demonstrated an acceptable model fit with Root Mean Square Error of Approximation (RMSEA) being .078 and Standardized Root Mean Square Residual (SRMR) being .066. Validity evidence based on relationships with other variables was provided by positive or negative correlations between the questionnaire subscales and emotional exhaustion, stress overload, and other variables associated with end-of-life decision. The average content validity index was .96. The Cronbach’s α and test–retest reliability was outstanding.ConclusionsThe Chinese version of the End-of-Life Decision Making and Associated Staff Stress Questionnaire is a reliable and valid instrument for measuring the facilitators and hinders to facilitate the end-of-life decision-making, communication and the associated pressure perceived by relevant Intensive Care Unit medical staff among the Chinese population.     

The passion scale: Aspects of reliability and validity of a new 8-item scale assessing passion.

In this article, the psychometric properties of a new scale aimed at quantifying passion are explored, i.e. passion related to becoming good or achieving in some area/theme/skill.The Passion Scale was designed to be quantitative, simple to administer, applicable for large-group testing, and reliable in monitoring passion.A total of 126 participants between 18 and 47 years of age (mean age = 21.65, SD = 3.45) completed an assessment of Passion Scale, enabling us to investigate its feasibility, internal consistency, construct validity and test-retest reliability.FeasibilityThe overall pattern of results suggest that the scale for passion presented here is applicable for the age studied (18–47).Internal consistencyAll individual item scores correlated positively with the total score, with correlations ranging from 0.51 to 0.69. The Cronbach's alpha value for the standardized items was 0.86.Construct validityPearson correlations coefficient between total score passion scale and Grit-S scale were 0.39 for adults, mean age 21.23 (SD = 3.45) (N = 107).Test-retest reliability: Intraclass correlation coefficient (ICCs) between test and retest scores for the total score was 0.92.These promising results warrant further development of the passion scale, including normalization based on a large, representative sample.     

Construct Validity of the Iowa Gambling Task

The Iowa Gambling Task (IGT) was created to assess real-world decision making in a laboratory setting and has been applied to various clinical populations (i.e., substance abuse, schizophrenia, pathological gamblers) outside those with orbitofrontal cortex damage, for whom it was originally developed. The current review provides a critical examination of lesion, functional neuroimaging, developmental, and clinical studies in order to examine the construct validity of the IGT. The preponderance of evidence provides support for the use of the IGT to detect decision making deficits in clinical populations, in the context of a more comprehensive evaluation. The review includes a discussion of three critical issues affecting the validity of the IGT, as it has recently become available as a clinical instrument: the lack of a concise definition as to what aspect of decision making the IGT measures, the lack of data regarding reliability of the IGT, and the influence of personality and state mood on IGT performance.     

Motor assessment of preschool aged children: A preliminary investigation of the validity of the Bruininks–Oseretsky test of motor proficiency – Short form

The aim of this study was to examine the validity of the Bruininks–Oseretsky test of motor proficiency – short form (BOTMP-SF) [Bruininks, R. (1978). Bruininks–Oseretsky test of motor proficiency: Examiners manual. Circle Pines, MN] for the assessment of preschool aged children. Three-hundred and eighteen children 48–71 months old (M = 58.97 months, SD = 6.73) participated in the study. For the data analysis both an ANOVA and a MANOVA were applied with the total battery score and the 14 item scores being the dependent variables, respectively. Age was found to have a significant effect on both children’s total battery score (F(3, 314) = 110.65, p < .001, η² = .68) and 13 item scores (minimum F(3, 314) = 8.75, p < .001, minimum η² = .145). Although the aforementioned results represent an evidence for the validity of the BOTMP-SF, a closer study on the score of each item revealed a high percentage of zero scores on four items. Task difficulty has caused a floor effect, constituting a threat to the validity of the preschoolers’ motor assessment with the above battery. A modification of the battery items is suggested, so that the BOTMP-SF will give valid results for children 4–6 years old.     

Rating Scales for the Assessment of Empathic Communication in Medical Interviews (REM): Scale Development,Reliability, and Validity

Empathy has been shown to affect the quality of care by enhancing the physician–patient relationship and promoting effective communication. In this study, the newly developed Rating Scales for the Assessment of Empathic Communication in Medical Interviews (REM), designed to assess empathy and confrontation in physician–patient interactions, were subjected to preliminary psychometric testing. Particular attention was paid to face validity, reliability, sensitivity, and practicality. A total of 118 volunteers were asked to assess transcribed interactions between physicians and a standardized patient using the REM. In order to assess the convergent validity of the REM, the Motivational Interviewing Treatment Integrity Code (MITI) and the Behavior Change Counseling Index (BECCI) were used. Factor analysis identified two subscales, reflecting empathy and confrontation. Interrater reliability coefficients for items ranged from r = .82 to .97; Cronbach’s alphas for the two subscales were .89 and .88, respectively. The convergent validity was supported by substantial correlations between the REM scores and the MITI scores and by significant correlations between the REM scores and the BECCI score. The REM demonstrated good psychometric properties suggesting the rating scales might be useful in clinical practice, research endeavors, and medical education. Further research is necessary to assess the test-retest reliability as well as the predictive validity of this instrument.