An Assessment of the Psychometric Adequacy of the British Adaptation of the Career Development Inventory

The concept of 'vocational maturity' is defined, and the development of methods for assessing it is briefly outlined. The adaptation of one such measure - Super's Career Development Inventory - is described, and some results are reported from a study designed to assess the psychometric adequacy of this adaptation. These showed the measure to be reliable enough for use with groups but not with individuals. Scores were found to be more highly related to concomitant variables than to direct measures of vocational maturity, though this may have been due to the poor quality of some of the latter. The convergent validity of the measure was supported, but the discriminant validity was not. The implications of these results for using the measure, and particularly the need for further development work, are briefly considered. It is suggested that such work will also need to address the utility of the construct itself for Britain in the 1980s.     

Structural equation models and personality research

This article is a nontechnical introduction to the use of structural equation models in personality research Although such models can be fruitfully used to address a variety of important theoretical issues, the substantive focus in this article is on the use of such models for elucidating the construct validity of personality measures We include numerous more specific topics under our treatment of construct validity First of all, we show how structural equation models can be applied to the issues of convergent and discriminant validity Do our variables measure the constructs we want them to measure and not other constructs that we would prefer not to measure? Second, we show the utility of structural equation models for predictive validity Do our variables reliably predict other constructs with which they are theoretically linked? Finally, we examine the stability of personality constructs through structural equation models Through-out, our emphasis is on the particular advantages that structural equation models bring to these analytic tasks Ultimately, such models must be used in the service of theory, and when used appropriately, they can help us to refine both our measures and our theories of individual differences     

Demonstrating specific cognitive deficits: a psychometric perspective

Cognitive deficits associated with psychopathology often do not occur in isolation. Consequently, identifying a specific deficit in a disorder requires comparing the magnitude of group differences on theoretically relevant measures with those on control tasks measuring other constructs. L. J. Chapman and J. P. Chapman (1973) noted that common forms of such Group x Task interactions are theoretically ambiguous unless performance measures have comparable discriminating power. The principles of psychometric matching for discriminating power developed in the Chapmans' research program are reviewed, and both criticisms and alternative psychometric approaches are evaluated. Psychometric matching can be mindful of threats to the construct validity of measures and frequently remains methodologically necessary. Otherwise, interactions involving measures that vary in sensitivity to individual differences may be misinterpreted as evidence for specific deficits.     

The International Test Commission Guidelines on the Security of Tests,Examinations, and Other Assessments

The amount and severity of security threats have increased considerably over the past two decades, calling into question the validity of assessments administered around the world. These threats have increased for a number of reasons, including the popular use of computerized and online technologies for test administration and the use of almost undetectable technologies for capturing test content and illegally sharing it instantly across borders and cultures. No assessment program, large or small, is immune to this potential damage. The International Test Commission has recognized the critical need for every organization with an important assessment program to be aware of these and be prepared to counter them. It was for this purpose that these guidelines were developed. Knowing the threats and the guidelines will lead to effective measures to protect the program and its assets, maintaining the value of the tests and assessments to the international community. The guidelines listed in this article provide recommendations on planning for better security, maintaining security during the development of tests and while they are administered, and responding well when a security breach occurs. Following these guidelines will create a significant protective barrier between those who willingly commit test fraud and the valuable assets a program has spent time and money to build.     

Content Validity and Cognitive Tests: Response to Kehoe (), Ployhart (), and Sackett ()

The most important fact emerging from the combination of my article and the three commentaries is the consensus judgment that content validity is appropriate scientifically and professionally for use with tests of specific cognitive skills used in job performance. This is important because the 1978 Uniform Guidelines on Employee Selection Procedures have typically been interpreted as not permitting such usage, and this is particularly the case in the interpretation given to the Guidelines by federal government enforcement agencies. Although the Society for Industrial and Organizational Psychology Principles and the Standards do not prohibit such usage, many industrial–organizational psychologists believe that it is not professionally or scientifically appropriate to employ content validity methods with cognitive measures. The hope is that this series will convince them otherwise. On this point, all four authors in the series are in agreement. The major disagreement among us concerns whether specific cognitive skills used in content valid tests must be considered constructs or not. My position, and apparently that of Kehoe, is that they need not be so considered. I argue that constructs must be invoked only in the context of a substantive theory. Sackett and Ployhart, on the other hand, argue that all measures taken on people must be viewed as constructs, regardless of whether any theoretical propositions and assumptions are involved. In this response, I present reasons why this need not be the case.     

Catastrophizing, confounds, and depression: a comment on Sullivan and D'Eon (1990)

Sullivan and D'Eon (1990) concluded that catastrophizing and depression were conceptually and operationally confounded. I argue that (a) the procedures they used to study confounding were suboptimal because multiple measures of depression and catastrophizing were not employed and (b) the distinctiveness of constructs might better be regarded as a continuous rather than all-or-none (having adequate discriminant validity versus being confounded) concept.     

The Construct Validity of Acceptance: A Multitrait-Multimethod Investigation

Despite increasing clinical and empirical attention, the construct validity of acceptance has not been extensively investigated. The present study utilized a multitrait-multimethod design and a correlated trait-correlated method minus one [CT-C(M-1)] confirmatory factor analytic model to assess acceptance's convergent validity across methods and discriminant validity in comparison to cognitive reappraisal and perceived emotional control in a sample of 210 outpatients with anxiety and mood disorders. In addition, the study evaluated acceptance's concurrent validity by investigating the extent to which it was associated with variables of clinical interest over and above the two rival constructs. Results of confirmatory factor analyses supported acceptance's convergent and discriminant validity in comparison to the two neighboring constructs, and thereby provided partial support for its construct validity. However, contrary to prediction, acceptance was not significantly associated with concurrent validation measures. These results are discussed in the context of acceptance's potential therapeutic utility and functional relationships with associated constructs.     

An Investigation of the Structural Validity of Loevinger's Model and Measure of Ego Development

This study demonstrates the use of structural equation modeling for investigating a hierarchical model of personality based on Loevinger's (1976) conceptualization. This model was constructed to have a multifactorial and hierarchical structure in which the ego development construct served as the source of covariation for specific personality constructs of impulse control, interpersonal style, conscious preoccupations, and cognitive style. This model and three other conceivable models were investigated to determine which structure best represented the relations among specific personality constructs and ego development. The specific personality constructs were identified by measures chosen from the available pool of measures that were compatible with Loevinger's conceptualization. The four conceivable models were fitted to sample data from 267 adult subjects using confirmatory factor analysis and were compared with indices of model fit and methods for comparison of nested models. Results suggested that the four specific personality constructs and ego development are part of the same process. Although a hierarchical structure appeared tenable for this process, the role of ego development perhaps is not as dominant as envisioned by Loevinger. These results must be considered tentative due to measurement limitations.     

Cyberbullying assessment instruments: A systematic review

Although several instruments to assess cyberbullying have been developed, there is nevertheless a lack of knowledge about their psychometric properties. The aim of the present systematic review is to provide a representative overview of the current instruments designed to assess cyberbullying. Further, emphasis will be placed on the structural and psychometric properties of cyberbullying instruments, such as validity and reliability, as well as their conceptual and definitional bases. It will also provide criteria for readers to evaluate and choose instruments according to their own aims. A systematic literature review, limited to publications published prior to October 2010, generated 636 citations. A total of 61 publications fulfilled the delineated selection criteria and were included in the review, resulting in 44 instruments. Following a rater training, relevant information was coded by using a structured coding manual. The raters were the nine authors of this review. Almost half of the instruments included in this review do not use the concept of cyberbullying. The constructs measured by the instruments range from internet harassment behavior to electronic bullying behavior to cyberbullying. Even though many of the authors use other concepts than cyberbullying they claim that their instruments do measure it. For the purpose of this systematic review, we have chosen to categorize them as two different groups, cyberbullying instruments and related instruments. Additionally, most of the included instruments had limited reports of reliability and validity testing. The systematic review reveals a need for investigating the validity and reliability of most of the existing instruments, and resolving the conceptual and definitional fluctuations related to cyberbullying.     

Using Indirect Measurement Tasks to Assess the Self‐concept of Personality: A Systematic Review and Meta‐analyses

This systematic review presents the current state of research investigating the implicit self‐concept of personality. First, we present results on meta‐analyses estimating internal consistency, reliability coefficients, the implicit–explicit consistency and the single association predictive effect of implicit self‐concept of personality measures. To do this, studies were aggregated over personality domains. Second, for each of the Five Factor personality domains, different aspects of construct validity and predictive validity are reviewed in a narrative way. Results show that implicit self‐concept of personality measures are reliable, and there is evidence for the construct and predictive validity of these implicit measures, especially in the extraversion and agreeableness domains of personality. However, it must be kept in mind that clear evidence for publication bias was found for studies examining the single association predictive pattern. Finally, this systematic review identifies some achievable improvements that are needed in future research. Large cross‐lab efforts are important in this respect. Moreover, the implicit self‐concept of personality field must move from an ‘ad hoc’ to a ‘validation’ approach in developing new indirect measurement tasks. By adopting these research objectives, the information processing account of personality will increase its potential to become integrated into mainstream personality theory and research. Copyright © 2017 European Association of Personality Psychology     

The Investment Model Scale: Measuring commitment level,satisfaction level,quality of alternatives,and investment size

Three studies evaluated the reliability and validity of the Investment Model Scale, an instrument designed to measure four constructs, including commitment level and three bases of dependence–satisfaction level, quality of alternatives, and investment size. In all three studies, reliability analyses revealed good internal consistency among items designed to measure each construct. Also, principal components analyses performed on scale items revealed evidence of four factors, with items designed to measure each construct loading on independent factors. Studies 2 and 3 examined associations of model variables with instruments measuring diverse qualities of relationships and assorted personal dispositions. As anticipated, Investment Model variables were moderately associated with other measures reflecting superior couple functioning (e.g., dyadic adjustment, trust level, inclusion of other in the self), and were essentially unrelated to measures assessing personal dispositions (e.g., need for cognition, self-esteem). In addition, Study 3 demonstrated that earlier measures of Investment Model variables predicted later levels of dyadic adjustment and later relationship status (persisted vs. ended). It is hoped that the existence of a reliable and valid Investment Model Scale will promote further research regarding commitment and interdependence in ongoing close relationships.     

Job Embeddedness: A new attitudinal measure

Job embeddedness is a relatively new concept that offers the potential to improve our ability to explain why people stay in their jobs. This article outlines the development and testing of new measures of on‐ and off‐the‐job embeddedness. Analyses of survey data show the measures demonstrate adequate psychometric properties across samples (three military and one nonmilitary organization) and across different organizational levels, genders, and tenure, as well as discriminant validity over other turnover‐relevant constructs and appropriate convergent validity with a number of further constructs identified in the literature.     

Incremental validity of new clinical assessment measures

The authors address conceptual and methodological foundations of incremental validity in the evaluation of newly developed clinical assessment measures. Incremental validity is defined as the degree to which a measure explains or predicts a phenomenon of interest, relative to other measures. Incremental validity can be evaluated on several dimensions, such as sensitivity to change, diagnostic efficacy, content validity, treatment design and outcome, and convergent validity. Indices of incremental validity can vary depending on the criterion measures, comparison measures, and individual differences in samples. The authors review the rationale for, principles, and methods of incremental validation, including the selection of comparison and criterion measures, and address data analytic strategies and the conditional nature of incremental validity evaluations in the selection of measures. Incremental validity contributes to, but is different from, cost-benefits, which reflect the cost of acquiring the data and the benefits from the data. The impact of an incremental validity index on whether a measure is selected will be moderated by the cost of acquiring the new data, the importance of the measured phenomenon, and the clinical utility of the new data.     

Shyness: conceptualization and measurement

The concept of shyness and its measurement were investigated in a series of studies (total N = 1,687). Data collection and analysis proceeded in three phases: the revision and continued development of a measure of shyness, the Social Reticence Scale; a psychometric comparison among five measures of shyness; and an examination of the factor structure underlying the construct of shyness. Phase 1 assessed the reliability and validity of the Social Reticence Scale, including ratings of videotaped monologues and ratings by significant others. Phase 2 compared the five shyness measures with one another on indices of internal consistency and with other relevant measures of emotionality, personality, relationships, and behavior. Items from the five shyness measures were combined in a factor analysis in Phase 3, and the resulting factors were correlated with the self-report and rating data obtained in Phase 2. Overall, the results from these studies confirmed that the shyness measures were valid, reliable, and empirically distinct from measures of related constructs. Behavioral validity was observed for several of the shyness scales. Additional analyses suggested that three interpretable factors underlie responses to the shyness scales but provided little support for drawing conceptual distinctions among types of shyness. Discussion focuses on the implications of these data for the measurement and conceptualization of shyness.     

A Multi-Method Investigation of Two Family Constructs

Two family constructs—prominence and interpersonal distance—are examined. Their origins are traced in the experimental and theoretical literature. The validity of each construct is investigated using data obtained from 15 families with a symptom-bearing child. Validity is supported in both cases through the occurrence of significant correlations among different measures of the same construct. It is suggested that the current distress of the families being studied may have increased the clarity of the prominence hierarchies and dyadic distances that emerged.     

Evaluating the validity of anger regulation problems, interpersonal style, and disturbed mental state for predicting inpatient violence

Research has been conducted to try to identify risk factors to help predict which patients will be violent during psychiatric hospitalization. Despite the relatively large amount of research conducted, it is difficult to draw any firm conclusions, as the studies vary considerably in study design, methods used, and choice of outcome measures. Studies also tend to focus on risk prediction, even though risk management is the primary aim of clinical practice in mental health services and few studies have focused on a theoretical basis for understanding violence. This study assessed the predictive validity of brief assessment scales in a sample of 94 forensic inpatients who had been inpatient for a median of 521 days, to test the hypotheses that anger regulation problems, interpersonal style, and disturbed mental state would be linked to increased violence risk in a forensic hospital during a hospital stay. The outcome variables for this study were physical violence against another and/or clear threats of physical violence. The results of this study provide support for the hypotheses, and this remained the case after controlling for age, gender, length of stay, and presence of major mental disorder. The findings should not only assist clinicians with assessment and management of risk but also support the reconceptualizing of risk prediction research to reflect the task of clinical risk management.     

The ecological validity of laboratory and analogue assessment methods of ADHD symptoms

An important question in the attempt to generalize laboratory findings on attention deficit-hyperactivity disorder (ADHD) children to clinical issues concerning their assessment, diagnosis, outcome, and treatment response is the ecological validity of the commonly used tasks and analogue behavioral observation procedures. This paper examines the concept of ecological validity and issues inherent in its evaluation. The evidence from a variety of sources is then reviewed on the relationship between laboratory methods of assessing inattention, impulsivity, and overactivity and measures of these same constructs in natural settings. Additional findings pertaining to this issue from a recent study of 140 ADHD and normal children and 159 ADHD and normal adolescents using a multimethod battery of tests are also reported. In general, the ecological validity of most methods is of a low to moderate degree, with some traditional laboratory tasks proving unsatisfactory. A few tasks demonstrated acceptable degrees of ecological validity but even these require improvement. It is concluded that future advances in ecological validity are likely to come from: (a) a greater reliance on assessments of the target behaviors in natural settings and (b) combining several of the more promising tasks and analogue methods into a battery that is taken over longer time intervals than has been customary and averaged across repeated administrations.An earlier version of this paper was presented at the International Symposium on Attention Deficit Hyperactivity Disorder, Amsterdam, The Netherlands, April 6 and 7, 1989. The author wishes to thank Charles E. Cunningham, Ph.D., Susan Campbell, Ph.D., Mark Rapport, Ph.D., Eric Mash, Ph.D., and Herbert Quay, Ph.D., for their comments on an earlier version of this paper, and to Marcel Kinsbourne, M.D., for his comments on the presentation itself. The author was supported by funds from NIMH grants 41464, 41583, and 42181 during the preparation of this review.     

ON THE MEANING AND VALIDITY OF TELEVISION VIEWING

The construct validity of “television viewing” is developed and explicated. It is suggested that while this concept is usually used as a strict behavioral variable, many different constructs are implied by it. The construct validity of television viewing can be derived from a conceptualization of what it is in television which, when viewed, or exposed to, interacts with human behavior and attributes. Four such conceptualizations are illustrated, each leading to a different construct of “viewing”: (1) television as a social-situational factor involving choice behavior between activities; (2) television as a transmission of a content repertoire, implying decision-making regarding messages; (3) television as a source of content-messages dealing with recall, comprehension, and attitude change; and (4) television as a “language” pertaining to the processes of encoding messages in the service of extracting knowledge.