Construct validity and case validity in assessment

Clinical assessment relies on both construct validity, which focuses on the accuracy of conclusions about a psychological phenomenon drawn from responses to a measure, and case validity, which focuses on the synthesis of the full range of psychological phenomena pertaining to the concern or question at hand. Whereas construct validity is grounded in understanding causal influences of a distinct phenomenon on responses to various measures and life contexts, case validity encompasses the joint influences of multiple phenomena on individuals' responses. Two sets of distinctions essential to understanding psychological phenomena, hence to understanding construct validity, are (a) implicit and explicit versions of personality constructs and (b) ability and personality as versions of constructs measured by performance tests presenting maximal and typical conditions, respectively. Since both implicit and explicit versions of constructs interface with maximal or typical performance conditions, case validity requires systematic inclusion of these distinctions in assessment protocols.     

The pathology of validity

Stephen Read has presented an argument for the inconsistency of the concept of validity. We extend Read’s results and show that this inconsistency is but one half of a larger problem. Like the concept of truth, validity is infected with what we call semantic pathology, a condition that actually gives rise to two symptoms: inconsistency and indeterminacy. After sketching the basic ideas behind semantic pathology and explaining how it manifests both symptoms in the concept of truth, we present cases that establish the indeterminacy of validity and that link this indeterminacy with the concept’s inconsistency. Our conclusion is that an adequate treatment of the semantic pathology thus revealed must deal with both of its symptoms. Further, it must extend to the occurrences of this condition elsewhere: in the concept of truth, in the other central semantic notions, and even in certain philosophical concepts outside semantics.     

The concept of validity

This article advances a simple conception of test validity: A test is valid for measuring an attribute if (a) the attribute exists and (b) variations in the attribute causally produce variation in the measurement outcomes. This conception is shown to diverge from current validity theory in several respects. In particular, the emphasis in the proposed conception is on ontology, reference, and causality, whereas current validity theory focuses on epistemology, meaning, and correlation. It is argued that the proposed conception is not only simpler but also theoretically superior to the position taken in the existing literature. Further, it has clear theoretical and practical implications for validation research. Most important, validation research must not be directed at the relation between the measured attribute and other attributes but at the processes that convey the effect of the measured attribute on the test scores.     

Criterion-related construct validity

Established results on latent variable models are applied to the study of the validity of a psychological test. When the test predicts a criterion by measuring a unidimensional latent construct, not only must the total score predict the criterion, but the joint distribution of criterion scores and item responses must exhibit a certain pattern. The presence of this population pattern may be tested with sample data using the stratified Wilcoxon rank sum test. Often, criterion information is available only for selected examinees, for instance, those who are admitted or hired. Three cases are discussed: (i) selection at random, (ii) selection based on the current test, and (iii) selection based on other measures of the latent construct. Discriminant validity is also discussed.This work was supported in part by Grant SES-87-01890 from the Measurement Methods and Data Improvement Program of the U.S. National Science Foundation.     

The validity of verbal protocols

The reactivity of a "think aloud" verbal protocol and the veridicality of different retrospective protocols were tested over four dissimilar tasks. Generating a concurrent protocol altered accuracy in two tasks, simple addition and a choice between two gambles, and generally prolonged response times. Such reactivity partially qualifies the dominant theory of protocol generation (Ericsson & Simon, 1984). Retrospective protocols yielded substantial forgetting or fabrication in all tasks, supporting the consensus on the nonveridicality of these methods. It is concluded that protocol validity should be based on an empirical check rather than on theory-based assurances.     

The construct validity of burnout

Considerable confusion exists in the burnout literature about the similarities and separateness among the constructs of burnout, depression and job satisfaction. This study examined the construct validity of burnout using Campbell & Fiske's multitrait-multimethod approach. The constructs of burnout, depression and orderliness (chosen as a discriminant construct) were assessed by three types of self-report instruments. Data were analysed for 181 females and 139 male faculty members who spent at least 50 per cent of their time in teaching. Results indicated that measures of burnout largely met Campbell & Fiske's criteria for construct validity: burnout measures possessed adequate to good internal reliability and convergent validity. However, burnout measures also correlated substantially with measures of depression. Results are discussed in terms of possible explanations for the substantial overlap that has been demonstrated among measures of burnout, depression and job satisfaction.     

The validity of psychotherapy 1

How good is psychotherapy as a tool of research into human nature? There is an orthodox defence of it as a research tool, which relies on showing that interpretations are true of the patient when they satisfy certain criteria. This defence is examined and rejected. The reply is considered that an interpretation which ‘keeps things moving’ is true, or an approximation to the truth. This reply is rejected by comparing and contrasting an interpretation in psychotherapy with one from brainwashing sessions. Two suppositions about psychotherapy (which weaken its discovery powers) are deliberately adopted — the suppositions that the material produced is ‘perspective’ and ‘method’ dependent. An unorthodox defence of psychotherapy is then offered — by means of a watery sense of ‘discover’. This enables us to outline a weak sense in which psychotherapy is a valid enterprise, but one which is still strong enough for research purposes.     

The validity of aphasic subtypes

The validity of aphasic subtypes derived by the Norwegian Basic Aphasia Assessment, NBAA, is discussed. 193 aphasic patients were reclassified according to the criteria of classification used by the Western Aphasia Battery. 85% agreement was established with mixed cases held apart, Wernicke's and Counduction aphasia being the least stable groups. Cluster analysis produced 9 clusters, of which 8 proved statistically valid through discriminant analysis and clinically valid through inspection of the derived test profiles. These were: Global, Broca's, Wernicke's and Anomic aphasia, each separated in a major and a minor impaired group. Accepting unclassifiable cases and specifying the desired domain of validity when classifying aphasia, are discussed.     

Prognostic validity of the Rorschach

Fifty adult stutterers entering therapy at the UCLA Psychology Clinic were administered the Rorschach, with a Klopfer method inquiry, scoring, form level rating, and calculation of scores on the Rorschach Prognostic Rating Scale (RPRS). On the basis of independent clinician ratings of attitudinal or psychotherapeutic improvement, subjects were divided into groups of Improved Most (n = 21) and Improved Least (n = 29). Subjects were also divided into Continued (n = 43) and Dropped (n = 7). Logistic regression was employed to compare groups on the following Rorschach dimensions: Prognostic Score (RPRS); Human Movement; Animal Movement; Inanimate Movement; Shading; Color; Form Level. The Improved Most group was significantly higher in M, FM, Shading, and Productivity. The finding that M and FM discriminates between improvement groups corroborates results obtained in a previous study (Sheehan et al., 1954). The Rorschach movement variables, particularly M and FM, seem to be stable indicators of capacity for improvement in psychotherapy.     

Factorial validity of imagery measures

Two correlational studies and related factor analyses (N = 70. N = 76) were addressed to four psychometric hypotheses derived from the imagery-desensitization literature. Consistent with prior research, self-ratings of imaginal vividness correlated moderately positively across sense modalities. Vividness ratings were inconsistently related to imagery control scores however. The question of the extent to which imagery scales reflect the influences of response sets and general intelligence has been raised recently in this journal Behav. Res. and Therapy 10, 137 (1972). Acquiescence tendency does not appear to influence imagery questionnaire results and there is only a small contamination effect from the social desirability response style, thus suggesting that these questionnaires are substantially free of response bias invalidation. Verbal intelligence is unrelated to researched imaginal attributes, as measured by self-report scales and several behavioral measures, including location memory and spatial visualization. These behavioral measures, in turn, do not correlate with ratings of imagery characteristics, a finding in accord with previous literature. Although self-report imagery questionnaires stand up better to psychometric scrutiny than might have been expected, and thus prior research employing them cannot be dismissed summarily, it is argued that these questionnaires may not tap the variables most relevant to desensitization outcome or mediation. The necessity of using state rather than trait measures, and of relating these to physiological indices, is highlighted.     

Two notions of epistemic validity

How to accept a conditional? F. P. Ramsey proposed the following test in (Ramsey 1990). (RT) ‘If A, then B’ must be accepted with respect to the current epistemic state iff the minimal hypothetical change of it needed to accept A also requires accepting B. In this article we propose a formulation of (RT), which unlike some of its predecessors, is compatible with our best theory of belief revision, the so-called AGM theory (see (Gärdenfors 1988), chapters 1–5 for a survey). The new test, which, we claim, encodes some of the crucial insights defended by F. P. Ramsey in (Ramsey 1990), is used to study the conditionals epistemically validated by the AGM postulates. Our notion of validity (PV) is compared with the notion of negative validity (NV) used by Gärdenfors in (Gärdenfors 1988). It is observed that the notions of PV and NV will in general differ and that when these differences arise it is the notion of PV that is preferable. Finally we compare our formulation of the Ramsey test with a previous formulation offered by Gärdenfors (GRT). We show that any attempt to interpret (GRT) as delivering acceptance conditions for Ramsey's conditionals is doomed to failure.     

Habermas and validity claims

At the heart of Jürgen Habermas’s explication of communicative rationality is the contention that all speech acts oriented to understanding raise three different kinds of validity claims simultaneously: claims to truth, truthfulness, and normative rightness. This paper argues that Habermas presents exactly three distinct, logically independent arguments for his simultaneity thesis: an argument from structure; an argument from criticizability/rejectability; and an argument from understanding/reaching understanding. It is further maintained that the simultaneity thesis receives cogent support only from the Argument from understanding/reaching understanding, and only if the notion of ‘understanding’ is expanded to that of ‘agreement’.     

Faithfulness for naive validity

Nontransitive responses to the validity Curry paradox face a dilemma that was recently formulated by Barrio, Rosenblatt and Tajer. It seems that, in the nontransitive logic ST enriched with a validity predicate, either you cannot prove that all derivable metarules preserve validity, or you can prove that instances of Cut that are not admissible in the logic preserve validity. I respond on behalf of the nontransitive approach. The paper argues, first, that we should reject the detachment principle for naive validity. Secondly, I show how to add a validity predicate to ST while avoiding the dilemma.

     

Deculturation: its lack of validity

Deculturation's history as well as its use in the social science literature is reviewed and its construct validity questioned. According to J. W. Berry (1980), deculturation results when members of nondominant cultures become alienated from the dominant culture and from their own minority society. A reported consequence of deculturation is increased stress and psychopathology for the individuals involved. Deculturation's basic assumption that there can be culture loss without replacement is critically assessed. In particular, the idea that members of nondominant groups can become deculturated, devoid of any culture, is challenged.