Validity of self-reports of alcohol and other drug use: a multitrait-multimethod assessment

Four methods (self-reported rating, self-reported intake, peer-reported rating, peer-reported intake) and multiple analytical criteria were employed to assess the construct validity of reports of three types of substance use (alcohol, marijuana, and cigarettes) in a study of 194 college students who were members of friendship pairs. The resulting multitrait-multimethod matrix was analyzed in the traditional fashion as well as with confirmatory factor analysis. Convergent and discriminant validity for the three substance use traits were quite adequate, although the validity of one measure (the peer-reported intake measure of alcohol use) may be considered inadequate according to some criteria. This study demonstrates the usefulness of multimethod assessment and confirmatory factor analysis for research on construct validity and for derivation of more accurate measures of drug use.     

Assessing hopelessness in terminally ill cancer patients: development of the Hopelessness Assessment in Illness Questionnaire

Hopelessness has become an increasingly important construct in palliative care research, yet concerns exist regarding the utility of existing measures when applied to patients with a terminal illness. This article describes a series of studies focused on the exploration, development, and analysis of a measure of hopelessness specifically intended for use with terminally ill cancer patients. The 1st stage of measure development involved interviews with 13 palliative care experts and 30 terminally ill patients. Qualitative analysis of the patient interviews culminated in the development of a set of potential questionnaire items. In the 2nd study phase, we evaluated these preliminary items with a sample of 314 participants, using item response theory and classical test theory to identify optimal items and response format. These analyses generated an 8-item measure that we tested in a final study phase, using a 3rd sample (n = 228) to assess reliability and concurrent validity. These analyses demonstrated strong support for the Hopelessness Assessment in Illness Questionnaire providing greater explanatory power than existing measures of hopelessness and found little evidence that this assessment was confounded by illness-related variables (e.g., prognosis). In summary, these 3 studies suggest that this brief measure of hopelessness is particularly useful for palliative care settings. Further research is needed to assess the applicability of the measure to other populations and contexts.     

Further validation of a measure of irrational belief

Malouff and Schutte (1986) developed a measure of irrational belief intended to be shorter and more content valid than other such measures and reported preliminary evidence of reliability and validity for the measure. The present two studies examined further the validity of the new measure. Study 1 produced evidence of construct validity in that a significant correlation was found between scores on the Malouff & Schutte measure and scores on a measure of hostility, which is a theoretically related construct. Study 2 produced evidence of construct validity in that scores on the Malouff & Schutte measure were found to be correlated with scores on two measures of theoretically related constructs: anxiety and irrational tendencies/beliefs. Study 2 also produced evidence of discriminant validity in that scores on the Malouff & Schutte measure were found to be significantly more highly correlated with scores on the more theoretically similar irrational tendencies/beliefs scale than with scores on the anxiety measure.     

Constructs and developmental assessment centers: Further troubling empirical findings

Research indicates that assessment center (AC) ratings typically demonstrate poor construct validity; that is, they do not measure the intended dimensions of managerial performance (e.g., Sackett & Harris, 1988). The purpose of this study was to investigate the construct validity of dimension ratings from a developmental assessment center (N=102), using multitrait-multimethod analysis and factor analysis. The relationships between AC ratings, job performance ratings, and personality measures also were investigated. Results indicate that the AC ratings failed to demonstrate construct validity. The ratings did not show the expected relationships with the job performance and personality measures. Additionally, the factors underlying these ratings were found to be the AC exercises, rather than the managerial dimensions as expected. Potentially, this lack of construct validity of the dimension ratings is a serious problem for a developmental assessment center. There is little evidence that the managerial weaknesses identified by the AC are the dimensions that actually need to be improved on the job. Methods are discussed for improving the construct validity of AC ratings, for example, by decreasing the cognitive demands on the assessors.This study is based on a dissertation submitted to North Carolina State University. Portions of this paper were presented at the meeting of the Society for Industrial and Organizational Psychology in Montreal, Quebec, May, 1992. I am grateful to Paul Thayer, Bert Westbrook, James W. Cunningham, and Patrick Hauenstein for their contributions to this research. I also thank several anonymous reviewers for their comments on this article.     

An item‐level investigation of conceptual and empirical distinctiveness of proactivity constructs

Beyond the prior investigations that took scale‐level approaches to determining discriminant validity in proactivity constructs, the current study contributes a much‐needed interrogation of the items used to measure the behaviors in this domain. The substantive validity (SV) assessments (Study 1) showed that many of the items were judged to be inconsistent with the definition of the construct they assess or, alternatively, more consistent with the definition of a different construct in the domain. Further, exploratory factor analysis revealed the difficulty in empirically separating the four behaviors, while BiM results also advocated against the unique variance of them after accounting for a general factor (Study 2). Altogether, our results show that the items are partly to blame for the empirical redundancy issue.     

对学前儿童语言学习能力诊断量表的效度评价

以所编制的量表为工具 ,对采集的数据进行效度分析 ,结果表明各分测验与全量表有较好的相关 ,说明量表的内容效度是比较高的。使用因素分析的方法 ,将全部变量作系统分类 ,研究量表的结构 ,绝大部分分测验在所得的四个因素上的共通性都大于 0 .70 ;保留下的分测验与所属因素的相关系数在0 .5 3 -0 .84之间 ,它们在各个因素上有较高的负荷量 ,说明量表有较好的结构效度。从效度分析的结果看 ,本量表的测量结果应该是准确的。另外 ,还根据因素分析结果指示的方向 ,调整了分测验 ,调整后的量表结构不但与假设的量表结构十分吻合 ,而且更条理化。     

Trait emotional intelligence: psychometric investigation with reference to established trait taxonomies

This paper sets out the theoretical foundation of emotional intelligence (EI) as a constellation of traits and self‐perceived abilities. The discriminant validity of trait EI is explored in two studies. In study 1 (N = 227), the psychometric properties of the BarOn Emotional Quotient inventory were scrutinized through confirmatory factor analysis and the measure was found to be unifactorial. When the EQ‐i was examined concurrently with the Eysenck Personality Profiler, a clear trait EI factor emerged in Eysenckian factor space. In study 2 (N = 166), a modified version of the EQ‐i was examined concurrently with the NEO PI‐R and a truncated trait EI factor was isolated within the Five‐Factor Model. Results are discussed with explicit reference to established personality models and it is concluded that trait EI can be conceptualized as a distinct composite construct at the primary level of hierarchical trait structures. Copyright © 2001 John Wiley & Sons, Ltd.     

The depressive personality disorder inventory and current depressive symptoms: implications for the assessment of depressive personality

The Depressive Personality Disorder Inventory (DPDI; Huprich, Margrett, Barthelemy, & Fine, 1996; see Appendix) was created to assess Depressive Personality Disorder in clinical and nonclinical samples. Since its creation, the DPDI has been used in multiple studies, and the psychometric properties of the measure have generally supported its reliability, convergent validity, and construct validity; however, evidence for the measure's discriminant validity has been mixed. Specifically, the DPDI tends to correlate highly with measures of current depressive symptoms, which limits its efficacy in differentiating current depressive symptoms from a depressive personality structure. A principal components analysis of 362 individuals who completed both the DPDI and Beck Depression Inventory (BDI-II; Beck, Steer, & Brown, 1996) found that 49% of the variance was accounted for in two components. Seven items from the DPDI loaded more strongly on the first component composed of many BDI-II items. These items were removed in order to create a measure believed to assess DPD without the confounding influence of current depressive symptomology. Principal components analysis of the revised measure yielded three components, accounting for 46% of the variance. The revised DPDI was used to calculate convergent, discriminant, and construct validity coefficients from measures used in former studies. Virtually no improvement in the validity coefficients was observed. It is concluded that assessing DPD via self-report is limited in its utility.     

Validity and Reliability of a Turkish Version of the Modified Moral Sensitivity Questionnaire for Student Nurses

This study examined the validity and reliability of a Turkish version of the Modified Moral Sensitivity Questionnaire for Student Nurses (MMSQSN). After obtaining permission to adapt the MMSQSN into Turkish, the translation/back-translation method was used with expert opinions to determine content validity. Factor analysis was conducted to examine the construct validity and test–retest was performed on the questionnaire to determine reliability. Cronbach’s alpha coefficients were calculated to assess for internal consistency. Participants included 272 baccalaureate degree student nurses who took ethics lessons prior to their clinical internship. The factor analysis revealed that even though the factor structure in the original scale was the same, relevant items were categorized with similar components, and factor loads were sufficient. The correlation coefficient in the analyses of test–retest scores was .66 for the total scale (p < .05) and the Cronbach’s alpha was .73 for the total scale. The translated MMSQSN is a valid and reliable measure of ethical sensitivity in student nurses in Turkey.     

Initial development of a brief measure of psychological distress

This paper addresses the preliminary development, construct validity, and psychometric properties of a brief self-report measure of psychological distress. 40 items were originally generated by doctoral level psychologists for use in the preliminary clinical sample. Inpatients from a psychiatric unit (N = 125) completed the items, and a principal-components analysis with a direct oblimin rotation was used to evaluate construct validity. The study indicated a four-factor solution, using the constructs of Depression, Hopelessness, Anxiety, and Anger, with good estimates of reliability. After evaluation of factor structure, item analyses, and reliability estimates, a redacted 19-item scale was identified.     

Differentiating anxious,aggressive, and socially competent preschool children: validation of the Social Competence and Behavior Evaluation-30 (parent version)

The present study examined the factor structure, internal consistency, and construct validity of the parent version of the Social Competence and Behavior Evaluation-30 for preschoolers (SCBE-30; LaFreniere, P. J. (1990). Social competence and behavior evaluation-30. Unpublished measure.), an adaptation of the validated teacher version of the same measure (LaFreniere & Dumas, Psychol. Asses. 8 (1996) 369). The parent version of the SCBE-30 is a 30-item Likert rating scale questionnaire designed to assess patterns of anxiety/withdrawal, anger/aggression, and social competence. Principal components analysis was used to identify the factor structure of the parent version of the SCBE-30 (N = 218 preschool children). To assess construct validity, a compliance task was utilized to determine whether children identified as high on anxiety/withdrawal, anger/aggression, or social competence with the parent version of the SCBE-30 (n = 20 for each group) could be distinguished behaviorally on several observational variables. Principal components analysis identified three factors accounting for 44% of the variance. Ten items positively loaded onto each factor and matched conceptual expectations. A between-subjects MANOVA demonstrated significant group differences in observed child behaviors including compliance, noncompliance, subtypes of noncompliance, and aversive behavior. Results of the current study suggested that the parent version of the SCBE-30 demonstrated both internal consistency and construct validity, and findings paralleled many of the results from LaFreniere and Dumas' validation of the teacher version of the SCBE-30.     

Assessing metacognitive skills in waking and sleep: a psychometric analysis of the Metacognitive, Affective, Cognitive Experience (MACE) questionnaire

The Metacognitive, Affective, Cognitive Experience (MACE) questionnaire was designed to assess metacognition across sleep and waking (Kahan & LaBerge, 1996). The present research evaluates the psychometric properties of the MACE. Data from two recent studies (N=185) were used to assess the inter-item consistency, test-retest reliability, and factorial, convergent, and discriminant validity of the MACE. Results show that the MACE is a reliable measure with good construct validity. Exploratory factor analyses revealed one self-regulation and two monitoring factors. One monitoring factor emphasized monitoring internal conditions; the other emphasized monitoring external conditions. This factor structure is consistent with the Metacognitive Model (Nelson & Narens, 1990). Tests of convergent and discriminant validity suggest that the MACE is assessing metacognition and is appropriately related to similar constructs such as mindfulness and self-consciousness. The implication of these findings as well as suggestions for research and clinical applications of the MACE are discussed.     

The General Environment Fit Scale: A Factor Analysis and Test of Convergent Construct Validity

Person-environment fit (P-E fit) was initially espoused as an important construct in the field of community psychology; however, most of the theoretical and empirical development of the construct has been conducted by the industrial/organizational (I/O) psychologists and business management fields. In the current study, the GEFS-a P-E fit measure that was developed from organizational perspectives on fit-was administered to 246 attendees of an annual convention for residents and alumni of Oxford House, a network of over 1,400 mutual-help addiction recovery homes. The authors conducted confirmatory factor and convergent construct validity analyses with the GEFS. The results suggested that the theoretical factor structure of the measure adequately fit the data and provided limited support for the measure's validity. Sufficient supply of resident needs by the Oxford House and similarity between residents and their housemates predicted satisfaction with the recovery home, but only perceived similarity to housemates predicted how long residents intended to stay in the Oxford Houses.     

Factor Analysis of the Mystical Experience Questionnaire: A Study of Experiences Occasioned by the Hallucinogen Psilocybin

A large body of historical evidence describes the use of hallucinogenic compounds, such as psilocybin mushrooms, for religious purposes. But few scientific studies have attempted to measure or characterize hallucinogen‐occasioned spiritual experiences. The present study examined the factor structure of the Mystical Experience Questionnaire (MEQ), a self‐report measure that has been used to assess the effects of hallucinogens in laboratory studies. Participants (N = 1,602) completed the 43‐item MEQ in reference to a mystical or profound experience they had had after ingesting psilocybin. Exploratory factor analysis of the MEQ retained 30 items and revealed a four‐factor structure covering the dimensions of classic mystical experience: unity, noetic quality, sacredness (F1); positive mood (F2); transcendence of time/space (F3); and ineffability (F4). MEQ factor scores showed good internal reliability and correlated with the Hood Mysticism Scale, indicating convergent validity. Participants who endorsed having had a mystical experience on psilocybin, compared to those who did not, had significantly higher factor scores, indicating construct validity. The four‐factor structure was confirmed in a second sample (N = 440) and demonstrated superior fit compared to alternative models. The results provide initial evidence of the validity, reliability, and factor structure of a 30‐item scale for measuring single, hallucinogen‐occasioned mystical experiences, which may be a useful tool in the scientific study of mysticism.     

Examining the validity of self‐reports on scales measuring students' strategic processing

Background. Self‐report inventories trying to measure strategic processing at a global level have been much used in both basic and applied research. However, the validity of global strategy scores is open to question because such inventories assess strategy perceptions outside the context of specific task performance. Aims. The primary aim was to examine the criterion‐related and construct validity of the global strategy data obtained with the Cross‐Curricular Competencies (CCC) scale. Additionally, we wanted to compare the validity of these data with the validity of data obtained with a task‐specific self‐report inventory focusing on the same types of strategies. Sample. The sample included 269 10th‐grade students from 12 different junior high schools. Methods. Global strategy use as assessed with the CCC was compared with task‐specific strategy use reported in three different reading situations. Moreover, relationships between scores on the CCC and scores on measures of text comprehension were examined and compared with relationships between scores on the task‐specific strategy measure and the same comprehension measures. Results. The comparison between the CCC strategy scores and the task‐specific strategy scores suggested only modest criterion‐related validity for the data obtained with the global strategy inventory. The CCC strategy scores were also not related to the text comprehension measures, indicating poor construct validity. In contrast, the task‐specific strategy scores were positively related to the comprehension measures, indicating good construct validity. Conclusion. Attempts to measure strategic processing at a global level seem to have limited validity and utility.     

Initial Development and Validation of a Measure of Intercultural Development Scale

Intercultural development in global employees will aid organizations to function successfully in the current global work scene. However, there is a need for a psychometrically valid and reliable measure, which can be used by practitioners and academics to assess the capabilities of individuals. This vignette based intercultural development scale was conceived on the basis of Bennett’s developmental model of intercultural sensitivity. The content was generated using interviews with international employees and subject matter experts. Psychometric properties were established with both qualitative and quantitative analysis. A sample of 461 international employees was used to test the validity of the scale. Confirmatory factor analysis, reliability analysis and construct validity was established with cosmopolitanism and openness to experience scales. The study confirms the scale’s dimensionality and established convergent, discriminant and nomological validity. The applications of this scale in building capabilities, talent management and strategic decisions in both practical and theoretical assessment are outlined.     

Psychometric Properties of the Persian Version of Spiritual Well-Being Scale in Patients with Acute Myocardial Infarction

The purpose of this study was to assess the psychometric properties of the Persian version of Spiritual Well-Being Scale (SWBS) in patients with acute myocardial infarction. A multisite, cross-sectional survey was employed to determine the instrument’s reliability (Cronbach’s α and construct reliability) and validity (face, content, and construct). Using systematic sampling of adult outpatients at primary care clinic sites in the Qazvin City, Iran (N = 300), it was found that the Cronbach’s alpha and construct reliability of both factors associated with the SWBS were above 0.7. The construct validity of the scale was determined using exploratory factor analysis. The findings supported two factors: relation with God and relation with life. Further investigation through confirmatory factor analysis (eigenvalues of greater than one) confirmed a third factor construct associated with the SWBS. A total of 50.65 % of the variance were explained by these three factors. The overall findings of the study demonstrated that the SWBS is a valid and reliable instrument that has potential utility in future research and clinical practice settings.     

Coping style in predicting posttraumatic stress disorder among Israeli students

Abstract

Two possible motives for sensation seeking behavior, escape from self-awareness and compensatory self-regulation, were investigated in two studies. In Study I a questionnaire was developed to identify the putative escape and compensation motives suspected to contribute to sensation seeking behaviors. In Study II the factor structure of this new measure, called the Risk and Excitement Inventory (REI), was cross-validated using confirmatory factor analysis, and construct validity was examined. Together these studies suggest that the REI comprises two reliable factors, compensation and escape from self-awareness. Study II showed that these factors have a reasonable degree of construct validity. Limits of the measure and the model are discussed, and suggestions are offered for future research.     

Rainbows in the society: A measure of hope for society

The aim of this study was to develop and validate a measure of social hope. Previous studies have extensively reported on personal hope and its outcomes. Although a few studies have suggested the concept of collective or social hope, there is no instrument to assess the construct. In this study, we created a 5‐item scale, drawing on Snyder's hope theory, and collected two sets of survey data online. Analysis of these data sets revealed that the new scale was structurally unidimensional and internally consistent. In association with Snyder's State Hope Scale, the Social Hope Scale showed convergent and discriminant validity. The social hope construct also exhibited theoretically meaningful relationships with political efficacy and self‐construal (concurrent validity). Finally, social hope was higher for those who had participated in certain types of social activism than it was for those who had not (predictive validity). These results suggest that the Social Hope Scale is a reliable and valid self‐report measure of hopeful thinking for society.     

The development of the Cultural Health Attributions Questionnaire (CHAQ)

Current general health belief measures that are used to assess health behaviors do not capture the full range of health beliefs present among Latinos. The purpose of this study was to develop a reliable and valid measure of Latino health beliefs, the Cultural Health Attributions Questionnaire (CHAQ). Three hundred forty participants were recruited in 2 metropolitan areas with large Latino populations. Exploratory factor analysis revealed 2 highly interpretable 12-item subscales: the Equity Attribution and the Behavioral-Environmental Attribution scales. Examination of the relationships between the subscales and measures of acculturation provided evidence of construct validity. Moreover, the prediction of health care behavior by the CHAQ also indicated initial criterion validity.