The Perceived Leadership Communication Questionnaire (PLCQ): Development and Validation

The Perceived Leadership Communication Questionnaire (PLCQ) is a short, reliable, and valid instrument for measuring leadership communication from both perspectives of the leader and the follower. Drawing on a communication-based approach to leadership and following a theoretical framework of interpersonal communication processes in organizations, this article describes the development and validation of a one-dimensional 6-item scale in four studies (total N = 604). Results from Study 1 and 2 provide evidence for the internal consistency and factorial validity of the PLCQ's self-rating version (PLCQ-SR)—a version for measuring how leaders perceive their own communication with their followers. Results from Study 3 and 4 show internal consistency, construct validity, and criterion validity of the PLCQ's other-rating version (PLCQ-OR)—a version for measuring how followers perceive the communication of their leaders. Cronbach's α had an average of.80 over the four studies. All confirmatory factor analyses yielded good to excellent model fit indices. Convergent validity was established by average positive correlations of.69 with subdimensions of transformational leadership and leader–member exchange scales. Furthermore, nonsignificant correlations with socially desirable responding indicated discriminant validity. Last, criterion validity was supported by a moderately positive correlation with job satisfaction (r =.31).     

Development and Validation of a Deontic Justice Scale

This paper describes the development and validation of a deontic justice scale (DJS). Study 1 (n = 124) was conducted to test the initial 36‐item version of the scale. It resulted in the reduction of the initial scale to 18 items, including 3 dimensions: moral obligation (8 items), moral accountability (6 items), and moral outrage (4 items). Study 2 (n = 101) was conducted to examine the construct validity and confirm the factor structure of the DJS. Findings from both studies showed evidence of the scale's construct validity. They also showed that deontic justice is a multidimensional construct encompassing moral obligation, moral accountability, and moral outrage. The scale's implications for use as an adequate research instrument are discussed.     

Cognitive ability in virtual reality: Validity evidence for VR game‐based assessments

Virtual reality (VR) technology may provide unique benefits for assessing individual characteristics, in contrast with traditional assessments. This study examined validity evidence for the use of VR game (VRG) scores to indicate specific cognitive abilities. Participants completed three VRGs, three computer‐based assessments (CBAs), and additional scales. To evaluate convergent validity, VRG scores were compared with CBA scores. For divergent validity, VRG scores were compared with five factors of personality based on the Five Factor Model. For criterion‐related validity, VRG scores were compared with academic achievement (i.e., grade point average). Findings generally provided support that the VRG scores are related to certain spatial reasoning subdimensions of cognitive ability. However, further research is needed to examine the qualities of VRGs as assessment tools.     

Increasing Validity with Forced‐Choice Criterion Measurement Formats

The relative validities of forced‐choice (ipsative) and Likert rating‐scale item formats as criterion measures are examined. While there has been much debate about the relative technical and psychometric merits and demerits of ipsative instruments, the present research focused on the crucial question of whether the use of this format has any practical benefit – in terms of improved validity. An analysis is reported from a meta‐analysis data set. This demonstrates that higher operational validity coefficients (prediction of line‐manager ratings of competencies) are associated with the use of forced‐choice (r=.38) rather than rating scale (r=.25) item formats for the criterion measurement instrument when performance is rated by the same line managers on both formats and where the predictor is held constant. Thus the apparent criterion‐related validity of a predictor can increase by 50% simply by changing the format of the criterion measurement instrument. The implications of this for practice are discussed.     

The positive and negative semantic dimensions of relationship satisfaction

Semantic differential items were reconfigured to assess relationship satisfaction across separate positive and negative attitude dimensions. Study 1 (N = 1,656) supported a 2‐factor model for the Positive and Negative Semantic Differential (PN–SMD), as well as its convergent, criterion‐related, and incremental validity over the 16‐item Couples Satisfaction Index (CSI; J. L. Funk & R. D. Rogge, 2007) using known correlates of relationship satisfaction as criteria. Study 2 (N = 89) replicated the convergent, criterion‐related, and incremental validity findings of Study 1 using different criterion measures, the CSI, a bipolar semantic differential measure designed for assessing relationship satisfaction, and an existing 2‐dimensional measure of relationship satisfaction. The authors demonstrated across studies that the PN–SMD captures criterion‐relevant information about ambivalence versus indifference toward the relationship—associations that are only detectable when using a 2‐dimensional satisfaction measure.     

Health anxiety and hypochondriasis in the light of DSM-5

Background: In the DSM-5, the diagnosis of hypochondriasis was replaced by two new diagnositic entities: somatic symptom disorder (SSD) and illness anxiety disorder (IAD). Both diagnoses share high health anxiety as a common criterion, but additonal somatic symptoms are only required for SSD but not IAD. Design: Our aim was to provide empirical evidence for the validity of these new diagnoses using data from a case–control study of highly health-anxious (n = 96), depressed (n = 52), and healthy (n = 52) individuals. Results: The individuals originally diagnosed as DSM-IV hypochondriasis predominantly met criteria for SSD (74%) and rarely for IAD (26%). Individuals with SSD were more impaired, had more often comorbid panic and generalized anxiety disorders, and had more medical consultations as those with IAD. Yet, no significant differences were found between SSD and IAD with regard to levels of health anxiety, other hypochondriacial characteristics, illness behavior, somatic symptom attributions, and physical concerns, whereas both groups differed significantly from clinical and healthy controls in all of these variables. Conclusion: These results do not support the proposed splitting of health anxiety/hypochondriasis into two diagnoses. Further validation studies with larger samples and additional control groups are warranted to prove the validity of the new diagnoses.     

It’s security,stupid! Voters’ perceptions of immigrants as a security risk predicted support for Donald Trump in the 2016 US presidential election

We analyzed two datasets to determine the predictive validity of four explanations of support for Donald Trump during the 2016 US presidential election: (a) security concerns regarding immigrants, (b) economic concerns regarding immigrants, (c) cultural concerns regarding immigrants, and (d) social dominance orientation. Results of a two‐phase study (N = 354) suggested that perceiving immigrants as a security concern was predictive of increased support for and greater odds of voting for Donald Trump three weeks later. Perceiving immigrants as an economic threat predicted odds of voting for Donald Trump, but only among liberals and there was no evidence of cultural concern or social dominance orientation (SDO) predicting support for Donald Trump or odds of voting for Trump. A follow‐up analysis of the cross‐sectional ANES survey corroborated that security concerns were an important correlate of voting for Trump, but also that SDO was correlated with having previously voted for Donald Trump. While our two‐phase study has the benefit of prediction, the cross‐sectional ANES data does not—“predictors” in these data were collected up to two months post‐election.     

Development and content validity of a screening instrument for gaming addiction in adolescents: The Gaming Addiction Identification Test (GAIT)

This study describes the development of a screening tool for gaming addiction in adolescents – the Gaming Addiction Identification Test (GAIT). Its development was based on the research literature on gaming and addiction. An expert panel comprising professional raters (n = 7), experiential adolescent raters (n = 10), and parent raters (n = 10) estimated the content validity of each item (I‐CVI) as well as of the whole scale (S‐CVI/Ave), and participated in a cognitive interview about the GAIT scale. The mean scores for both I‐CVI and S‐CVI/Ave ranged between 0.97 and 0.99 compared with the lowest recommended I‐CVI value of 0.78 and the S‐CVI/Ave value of 0.90. There were no sex differences and no differences between expert groups regarding ratings in content validity. No differences in the overall evaluation of the scale emerged in the cognitive interviews. Our conclusions were that GAIT showed good content validity in capturing gaming addiction. The GAIT needs further investigation into its psychometric properties of construct validity (convergent and divergent validity) and criterion‐related validity, as well as its reliability in both clinical settings and in community settings with adolescents.     

The Variability of Criterion‐Related Validity Estimates Among Interviewers and Interview Panels

The authors examined differences in criterion‐related validity estimates among ratings from individual interviewers and interview panels within a structured interview. Senior non‐commissioned officers (NCOs) in the U.S. Army (N=64) conducted panel interviews with 944 junior NCOs during a concurrent validation project. Analysis of the data revealed considerable variation in interviewer validity coefficients in relation to multiple performance criteria. Results also indicated the importance of adopting a multivariate perspective when evaluating interviewer validity differences in that the amount of variation in validity coefficients differed both by interview dimension and criterion. A similar pattern of findings emerged when analyses were performed on ratings averaged within interview panel. Nonetheless, when meta‐analysis was used to estimate the amount of true variance in interviewer‐ and panel‐level validity coefficients, most or all of the variance for some interview‐criterion combinations appeared to be due to statistical artifacts.     

A Psychometric Network Perspective on the Validity and Validation of Personality Trait Questionnaires

This article reviews the causal implications of latent variable and psychometric network models for the validation of personality trait questionnaires. These models imply different data generating mechanisms that have important consequences for the validity and validation of questionnaires. From this review, we formalize a framework for assessing the evidence for the validity of questionnaires from the psychometric network perspective. We focus specifically on the structural phase of validation, where items are assessed for redundancy, dimensionality, and internal structure. In this discussion, we underline the importance of identifying unique personality components (i.e. an item or set of items that share a unique common cause) and representing the breadth of each trait's domain in personality networks. After, we argue that psychometric network models have measures that are statistically equivalent to factor models but we suggest that their substantive interpretations differ. Finally, we provide a novel measure of structural consistency, which provides complementary information to internal consistency measures. We close with future directions for how external validation can be executed using psychometric network models. © 2020 European Association of Personality Psychology     

Factor Analysis of the Mystical Experience Questionnaire: A Study of Experiences Occasioned by the Hallucinogen Psilocybin

A large body of historical evidence describes the use of hallucinogenic compounds, such as psilocybin mushrooms, for religious purposes. But few scientific studies have attempted to measure or characterize hallucinogen‐occasioned spiritual experiences. The present study examined the factor structure of the Mystical Experience Questionnaire (MEQ), a self‐report measure that has been used to assess the effects of hallucinogens in laboratory studies. Participants (N = 1,602) completed the 43‐item MEQ in reference to a mystical or profound experience they had had after ingesting psilocybin. Exploratory factor analysis of the MEQ retained 30 items and revealed a four‐factor structure covering the dimensions of classic mystical experience: unity, noetic quality, sacredness (F1); positive mood (F2); transcendence of time/space (F3); and ineffability (F4). MEQ factor scores showed good internal reliability and correlated with the Hood Mysticism Scale, indicating convergent validity. Participants who endorsed having had a mystical experience on psilocybin, compared to those who did not, had significantly higher factor scores, indicating construct validity. The four‐factor structure was confirmed in a second sample (N = 440) and demonstrated superior fit compared to alternative models. The results provide initial evidence of the validity, reliability, and factor structure of a 30‐item scale for measuring single, hallucinogen‐occasioned mystical experiences, which may be a useful tool in the scientific study of mysticism.     

Measuring test anxiety in children: Scale development and internal construct validation

Given the increased testing of school-aged children in the United States there is a need for a current and valid scale to measure the effects of test anxiety in children. The domain of children's test anxiety was theorized to be comprised of three dimensions: thoughts, autonomic reactions, and off-task behaviors. Four stages are described in the evolution of the Children's Test Anxiety Scale (CTAS): planning, construction, quantitative evaluation, and validation. A 50-item scale was administered to a development sample (N=230) of children in grades 3–6 to obtain item analysis and reliability estimates which resulted in a refined 30-item scale. The reduced scale was administered to a validation sample (N=261) to obtain construct validity evidence. A three-factor structure fit the data reasonably well. Recommendations for future research with the scale are described.     

Development and validation of the Job Security Index and the Job Security Satisfaction scale: A classical test theory and IRT approach

This study describes the development and validation of two scales measuring job security: the Job Security Index (JSI), measuring an individual's cognitive appraisal of the future of his or her job with respect to the perceived level of stability and continuance of that job, and the Job Security Satisfaction (JSS) scale, measuring employee satisfaction with a perceived level of job security. Item response theory and classical test theory analyses indicate that the scales are highly reliable and exhibit good discriminant and criterion‐related validity. Future researchers are advised to apply confirmatory factor analysis (CFA) to the scales to confirm their factor structures.     

Conceptual foundations and validation of the Pseudoscientific Belief Scale

In this study, we present the Pseudoscientific Belief Scale (PSEUDO; α = 0.90). The conceptual foundations of the scale include (a) a philosophically grounded and functional demarcation criterion between science and pseudoscience; (b) an analysis of the pseudoscientific status of science denialism, which is integrated into the scale; and (c) a bibliographic justification to back up the inclusion of each item. The validation process is carried out based on two studies. Study 1 (N = 3,416) uses both exploratory factor analysis and cluster analysis: Conceptually congruent factors are not reported. Study 2 (N = 292) includes a total of 21 sociodemographic, personality, and cognitive independent variables, as well as the two thematic subscales of PSEUDO as dependent variables, in order to analyze the psychological profile of both sets of believers. Results show high similarity between both groups, confirming construct validity.     

Development and validation of the Indonesian Well‐being Scale

The present study describes the development and validation of a new instrument to measure the well‐being of Indonesian people. Items were generated by taking into account the recent cross‐cultural developments in the literature. Participants (N = 1,028) from a number of provinces in Indonesia completed an online or a paper questionnaire containing a battery of measures, including the new well‐being instrument. The total sample was randomly split into two equal groups. An exploratory factor analysis (n = 516) was conducted on one half to explore the factor structure of the new scale, which resulted in a 24‐item scale with a four‐factor solution. The four factors were (a) Basic Needs, (b) Social Relation, (c) Acceptance, and (d) Spirituality. A confirmatory factor analysis was conducted on the second half (n = 516), which confirmed the four‐factor structure with 20 items emerging as the best and most parsimonious fit of the data. The psychometric properties of the final scale were tested on the whole sample. The new scale displayed sound internal consistency, test‐retest reliability, and divergent, convergent, and discriminant validity. The scale has the potential to be used in future research which examines well‐being in the Indonesia context. Future directions and limitations of the study are discussed.     

The Anxiety Depression Distress Inventory–27 (ADDI–27): New Evidence of Factor Structure,Item-Level Measurement Invariance,and Validity

Three studies examining the factor structure and psychometric properties of the Anxiety Depression Distress Inventory–27 (ADDI–27) extended the initial instrument development studies for this recently introduced inventory. The ADDI–27 is an empirically derived short form of the Mood and Anxiety Questionaire–90 (MASQ–90) comprising three scales: Positive Affect, Somatic Anxiety, and General Distress. The main objectives of Study 1 (N = 700) were to examine the factor structure of the ADDI–27 and its measurement invariance across gender at the item level. The objective of Study 2 (N = 538) was to examine evidence for the convergent and discriminant validity of scores on the ADDI–27. The objective of Study 3 (N = 240) was to assess further evidence for the nomological network and convergent and discriminant validity of the ADDI–27 scores. Results of exploratory structural equation modeling yielded strong support for a 3-factor model, with approximate fit indexes meeting or exceeding the conventional cutoffs. With p ≤ .001 as the criterion for detecting noninvariance, results of measurement invariance analysis suggested that all of the ADDI–27 items were invariant across gender. Results of multivariate validity analyses across 2 studies provided support for the convergent and discriminant validity of scores on the ADDI–27 scales.     

A New Multidimensional Measure of Spirituality‐Religiosity for Use in Diverse Substance Abuse Treatment Populations

Extant measures of spirituality‐religiosity, not developed specifically with substance abuse treatment populations in mind, may not be culturally appropriate in regard to either: (1) the 12‐step “culture” found in many treatment programs; or (2) racial‐ethnic minority populations overrepresented in publicly funded programs. A 40‐item four‐dimensional measure was developed, which differentiates “religiosity” (religious practices) from “spirituality” (an individual's relationships with God/higher power, others, and self). Instrument development involved: qualitative focus groups with individuals in seven diverse Texas treatment programs; quantitative exploratory and confirmatory (CFA) analyses to test the four‐factor model conducted using data from 237 diverse clients in treatment programs in Tennessee and Virginia. Confirmatory analyses indicate acceptable fit indices (>0.90) for the four‐factor model, and acceptable reliability estimates for all subdimensions (≥0.70) provide further support for the measures. Results support the potential usefulness of the measures.     

Perceived Benefits Following Sport Injury: A Qualitative Examination of their Antecedents and Underlying Mechanisms

The purpose of the present study was to describe the development and validation of the Self-Talk Questionnaire (S-TQ), designed to measure two functions of self-talk strategies, namely the motivational function and the cognitive function, used by athletes to enhance their sport performances. The study was conducted in three phases. The first phase targeted item selection and content relevance. The second phase set out to test the factor structure of the instrument. The aim of the third phase was focused on further examination of the psychometric properties of the instrument. The results of an exploratory factor analysis provided adequate support for the instrument. The S-TQ consists of 11 items, representing two factors (motivational and cognitive). A confirmatory factor analysis showed an acceptable fit to the data. The factors were internally consistent, the correlations with external criteria were significant, and test-retest reliability values were very good. Examining the relationships between the S-TQ and similar constructs supported convergent validity. Discriminant validity was supported by correlations between the S-TQ and a different construct. The S-TQ appears to be a promising psychometric instrument that can be used in research and psychological intervention programs.     

Development of the Collectivist‐Sensitive Trait Forgivingness Scale

Mounting evidence suggests that experiences of forgiveness vary across cultures. However, culturally sensitive conceptualizations of forgiveness lack empirical support, in part because psychometrically sound instruments designed to capture unique aspects of forgiveness in non‐Western cultures are rare. For this reason, we developed the Collectivist‐Sensitive Trait Forgivingness Scale (TFS‐CS), which is designed to measure trait forgivingness within societies characterized by a blend of individualistic and collectivistic worldviews. In Study 1 (N = 597), exploratory factor analysis revealed a 16‐item three‐factor structure of third‐party forgiveness, collectivistic forgiveness, and interpersonal resentment among South Africans. In Study 2 (N = 897), the three‐factor model replicated in an independent South African sample. Findings also offered preliminary evidence supporting the construct validity of the TFS‐CS. Overall, these studies support a conceptualization of trait forgivingness with similarities and differences relative to Western models and highlight the importance of appreciating the influence of culture when measuring forgiveness.     

Self‐Disclosure Characteristics and Motivations in Social Media: Extending the Functional Model to Multiple Social Network Sites

The purpose of this study was to examine differences in self‐disclosure goals, privacy concerns, and self‐disclosure characteristics between Facebook and Twitter. These sites were compared in terms of audience representations, based on structural cues that suggest potential audiences for a user. We conceptualized audience representations in 2 ways: based on privacy boundaries that imply bounded versus unbounded audiences, and on network characteristics such as size and diversity for audiences within the boundary. Results revealed that self‐disclosure goals, privacy concerns, and self‐disclosure intimacy were different depending on the privacy boundary. Network characteristics were also important, but effects were moderated by the privacy boundary type, suggesting a complex interplay between the 2 types of audience representations in shaping self‐disclosure in social media.