Internal consistency reliability of the self-report antisocial process screening device

The self-report version of the Antisocial Process Screening Device (APSD) has become a popular measure for assessing psychopathic features in justice-involved adolescents. However, the internal consistency reliability of its component scales (Narcissism, Callous-Unemotional, and Impulsivity) has been questioned in several studies. This study evaluates the internal consistency reliability of the self-report APSD by examining various indices (e.g., Cronbach's alpha, mean interitem correlation) across 11 studies. Whereas the Narcissism and Impulsivity scales display moderate to good reliability, internal consistency indices for the Callous-Unemotional scale are consistently poor across studies. Suggestions are made for revisions to this scale that would substantially improve its internal consistency reliability.     

Internal Consistency and Power When Comparing Total Scores from Two Groups

Researchers now know that when theoretical reliability increases, power can increase, decrease, or stay the same. However, no analytic research has examined the relationship of power to the most commonly used type of reliability—internal consistency—and the most commonly used measures of internal consistency, coefficient alpha and ICC(A,k). We examine the relationship between the power of independent samples t tests and internal consistency. We explicate the mathematical model upon which researchers usually calculate internal consistency, one in which total scores are calculated as the sum of observed scores on K measures. Using this model, we derive a new formula for effect size to show that power and internal consistency are influenced by many of the same parameters but not always in the same direction. Changing an experiment in one way (e.g., lengthening the measure) is likely to influence multiple parameters simultaneously; thus, there are no simple relationships between such changes and internal consistency or power. If researchers revise measures to increase internal consistency, this might not increase power. To increase power, researchers should increase sample size, select measures that assess areas where group differences are largest, and use more powerful statistical procedures (e.g., ANCOVA).     

Issues of Reliability in Measuring Intimate Partner Violence during Courtship

The current paper focuses on problems in conceptualizing and establishing reliability when using self-administered measures of intimate partner violence to assess dating violence. Establishing reliability is an important step in the development of dating violence assessment instruments. However, the nature of dating violence can make it difficult to establish reliability. Most notably, measures of intimate partner violence in courtship yield data that are positively skewed, with almost no one reporting high levels of violence. This could have implications for the calculation of several forms of reliability that assume normality (e.g., Pearson correlations, intraclass correlations). In addition, there are other characteristics of dating violence that could impact reliability. For example, partner violence perpetrators do not necessarily use multiple acts (internal consistency reliability) or repeat specific acts (test-retest reliability). And, gender differences in the perception of partner violence may influence intra-couple reliability in heterosexual couples. Finally, statistical interdependence within couples makes current intra-couple reliability assessment suspect. Research is reviewed and recommendations are made concerning the establishment of test-retest reliability, intra-couple reliability, and internal consistency reliability for measures of dating violence.     

Reliability and plasticity of response inhibition and interference control

This study investigated the internal reliability, temporal stability and plasticity of commonly used measures of inhibition-related functions. Stop-signal, go/no-go, antisaccade, Simon, Eriksen flanker, Stroop and Continuous Performance tasks were administered twice to 23 healthy participants over a period of approximately 11 weeks in order to assess test–retest correlations, internal consistency (Cronbach’s alpha), and systematic between as well as within session performance changes. Most of the inhibition-related measures showed good test–retest reliabilities and internal consistencies, with the exception of the stop-signal reaction time measure, which showed poor reliability. Generally no systematic performance changes were observed across the two assessments with the exception of four variables of the Eriksen flanker, Simon and Stroop task which showed reduced variability of reaction time and an improvement in the response time for incongruent trials at second assessment. Predominantly stable performance within one test session was shown for most measures. Overall, these results are informative for studies with designs requiring temporally stable parameters e.g. genetic or longitudinal treatment studies.     

A psychometric evaluation of the Harvey Imposter Phenomenon Scale

In this article, we present psychometric properties on the 14-item Harvey Imposter Phenomenon (IP) Scale (Harvey, 1981) among a sample of U.S. adolescents (N = 136). Item analyses, internal consistency reliability, and principal components analysis (PCA) were computed to assess the reliability and factor structure of the IP scores. Although the internal consistency reliability (e.g., Cronbach's alpha) was reasonably acceptable (alpha =.70), the item analyses and subsequent PCA suggest concern for the content homogeneity and factor structure of composite scores obtained from the 14-item measure. The results of this study suggest that the indiscriminate use of the Harvey IP Scale is cautioned relative to the insufficient psychometric properties.     

The belief in a just world: an examination of reliability estimates across three measures

In this study, we utilized reliability generalization procedures to examine internal consistency estimates across 3 scales measuring the belief in a just world. The distribution of reliability estimates for the measures suggest low to moderate ranges of internal consistency reliability coefficients. The Global Belief in a Just World Scale (Lipkus, 1991) produced the highest average reliability score (alpha = .81) compared to the Just World Scale (Rubin & Peplau, 1973; alpha = .64) and the Just World Scale Revised (Rubin & Peplau, 1975; alpha = .68).     

Factor analysis to examine psychometric properties of family functioning measures

To assess the relationship of family functioning to problem behaviors and alcohol and drug use among youth, researchers must test the effects of interventions using suitably constructed and psychometrically sound scales. This study evaluated whether originally calculated coefficients alpha underestimate the reliability of the family functioning measures given. Through exploratory factor analysis, estimates of alternative internal consistency reliability which might improve the estimate of reliability were examined. Responses of 755 adults from Strengthening Multi-ethnic Families and Communities were analyzed. Coefficients alpha for two scales were modest (alpha = .68 and alpha = .75), and factor analysis indicated that the scales were multidimensional. After exploratory factor analysis, the reassessment of reliability based on the extracted factors indicated an overall increase in the coefficients alpha.     

Coping strategies questionnaire (CSQ): Reliability of the swedish version of the CSQ

Abstract

We present a study with the aim of investigating the internal consistency and reliability of a Swedish version of the Coping Strategies Questionnaire (CSQ). The study group consisted of 282 subjects suffering from long term back pain. Internal consistency was investigated by calculating alpha coefficients and test-retest reliability was investigated by a correlation analysis with two different test-retest intervals. The results reveal that the internal consistency of the Swedish CSQ is high (alpha range between 0.7 and 0.8) and consistent with the American version. The test-retest reliability was not equally satisfactory (correlation ranged between 0.4 and 0.9) but the results support the usefulness of the CSQ as a tool in the clinical assessment of pain coping strategies. The test-retest result indicate the need for further research. In conclusion, the results from the present study are encouraging and support the usefulness of the instrument but to improve the utility and application of the Swedish version of the CSQ further research is warranted.     

Evaluating a modified exercise self-efficacy scale for college-age women

This pilot study examined test-retest and internal consistency reliabilities of original and modified formats of the Exercise Self-efficacy Scale in college-age women. 30 completed original and modified versions of the scale. Data from both tests, administered 1 wk. apart, were analyzed using the intraclass correlation coefficient (ICC) to assess test-retest reliability and Cronbach coefficient alpha for internal consistency. Scores for both versions correlated .96. Cronbach coefficients alpha for the original scale were .96 for Time 1 and .98 for Time 2. Cronbach coefficients alpha for the revised scale were .95 for Time 1 and .98 for Time 2. Test-retest reliability and internal consistency remained consistently high for both versions of the scales within this sample. Implications for use of this scale and recommendations for research are given.     

测验信度估计：从α系数到内部一致性信度

沿用经典的测验信度定义, 简介了信度与a 系数的关系以及a系数的局限。为了推荐替代a系数的信度估计方法, 深入讨论了与a 系数关系密切的同质性信度和内部一致性信度。在很一般的条件下, 证明了a 系数和同质性信度都不超过内部一致性信度, 后者不超过测验信度, 说明内部一致性信度比较接近测验信度。总结出一个测验信度分析流程, 说明什么情况下a 系数还有参考价值; 什么情况下a 系数不再适用, 应当使用内部一致性信度(文献上也常称为合成信度)。提供了计算同质性信度和内部一致性信度的计算程序, 一般的应用工作者可以直接套用。     

Cross Racial Identity Scale (CRIS) scores: stability and relationships with psychological adjustment

We examined the structural validity, internal consistency (alpha and omega), and test-retest reliability of scores on the Cross Racial Identity Scale (CRIS; Vandiver et al., 2000 ; Worrell, Vandiver, & Cross, 2004 ), as well as the relationship between CRIS scores and several variables related to psychological adjustment. Participants consisted of several groups of African American college students (34 ≤ n ≤ 340) attending a predominantly White university in a Western state. Confirmatory factor analyses indicated an acceptable fit of the data to the theoretical model, and alpha and omega coefficients indicate that CRIS scores have moderate to high internal consistency. CRIS scores also demonstrated stability over periods between 2 and 20 months in ranges that suggest long-term stability of racial attitudes. As predicted by the expanded nigrescence model (Cross & Vandiver, 2001 ), only self-hatred attitudes had consistent, meaningful relationships with psychological adjustment.     

中文版过剩适应量表在大学生群体中的效度和信度

为修订中文版过剩适应量表(OAS-C),并检验其在中国大学生群体中的信效度,抽取589名大学生(样本1),278名大学生(样本2)和174名大学生(样本3)进行施测.效度分析结果表明,OAS-C为二因子结构,包括外部适应过剩和内部适应匮乏.该二因子模型拟合良好,且与各效标间呈显著正相关.信度分析结果显示,内部一致性系数...     

Development and preliminary validation of a semi-structured interview for the screening of law enforcement candidates

A standard practice among most law enforcement agencies is to include psychological screening in the selection of job candidates. Although the use of personality tests in predicting job performance of law enforcement officers has received empirical support (e.g., the Minnesota Multiphasic Personality Inventory, the Inwald Personality Inventory), there is a conspicuous absence of data regarding the reliability and validity of interview procedures. The present study represents the first step in the development of a standardized, semi-structured interview for use in the screening of law enforcement personnel. This interview, the Law Enforcement Candidate Interview, was constructed using content areas drawn from measures used in the screening of law enforcement personnel (e.g., Inwald Personality Inventory) and assessment of personality functioning (Structured Clinical Interview for DSM-III-R Personality Disorders). This interview was then administered by two doctoral students to a group of law enforcement academy cadets (n=34). A modest degree of inter-rater reliability was achieved, although internal consistency was somewhat lacking. Interview scores were correlated with measures of academy performance (final grade-point average and peer and supervisor ranking of academy performance). Results of these analyses suggest the LECI is a modest predictor of academy performance. Potential uses of the LECI and future research indications are outlined.     

The Italian Five Facet Mindfulness Questionnaire: A Contribution to its Validity and Reliability

Mindfulness refers to the ability to be aware of and attentive to internal and external events, without becoming overwhelmed or feeling the need to react automatically. Recent studies indicate that this ability is associated with the following factors: mental health, psychological functioning, and stress reduction. Although questionnaires have been developed to measure mindfulness, so far these have not been adapted for use within the Italian population. Thus, the aim of our studies was to investigate the reliability and validity of an Italian version of the Five Facet Mindfulness Questionnaire (FFMQ; Baer et al. Assessment, 13, 27–45, 2006). The FFMQ is a self-report measure that is based on a five-facet model (i.e., Observe, Describe, Act with Awareness, Nonjudge, and Nonreact). In the first study, internal consistency, factor structure, and concurrent validity analyses were carried out on a sample of 559 volunteers. The second study entailed test–retest analyses on a different sample of 43 students. The results showed that the Italian FFMQ: (a) has a similar factor structure to the original English version; (b) has good to excellent internal consistency as a whole (alpha?=?0.86) with sub-scale consistency ranging from 0.65 to 0.81, and test–retest stability for the total score being 0.71; (c) has good concurrent validity as demonstrated by significant correlations between the FFMQ scores and a number of self-report measures related to mindfulness. Our findings support both the adaptation of the FFMQ to the Italian context as well as the conceptualization of mindfulness in multi-faceted terms.     

Coefficient alpha and related internal consistency reliability coefficients

The author studied the conditions under which coefficient alpha and 10 related internal consistency reliability coefficients underestimate the reliability of a measure. Simulated data showed that alpha, though reasonably robust when computed on n components in moderately heterogeneous data, can under certain conditions seriously underestimate the reliability of a measure. Consequently, alpha, when used in corrections for attenuation, can result in nontrivial overestimation of the corrected correlation. Most of the coefficients studied, including lambda 2, did not improve the estimate to any great extent when the data were heterogeneous. The exceptions were stratified alpha and maximal reliability, which performed well when the components were grouped into two subsets, each measuring a different factor, and maximized lambda 4, which provided the most consistently accurate estimate of the reliability in all simulations studied.     

Assessing the Reliability of Situational Judgment Tests Used in High‐Stakes Situations

Assessing reliability of situational judgment tests (SJTs) in high‐stakes situations is problematic with reliability inappropriately measured by Cronbach's alpha when test items are heterogeneous. We computed the corrected, weighted mean alpha from 56 alpha coefficients, which produced a value of α = .46 and reviewed appropriate types of reliability to use with SJTs. In the current longitudinal study, SJT test–retest reliability was r = .82, compared with internal consistency, α = .46, and stratified alpha, α = .45 at Time 1 and α = .52 and stratified α = .51 at Time 2. We used a student sample (Time 1: n = 185; Time 2: n = 132) with items from a credentialing exam with ‘should do’ instructions. The SJT correlated significantly with cognitive ability, r = .30, and agreeableness, r = .24. In Study 2, we assessed test–retest reliability with Human Resource professionals (Time 1: n = 94; Time 2: n = 32) who had been recently credentialed and who participated in a pilot test of new SJT items with ‘most likely/least likely do’ response options. The SJT test–retest reliability was r = .66 compared with internal consistency, α = .43 and stratified α = .47 at Time 1 and α = .61 and stratified α = .67 at Time 2. We discuss the theoretical implications of the Study 1 results as well as the practical implications for use of SJTs in credentialing examinations.     

Reliability in Dream Research: A Methodological Note

The coefficients of internal consistency and retest reliability had been rarely investigated within the methodology of dream content analysis. Analyzing a dream series of elderly, healthy persons obtained from weekly telephone interviews, the internal consistency of a series of 20 dreams and retests after 4 or 22 weeks, respectively, had been computed. The findings indicate that dream recall and dream length are quite stable, but dream characteristics such as bizarreness and emotional tone underlie large intraindividual fluctuations. In order to obtain reliable measures for these variables which will be important for correlational studies, including waking-life trait measures, one has to obtain as many dreams as possible (about 20) in a very short time period. Further research is needed to extend the present findings to diary dreams and laboratory dreams.     

Psychometric Properties of the WHOQOL-BREF Questionnaire among Disabled Students in Malaysian Higher Learning Institutions

This study aims to evaluate reliability, internal consistency and dimensional structure of the World Health Organization Quality of Life (WHOQOL-BREF) among disabled students. The study covers 127 disabled students studying in public universities, 26 in private universities, 109 in polytechnic and 19 studying in community colleges. The World Health Organization Quality of Life Abbreviation (WHOQOL-BREF) has been used to assess QOL. The reliability and validity of the questionnaire were evaluated by Cronbach’s alpha (α) and Pearson’s correlation coefficients. Alpha scores greater than or equal to 0.6 are considered to denote acceptable and adequate internal consistency. Correlation matrix also showed satisfactory results in all domains. Factor analysis was carried out using the principal components method with varimax rotation to examine the dimensional structure of the questionnaire. This study has provided some preliminary evidence of the reliability and validity of the WHOQOL-BREF to be used for evaluating quality of life among disabled students.     

Reliability and validity of the Cognitive Slippage Scale in two populations.

Analyses of responses from a clinical sample of 120 patients (primarily schizophrenics) and from 158 college students to the Cognitive Slippage Scale, a scale designed by Miers and Raulin to identify speech deficits and confused thinking in schizophrenic and schizotypal personality disorders showed high internal reliability; Cronbach's coefficients alpha were .89 and .86 in the clinical and college student samples, respectively. The mean scale scores significantly differentiated the two samples. Also, change scores over 4 wk. showed adequate stability for both samples. Item analysis indicated Items 11, 20, 21, and 28 may not reliably discriminate between schizophrenic and college student samples. Over-all, these preliminary results are consistent with the reliability and validity of the scale.