An effect size index for comparing two independent alpha coefficients

Since Cronbach proposed the α coefficient in 1951, researchers have contributed to the derivation of its sampling distribution and the testing of related statistical hypotheses. Yet, there has been no research on effect size index relevant to coefficient α to our knowledge. Considering the importance of effect size in understanding quantitative research findings, we therefore developed an effect size index Δ for the comparison of two independent αs with equal test length based on the asymptotic distribution of under the assumptions of normality and compound symmetry. Simulations indicated that the index was applicable when the sample size was at least 100. The robustness of the derived index when the required assumptions were violated was also explored. It is suggested that the index should be applicable in most cases of unequal test lengths and could be extended to non‐normally distributed component scores. Moreover, a small simulation was conducted to explore the behaviour of Δ with correlated errors, a frequently studied situation violating the assumption of compound symmetry. The proposed index was found to be robust unless a large number of highly correlated errors were present in the data.     

The antisocial process screening device: an examination of its construct and criterion-related validity

The clinical assessment of psychopathy in adulthood is well established via programmatic research. More recently, psychopathy has been extended to children and adolescents with correlates to maladaptive personality traits, violent behavior, and noncompliance with institutional rules. To screen for adolescent psychopathy, the Antisocial Process Screening Device (APSD) was developed as a 20-item self-report measure of psychopathy. The original validation of the APSD was limited to samples of clinic-referred and community-based children. In extending this research to delinquent populations, the current article uses two separate samples of adolescent offenders incarcerated in a maximum security facility (n = 78) and a local juvenile detention facility (n = 77). As evidence of criterion-related validity, the APSD was compared with two versions of the Psychopathy Checklist that yielded mixed results. Construct validity was examined via a confirmatory factor analysis that provided support for a three-factor model of the APSD.     

从效应量应有的性质看中介效应量的合理性

效应量的作用有两个方面, 一是弥补了统计检验的不足, 二是使得效应有可比性。结合统计显著性和效应量, 才能得出适当的统计结论。效应量应当具有一些基本性质, 包括与测量单位无关、单调性、不受样本容量的影响。国际上流行的中介效应量κ平方就是因为缺乏单调性而引发质疑和研究, 从而被彻底终结了其作为中介效应量的合法性。R平方型中介效应量同样有缺乏单调性的问题。文末讨论了如何报告中介效应量以及有待研究的问题。     

Process assessment: An examination of reinforcer effectiveness and predictive validity

This research describes an assessment procedure based on measures of actual learning. First-grade children (N = 120) learned five sight vocabulary words under feedback only, and were then assigned to one of five conditions where they learned five additional words either under feedback, praise, candy, praise and candy, or reproof. In a sixth condition (reciprocal to the first treatment), students learned the first five works under praise and the second five under feedback only. Although results indicated no differences in learning rate under different reinforcement conditions, learning measures under feedback accounted for approximately 39% of the variance in achievement (r = .62), with combined reinforcement learning measures slightly improving prediction (r = .68). The order of treatments had no effect on learning rate. Implications for this process approach to assessment are discussed. Directions for future research are suggested.     

Social networking profiles: an examination of student attitudes regarding use and appropriateness of content

This study examined students' use of and attitudes toward social networking sites. Significant gender differences were found regarding the type of information posted and whether students were comfortable with employers seeing this information. There were several items that students were likely to post on their sites but were not comfortable with employers seeing. Implications and suggestions for future research are discussed.     

The indirect McCollough effect: An examination of an associative account

It has been found that viewing, for example, a red-and-black vertical grating alternating with a green homogeneous field produces a color aftereffect—a McCollough effect—on a black-and-white vertical grating (i.e., green). Viewing such colored patterns also produces an aftereffect on a noninduced horizontal grating (i.e., pink)—the indirect McCollough effect. Humphrey, Dodwell, and Emerson (1989) argued that the indirect McCollough effect is caused by opponent properties of the visual system that organize the processing of contour and color along contrasting, probably orthogonal, dimensions. Recently, however, their interpretation of the indirect McCollough effect has been challenged by some findings of Eissenberg, Allan, Siegel, and Petrov (1995). These researchers have proposed that the indirect McCollough effect, like the McCollough effect, can be explained by associative principles. The results reported here question crucial aspects of the hypothesis of Eissenberg et al.     

The content validity of juvenile psychopathy: an empirical examination

This study examined the content validity of a juvenile psychopathy measure, the Childhood Psychopathy Scale (CPS; D. R. Lynam, 1997), based on a downward translation of an adult instrument, the Hare Psychopathy Checklist-Revised (PCL-R; R. D. Hare, 1991). The CPS was compared with two other indices of juvenile psychopathy: (a) an index derived from expert ratings and (b) an empirical index based on correlations with adult psychopathy. The 100 items of the Common Language Q-Sort (CLQ; A. Caspi et al., 1992) provided a common metric for the comparison. Psychopathy and personality were assessed at age 13 years with the mother-reported CPS and the CLQ. Psychopathy was assessed at age 24 years with the interviewer-rated Psychopathy Checklist: Screening Version (PCL:SV; S. D. Hart, D. N. Cox, & R. D. Hare, 1995). Data from over 250 participants of the middle sample of the Pittsburgh Youth Study were used to examine these relations. Item content analyses demonstrated considerable overlap among the three indices, indicating that the downward translation utilizes criteria similar to those of experts and the empirically-derived measure. In addition, these indices, even after removing overlapping items, demonstrated considerable convergence, also supporting the content validity of the downward translation. These results suggest that the downward translation method is adequate for understanding the juvenile psychopathy construct.     

The Error-Oriented Motivation Scale: An examination of structural and convergent validity

The current paper describes an analysis of the factor structure and convergent validity of the Error-Oriented Motivation Scale (EOMS). The EOMS is an attempt to validate a context-general and brief measure of error orientations theoretically similar to the Error Orientation Questionnaire (EOQ) written by (Rybowiak, Garst, Frese, and Batinic (1999)). Motivation theory, particularly approach/avoidance motives and goal theory, was chosen as a foundational model for the new EOMS measure, arguing that errors take on properties of goals when they occur by creating a discrepancy, which generates arousal and direction toward a solution. In this article, the factor structure of the EOMS is examined, as well as expected correlations with subscales of the EOQ. The discussion focuses on the advantages of the EOMS measure and suggestions for its further validation and use in applied settings.     

An examination of the construct validity of the Social Anhedonia Scale

Two studies bearing on the construct validity of the Social Anhedonia Scale (SA) are reported. Study 1 reported convergent validity coefficients between the SA scale and numerous measures of traits reflecting a generalized attitude toward others (e.g., social interest and empathy) as well as several traits reflecting maladjustment without a social component as a test of discriminant validity (e.g., pessimism and rigidity). Study 1 found impressive convergent validity for the SA scale and mixed discriminant validity. Study 2 found that the SA scale was related to two dimensions of striving for security, one of which is strongly influenced by negative interpersonal tendencies, suggesting support for the validity of the SA scale. Implications for the measurement and concept of social anhedonia are presented.     

An examination of the predictive criterion-related validity of professional certification tests

This study examines the predictive criterion-related validity of a series of professional certification tests for water and wastewater management operators. Certification test data were obtained on 164 operators holding one of three jobs in water or wastewater management facilities. The certification test scores were broken down into four component scores and a total score. Criterion data consisted of performance evaluations obtained from the operator's supervisor, a self-rating from the operator, or both. Test scores were correlated with the job performance evaluations. The results indicated that scores on the certification tests were not related to rated job performance by job type, job level, source of performance evaluation, or component of job performance. The findings are discussed in the context of establishing an appropriate criterion against which to validate certification tests, and the practical problems of doing so in an applied setting.     

An index of item validity providing a correction for chance success

The KG Index described below is proposed for evaluation as one approach to the problem of providing an index giving comparable values for items (1) of equal discriminative power at all levels of difficulty and (2) of different numbers of alternative responses.     

An examination of the factor structure and convergent and discriminant validity of the SCL-90R in an outpatient clinic population

This study investigated the factor structure of the SCL-90R using an outpatient psychology clinic population. Six relatively homogeneous and stable factors were identified: Depression, Somatization, Anger-Hostility, Paranoid-Psychoticism, Phobic Anxiety, and Obsessive-Compulsive. However, a principal component analysis revealed that the first factor accounted for a large percentage of the variance, suggesting that this instrument measures a general dimension of psychopathology. In addition, significant correlations between the SCL-90R symptom dimensions and both the BDI and several MMPI scales were found. The results of the study lead to questions regarding the utility of the SCL-90R.     

Large-sample confidence intervals for validity and reliability coefficients

Large-sample confidence intervals (CI) for reliability, validity, and unattenuated validity are presented. The CI for unattenuated validity is based on the Bonferroni inequality, which relies on one CI for test-retest reliability and one for validity. Covered are four reliability-validity situations: (a) both estimates were from random samples; (b) reliability was from a random sample but validity was from a selected sample; (c) validity was from a random sample but reliability was from a selected sample; and (d) both estimates were from selected samples. All CIs were evaluated by using a simulation. CIs on reliability, validity, or unattenuated validity are accurate as long as selection ratio is at least 20% and selected sample size is 100 or larger. When selection ratio is less than 20%, estimators tend to underestimate their parameters.     

The effect of color intensity and appropriateness on color-induced odor enhancement

Coloring solutions has been shown to increase perceived odor intensity. In Experiment 1, subjects rated the odor intensity of red strawberry and green mint solutions that had four levels of color intensity. Ratings of strawberry odor peaked at the medium color intensity and ratings of mint odor increased monotonically with color intensity. Thus, color-induced odor enhancement can increase with increasing color intensity but need not. Experiment 2 found that the color intensities producing maximum odor enhancement in Experiment 1 are not always the ones perceived as most appropriate for the odorants. Using different odors, Experiment 3 found that color intensity has some influence on the strength of the color-induced odor enhancement and color appropriateness has little. The presence or absence of color in the solution seems to be the most important variable.     

An expanded childhood anxiety sensitivity index: its factor structure, reliability, and validity in a non-clinical adolescent sample

Anxiety sensitivity refers to the fear of anxiety-related bodily sensations that are interpreted as having potentially harmful somatic, psychological, or social consequences. The current study examined the factor analytic structure of anxiety sensitivity in a large sample of normal adolescents (N=518) using the revised childhood anxiety sensitivity index (CASI-R). Confirmatory factor analysis indicated that anxiety sensitivity as measured by the CASI-R can best be conceptualised as a hierarchical construct with four lower-order factors loading on a single higher-order factor. The lower-order factors were 'fear of cardiovascular symptoms', 'fear of publicly observable anxiety reactions', 'fear of cognitive dyscontrol', and 'fear of respiratory symptoms'. An additional aim of the present study was to investigate the psychometric properties of the CASI-R. Results showed the CASI-R to be a reliable scale in terms of internal consistency. Furthermore, CASI-R scores were substantially related to levels of anxiety sensitivity as measured by the original index, trait anxiety, symptoms of anxiety disorders, in particular 'panic disorder and agoraphobia', and depression. Finally, some evidence was found for the validity of the CASI-R factor scores. That is, all factors convincingly loaded on symptoms of 'panic disorder and agoraphobia', whereas the factor 'fear of publicly observable anxiety reactions' was also strongly associated with symptoms of 'social phobia'.     

Overt honesty measures predicting admissions: an index of validity or reliability

Honesty questionnaires are often validated against a paper-and-pencil criterion in which respondents are asked to admit to past incidents of dishonesty. However, substantial overlap in the methods of assessment and in item content between predictor and criterion suggest that it is not validity that is being assessed, but rather a form of reliability. In this study, the relations between two overt measures of honesty (the Phase II Profile and the Workplace Productivity Questionnaire) and an admissions criterion were evaluated. The results suggest that the strong correlations between the Phase II Profile and the admissions criterion (r = -.67) and between scores on the Workplace Productivity Questionnaire and the admissions criterion (r = -.62) may be largely due to overlapping item content.     

An examination of an enhancing effect of music on attentional abilities in older persons with mild cognitive impairment

While the effect of listening to music on cognitive abilities is highly debated, studies reporting an enhancing effect of music in elderly populations appear to be more consistent. In this study, the effects of listening to music on attention in groups of cognitively normal older adults and those with mild cognitive impairment were considered. Participants were exposed to both a music and silence condition, and after each condition performed Digit Span and Coding tasks which require attention for maximal performance. The hypothesis that listening to music, compared to a silence condition, enhances performance was not supported for either group. Various explanations for these findings are considered.     

The Agoraphobia Scale: an evaluation of its reliability and validity

This article presents the Agoraphobia Scale (AS), and evidence for its reliability, validity, and sensitivity to change after treatment. The scale consists of 20 items depicting various typical agoraphobic situations, which are rated for anxiety/discomfort (0-4) and avoidance (0-2). The results show that AS has high internal consistency. Regarding concurrent validity it correlated significantly with other self-reported measures of agoraphobia (Mobility Inventory and Fear Questionnaire). The scale's predictive validity was shown as it correlated with avoidance behavior and self-rated anxiety during both an individualized and a standardized behavioral test of agoraphobia. The AS also discriminated between an agoraphobic sample and a normal sample, and a sample of simple phobia patients. Finally, it was sensitive to changes after behavioral treatment. The AS is useful both as a state, and as an outcome self-report measure of agoraphobia.     

An examination of gender differences in the construct validity of the silencing the self scale

Jack’s (1991) theory of self-silencing was originally designed to explain higher rates of depression in women in comparison to men. However, research finding that men score equal or even higher than women on measures of self-silencing has lead theorists to speculate that self-silencing tendencies may be driven by different motivations and have different consequences for women versus men (Jack & Ali, 2010). Using a sample of 247 college students, we examined gender differences in the construct validity of the Silencing the Self Scale (STSS; Jack & Dill, 1992). We hypothesized that women would score higher on the Externalized Self-Perception subscale, but not the other three subscales. Gender differences in the relationship between the STSS subscales and theoretically-relevant constructs were also explored. The results indicated that women on average scored higher than men on the Externalized Self-Perception subscale, whereas men scored higher on the Care as Self-Sacrifice subscale. Further, there was a significant Gender × Care as Self-Sacrifice subscale interaction in the prediction of depression, such that this subscale was negatively correlated to depression in men, and uncorrelated in women. These results clarify how self-silencing might translate into different mental health outcomes for women and men.