Investigating measurement equivalence of visual analogue scales and Likert-type scales in Internet-based personality questionnaires

Visual analogue scales (VASs) have shown superior measurement qualities in comparison to traditional Likert-type response scales in previous studies. The present study expands the comparison of response scales to properties of Internet-based personality scales in a within-subjects design. A sample of 879 participants filled out an online questionnaire measuring Conscientiousness, Excitement Seeking, and Narcissism. The questionnaire contained all instruments in both answer scale versions in a counterbalanced design. Results show comparable reliabilities, means, and SDs for the VAS versions of the original scales, in comparison to Likert-type scales. To assess the validity of the measurements, age and gender were used as criteria, because all three constructs have shown non-zero correlations with age and gender in previous research. Both response scales showed a high overlap and the proposed relationships with age and gender. The associations were largely identical, with the exception of an increase in explained variance when predicting age from the VAS version of Excitement Seeking (B₁₀ =1318.95, ΔR ² =.025). VASs showed similar properties to Likert-type response scales in most cases.     

Intimacy of disclosure topics and sex differences in self-disclosure

A 25-item self-disclosure questionnaire with items scaled for intimacy level was given to 32 male and 32 female college students. It was hypothesized that males would report disclosing less than females concerning intimate topics while there would be no sex differences in disclosure on nonintimate topics. The hypothesis was supported by the data. The results demonstrate that scoring self-disclosure scales by summing across all items can mask sex differences in disclosure patterns. Results are interpreted as supporting the assertions of several theorists that males reveal less personal information about themselves than do females.     

Comparison of classical and modern methods for measuring and correcting for acquiescence

Likert-type self-report scales are frequently used in large-scale educational assessment of social-emotional skills. Self-report scales rely on the assumption that their items elicit information only about the trait they are supposed to measure. However, different response biases may threaten this assumption. Specifically, in children, the response style of acquiescence is an important source of systematic error. Balanced scales, including an equal number of positively and negatively keyed items, have been proposed as a solution to control for acquiescence, but the reasons why this design feature worked from the perspective of modern psychometric models have been underexplored. Three methods for controlling for acquiescence are compared: classical method by partialling out the mean; an item response theory method to measure differential person functioning (DPF); and multidimensional item response theory (MIRT) with random intercept. Comparative analyses are conducted on simulated ratings and on self-ratings provided by 40,649 students (aged 11–18) on a fully balanced 30-item scale assessing conscientious self-management. Acquiescence bias was explained as DPF and it was demonstrated that: the acquiescence index is highly related to DPF; balanced scales produce scores controlled for DPF; and MIRT factor scores are highly related to scores controlled for DPF and the random intercept is highly related to DPF.     

COMPARISON OF FEMINIST VERSUS PSYCHOANALYTIC/DYNAMIC AND OTHER THERAPISTS ON SELF-DISCLOSURE

The Feminist Self-Disclosure Inventory (FSDI) was developed to assess principles of therapist self-disclosure as described in the feminist therapy literature. This 18-item Likert-type scale was completed in a mailed survey by women psychotherapists (i.e., 41 self-identified feminist therapists, 34 psychoanalytic/dynamic, and 68 other therapists). Results indicated that the FSDI was comprised of five factors [i.e., Therapist Background (TB), Promotes Liberatory Feelings (PLF), Promotes Egalitarianism (PE), Therapist Availability (TA), and Empowering Client (EC)], and that the overall scale had excellent internal consistency and very satisfactory test-retest reliability. Also, feminist therapists endorsed FSDI factors significantly more than psychoanalytic/dynamic and other therapists with the greatest differences on items reflecting overall use of self-disclosure, disclosing sexual orientation, using disclosure to create egalitarianism, and encouraging the client's choice of a role model in the therapeutic relationship.     

Some thoughts on Likert-type scales

Many articles published in the International Journal of Clinical and Health Psychology describe the results obtained in studies that have used Likert-type scales. Such scales have advantages and disadvantages and in this article I comment on some of these. In particular I comment on the effects of using scales that start with the positive end and finish with the negative one, and on those that present high ratings on the left - as opposed to the right. I also consider that scores on negatively-worded items differ from those obtained on positively-worded ones and that it is not, therefore, appropriate to simply reverse the scores obtained on such items. Finally, I note that some items on these scales present more than one issue and it is hard to know whether when responding to these items, respondents are responding to all of the item or just one part.     

Measurement of depression: Comparison between self-reports and clinical assessments of depressed outpatients

Agreement among several depression scales was investigated as regard the relative influences of administration mode (self-rating or clinical rating) and scale content. The Beck Depression Inventory (BDI), the Self-Rating Depression Scale (SDS), the Hamilton Rating Scale for Depression (HRSD), and three corresponding scales with identical structure and content but the alternative administration mode were administered to 47 outpatients with diagnoses of DSM-III major depression disorders. Correlations between the total scores and the degrees of association between corresponding items of different scales were calculated. The results suggest that differences in content contribute more to inter-scale discrepancy than differences in administration mode. The implications for the evaluation of outpatients with major depression are discussed.     

THE IMPACT OF QUESTIONNAIRE ADMINISTRATION MODE ON RESPONSE RATE AND REPORTING OF CONSENSUAL AND NONCONSENSUAL SEXUAL BEHAVIOR

Computer-administered self-interviewing (CASI) has been advocated as a way of overcoming underreporting of sensitive sexual behaviors. The present study compared reports of consensual and nonconsensual sexual behaviors (including childhood and adulthood sexual victimization) among women using CASI ( n = 1014) versus a self-administered questionnaire (SAQ; n = 318). Women, ages 18–30 years old, were recruited through random-digit dialing in the community. Response rates were significantly higher for the SAQ, which was sent and returned by mail (87.6% of those eligible), than for the CASI, which involved in-person assessment at a central location (61.4%). There were a few modest differences in reports of consensual or nonconsensual sexual behavior according to mode of administration, suggesting greater willingness to report sexual experiences among the SAQ sample. Mailed SAQ following telephone contact and with financial incentive may be a cost-effective way of obtaining data on sexual behavior from a community sample.     

MULTIPLE-CHOICE AND CONSTRUCTED RESPONSE TESTS OF ABILITY: RACE-BASED SUBGROUP PERFORMANCE DIFFERENCES ON ALTERNATIVE PAPER-AND-PENCIL TEST FORMATS

We present an example of an innovative constructed response test format–a write-in/mark-in paper-and-pencil test–as an alternative to the traditional multiple-choice paper-and-pencil test, with the potential for reducing subgroup differences. We present subgroup differences data on these 2 paper-and-pencil test formats on an operational promotional exam in a sample of African American and White firefighters. The tests were designed to measure the same content domain. Using within-subjects data that compared the performance of 13 African American and 14 White fire captains, and between-subjects data that compared the performance of 21 African American and 49 White fire captains, several results were in the predicted direction such that subgroup differences were reduced on the constructed response test. However, these results did not reach statistical significance. Therefore, the study points to the need for additional research to further evaluate the promise of the constructed response test format.     

Computer-assisted self-interviews: A cost effectiveness analysis

Computer-assisted self-interview (CASI) questionnaires are being used with increased frequency to deliver surveys that previously were administered via self-administeredpaper-and-pencil questionnaires (SAQs).Although CASI may offer a number of advantages, an important consideration for researchers is the assessment modality's immediate and long-term costs. To facilitate researchers' choice between CASI and SAQ, this article provides theoretical cost models with specific parameters for comparing the costs for each assessment type. Utilizing these cost models, this study compared the cost effectiveness in a health behavior study in which both CASI (n = 100) and SAQ (n = 100) questionnaires were administered. Given the high initial costs, CASI was found to be less cost effective than SAQ for a single study. However, for studies with large sample sizes or when CASI software is to be used for multiple studies, CASI would be more cost effective and should be the assessment mode of choice.     

网络测验和纸笔测验的测量不变性研究：以生活满意度量表为例

以生活满意度量表为例,运用实证性因素分析,考察在中国文化下网络测验和传统纸笔测验之间的测量不变性。结果显示,网络测验和纸笔测验之间存在弱不变性,即网络测验和纸笔测验有着相同的测量单位;但网络测验和纸笔测验只存在部分的强不变性和部分的严格不变性,测验实施环境对结果的影响不可忽视。该研究表明,恰当设计的网络测验是可靠的,同时还提示,当一个测验在不同情境下运用时,检验测量不变性十分必要     

A methodological note on ordered Q-Sort ratings

Patterns of ratings using the Q-Sort method and the Likert-type method are compared. Ordering effects are found in Q-Sort ratings that are not present in Likert-type ratings. Specifically, item order is related to both item variance and item placement, such that items appearing near the end of the Q-Sort have less variance and more central placement. This finding is verified across three measures in several datasets spanning nearly 20 years of research. Such item order effects appear to attenuate average absolute relationships (covariances and correlations) between items appearing near the end of the Q-Sort and other measures. Randomization of items may be (in some situations) a viable course of action to minimize these effects at a sample level.     

Comparing the CASI-4R and the PGBI-10 M for Differentiating Bipolar Spectrum Disorders from Other Outpatient Diagnoses in Youth

We compared 2 rating scales with different manic symptom items on diagnostic accuracy for detecting pediatric bipolar spectrum disorder (BPSDs) in outpatient mental health clinics. Participants were 681 parents/guardians of eligible children (465 male, mean age = 9.34) who completed the Parent General Behavior Inventory-10-item Mania (PGBI-10 M) and mania subscale of the Child and Adolescent Symptom Inventory-Revised (CASI-4R). Diagnoses were based on KSADS interviews with parent and youth. Receiver operating characteristic (ROC) analyses and diagnostic likelihood ratios (DLRs) determined discriminative validity and provided clinical utility, respectively. Logistic regressions tested for incremental validity in the CASI-4R mania subscale and PGBI-10 M in predicting youth BPSD status above and beyond demographic and common diagnostic comorbidities. Both CASI-4R and PGBI-10 M scales significantly distinguished BPSD (N = 160) from other disorders (CASI-4R: Area under curve (AUC) = .80, p < 0.0005; PGBI-10 M: AUC = 0.79, p < 0.0005) even though scale items differed. Both scales performed equally well in differentiating BPSDs (Venkatraman test p > 0.05). Diagnostic likelihood ratios indicated low scores on either scale (CASI: 0–5; PGBI-10 M: 0–6) cut BPSD odds to 1/5 of those with high scores (CASI DLR- = 0.17; PGBI-10 M DLR- = 0.18). High scores on either scale (CASI: 14+; PGBI-10 M: 20+) increased BPSD odds about fourfold (CASI DLR+ = 4.53; PGBI-10 M DLR+ = 3.97). Logistic regressions indicated the CASI-4R mania subscale and PGBI-10 M each provided incremental validity in predicting youth BPSD status. The CASI-4R is at least as valid as the PGBI-10 M to help identify BPSDs, and can be considered as part of an assessment battery to screen for pediatric BPSDs.     

Activating and Suppressing Hostile and Benevolent Racism: Evidence for Comparative Media Stereotyping

This research examines the role of media literacy training and counter-stereotypical news stories in prejudice reduction. Research participants read either stereotypical or counter-stereotypical news stories after exposure to a media literacy video or a control video. After this, they completed a paper-and-pencil questionnaire that included Likert-type scales and feeling-thermometer ratings about their feelings toward African Americans, Asian-Indians, and Caucasian-Americans. The findings reveal that hostile prejudice is more likely to be expressed toward African-Americans and benevolent prejudice is more likely to be expressed toward Asian-Indians. As predicted, counter-stereotypical news stories as compared to stereotypical news stories decrease prejudice toward Asian-Indians. Contrary to expectations, the media literacy video seems to prime prejudices rather than suppress them. Interestingly, news stories about Asian-Indians increase hostility toward African-Americans. These comparative stereotyping are explained using modern racist beliefs and model minority stereotypes.     

Childhood anxiety sensitivity index factors predict unique variance in DSM-IV anxiety disorder symptoms

Anxiety sensitivity (AS) is an established cognitive risk factor for anxiety disorders. In children and adolescents, AS is usually measured with the Childhood Anxiety Sensitivity Index (CASI). Factor analytic studies suggest that the CASI is comprised of 3 lower-order factors pertaining to Physical, Psychological and Social Concerns. There has been little research on the validity of these lower-order factors. We examined the concurrent and incremental validity of the CASI and its lower-order factors in a non-clinical sample of 349 children and adolescents. CASI scores predicted symptoms of DSM-IV anxiety disorder subtypes as measured by the Spence Children's Anxiety Scale (SCAS) after accounting for variance due to State-Trait Anxiety Inventory scores. CASI Physical Concerns scores incrementally predicted scores on each of the SCAS scales, whereas scores on the Social and Psychological Concerns subscales incrementally predicted scores on conceptually related symptom scales (e.g. CASI Social Concerns scores predicted Social Phobia symptoms). Overall, this study demonstrates that there is added value in measuring AS factors in children and adolescents.     

Scale construction for measuring adolescent boys' and girls' attitudes,beliefs, perception of control,and intention to engage in leisure-time physical activity

Using the Theory of Planned Behavior as a framework, the Attitude to Leisure-time Physical Activity, Expectations of Others, Perceived Control, and Intention of Engage in Leisure-time Physical Activity scales were developed for use among high school students. The study population included 20 boys and 68 girls 13 to 17 years of age (for boys, M = 15.1 yr., SD = 1.0; for girls, M = 15.0 yr., SD = 1.1). Generation of items and the establishment of content validity were performed by professionals in exercise physiology, physical education, and clinical psychology. Each scale item was phrased in a Likert-type format. Both unipolar and bipolar scales with seven response choices were developed. Following the pilot testing and subsequent revisions, 32 items were retained in the Attitude to Leisure-time Physical Activity scale, 10 items were retained in the Expectations of Others scale, 3 items were retained in the Perceived Control Scale, and 24 items were retained in the Intention to Engage in Leisure-time Physical Activity scale. Coefficients indicated adequate stability and internal consistency with alpha ranging from .81 to .96. Studies of validities are underway, after which scales would be made available to those interested in intervention techniques for promoting positive attitudes toward physical fitness, perception of control over engaging in leisure-time physical activities, and good intentions to engage in leisure-time physical activities. The present results are encouraging.     

Childhood Anxiety Sensitivity Index Factors Predict Unique Variance in DSM‐IV Anxiety Disorder Symptoms

Anxiety sensitivity (AS) is an established cognitive risk factor for anxiety disorders. In children and adolescents, AS is usually measured with the Childhood Anxiety Sensitivity Index (CASI). Factor analytic studies suggest that the CASI is comprised of 3 lower‐order factors pertaining to Physical, Psychological and Social Concerns. There has been little research on the validity of these lower‐order factors. We examined the concurrent and incremental validity of the CASI and its lower‐order factors in a non‐clinical sample of 349 children and adolescents. CASI scores predicted symptoms of DSM‐IV anxiety disorder subtypes as measured by the Spence Children's Anxiety Scale (SCAS) after accounting for variance due to State‐Trait Anxiety Inventory scores. CASI Physical Concerns scores incrementally predicted scores on each of the SCAS scales, whereas scores on the Social and Psychological Concerns subscales incrementally predicted scores on conceptually related symptom scales (e.g. CASI Social Concerns scores predicted Social Phobia symptoms). Overall, this study demonstrates that there is added value in measuring AS factors in children and adolescents.     

Inverted-U effects generalize to the judgment of subjective properties of faces

Researchers studying absolute identification have long known that it takes more time to identify a stimulus in the middle of a range than one at the extremes. That is, there is an inverted-U relation between mean response time and response position. In this task, an inverted-U relation also exists between response uncertainty and response position. Similarly, an inverted-U relation between mean response time and response position has been found for psychometric measures involving questions about the self. However, psychophysicists explain these inverted-U effects differently than do self-schema researchers. We propose an integrative framework in which task constraints explain these effects. To verify the generality of these inverted-U effects, we hypothesized that they would exist in three tasks having similar constraints--in this case, tasks involving the judgment of subjective properties of faces on a Likert-type scale. Our results are consistent with this hypothesis. We discuss the relevance of the results for other applications of Likert-type scales.     

The Effects of Assessment Mode and Privacy Level on Self‐Reports of Risky Sexual Behaviors and Substance Use Among Young Women1

This study examined differences in reported behaviors by assessment mode under differing privacy levels. Females were randomized to a computer‐administered self‐interview (CASI) or self‐administered questionnaire (SAQ) assessment of sexual risk and substance use behaviors that was completed non‐anonymously or anonymously. There were few differences in reported behaviors between assessment modes and privacy levels. However, an assessment mode by privacy‐level interaction was found for 2 outcomes (unprotected oral sex, recent sexual partner). Greater disclosure occurred in the CASI‐anonymous condition than the CASI‐non‐anonymous condition. Those in the CASI condition disclosed more instances of drinking until intoxicated than did those completing the SAQ. Overall, the findings suggest that CASI and SAQ under differing privacy levels result in comparable health behavior self‐reports.     

Item factor analysis: current approaches and future directions

The rationale underlying factor analysis applies to continuous and categorical variables alike; however, the models and estimation methods for continuous (i.e., interval or ratio scale) data are not appropriate for item-level data that are categorical in nature. The authors provide a targeted review and synthesis of the item factor analysis (IFA) estimation literature for ordered-categorical data (e.g., Likert-type response scales) with specific attention paid to the problems of estimating models with many items and many factors. Popular IFA models and estimation methods found in the structural equation modeling and item response theory literatures are presented. Following this presentation, recent developments in the estimation of IFA parameters (e.g., Markov chain Monte Carlo) are discussed. The authors conclude with considerations for future research on IFA, simulated examples, and advice for applied researchers.