A Latent-Variable Modeling Approach to Assessing Reliability and Validity of a Creativity Instrument

ABSTRACT: A multiple-choice paper-and-pencil test was developed based on the constructs from the Torrance Test of Creative Thinking (TTCT). The rationale of this new instrument was to shorten the amount of time required for the administration and scoring of creativity tests. This instrument was translated into Spanish and administered along with the TTCT and the Villa and Auzmendi Creativity Test (VAT) to 2,270 students in Spain. Teachers' ratings of student creativity were also obtained. Using a traditional concurrent validation approach from an earlier study, we found low but significant correlations between the scores of the four subscales of the new test and the same subscales of the VAT and TTCT, as well as the students' academic achievement measures. In terms of reliability, mostly moderate (internal consistency) reliability coefficients were found for the new test. As an alternative to the traditional approach, this study employed structural equation modeling (SEM) with multiple indicators to examine the validity and reliability of the new creativity test. The results indicated that substantial improvements in the validity and reliability estimation over the traditional approaches can be made by using SEM with multiple indicators.     

父母教养方式问卷(PBI)的信效度研究

本研究对目前国外广泛使用的"父母教养方式问卷"(Parental Bonding Instrument,PBI)进行了中文版修订.708名大学生完成了测试.其中136人在两周后进行了重测,并用EMBu作为检验效度的指标之一.采用相关分析、验证性因素分析等方法考察问卷的信、效度.结果表明问卷的四因素结构模型优于二因素和三因素结构模型,各项拟合指标均在0.90以上;问卷的内部一致性信度在0.740-0.851之间.重测信度在0.619-0.765之间.均达到了心理测量学的要求.因此,修订后的中文版PBI具有良好的信效度指标.可以作为评定大学生父母教养方式的测量工具使用.     

A Level of Care Instrument for Children's Systems of Care: Construction, Reliability and Validity

The Child and Adolescent Level of Care System/Child and Adolescent Service Intensity Instrument (CALOCUS/ CASII) is designed to help determine the intensity of services needed for a child served in a mental health system of care. The instrument contains eight dimensions that are rated following a comprehensive clinical evaluation. The dimensions are risk of harm, functionality, co-morbidity (psychiatric, substance abuse, development disability and medical), environmental stressors, environmental supports, the child's resiliency, and the child and family's willingness to engage in treatment. An algorithm connects the ratings to a level of care recommendation. The instrument specifies six levels of care defined flexibly enough to consider whatever services are available. The results of psychometric testing using raters with a broad range of clinical experience and training from four different systems of care around the country are presented. The testing demonstrates excellent reliability when rating vignettes. Using children and adolescents in live system of care clinical settings, the CALOCUS/CASII demonstrates reasonable validity when compared with the Child Global Assessment Scale, and the Child and Adolescent Functional Assessment Scale.     

Reliability and Validity of a DSM-IV Based ADHD Screener

The Diagnostic Rating Scale (DRS) was completed by the parents and teachers of 82 children referred for clinical evaluations, 73 referred children seen twice, and 218 non-referred children from the community. The DRS, which uses a categorical rather than a dimensional rating approach, was 70% to 90% sensitive to diagnoses of Attention Deficit/Hyperactivity Disorder (ADHD) made by blind clinical teams. In research and clinical applications, the DRS could improve screening efficiency, especially in situations where it would be desirable to exclude all children who might have ADHD or identify all children with Hyperactive-Impulsive symptoms. Because of its objectivity and consistency with the Diagnostic and Statistical Manual (DSM)-IV criteria, the DRS could facilitate comparison of participant samples across studies.     

大学生自我怜悯问卷的修订及信效度研究

目的:对Neff编制的自我怜悯量表进行修订和信效度检验。方法:总共对440名大学生进行了问卷预测与正式测验,使用SPSS13.0和LISREL8.70软件对数据进行分析。结果:①修订后的量表包含6个维度、26个条目。②各维度和量表的同质信度在0.64-0.87之间;探索性因素分析表明各条目对所属因子有中度以上的负荷,抽取的6个因子解释了总方差的55.09%。验证性因素分析的拟合优度指数、相对拟合优度指数、非标准化拟合优度指数为0.85-0.94,近似误差均方根为0.059。结论:大学生自我怜悯量表的信效度符合心理测量学要求。     

Reliability and Validity of a Paper-and-Pencil Test Measuring Hemisphere Preference

The present studies examined the reliability and validity of the Preference Test (PT), a widely used self-report index of preferred hemisphere thinking styles. The PT consists of items that intend to tap left-hemisphere and right-hemisphere cognitions. In the first study (N=47), PT scores were found to have reasonable test–retest stability. In the second study (N=334), a factor analysis of PT scores showed that the PT has a two-factor solution that can best be interpreted in terms of putative left- and right-hemisphere items. In the third study (N=29), background EEG activity (F3, F4, P3 and P4) of subjects was recorded. PT scores were found to be related to EEG asymmetry patterns for frontal, but not for parietal, recording sites. More specifically, subjects with a left-hemisphere preference displayed relatively stronger left frontal activity than subjects with a right-hemisphere preference. By and large, the present findings provide strong support for the reliability and some suggestive evidence for the validity of the PT.     

An Analytic Creativity Assessment Scale for Digital Game Story Design: Construct Validity,Internal Consistency and Interrater Reliability

Mobile technology has rapidly made digital games a popular entertainment to this digital generation, and thus digital game design received considerable attention in both the game industry and design education. Digital game design involves diverse dimensions in which digital game story design (DGSD) particularly attracts our interest, as the literature needs more information, especially the creativity assessment of DGSD. Existing measuring tools do not adequately address the characteristics of game-story duality. Thus, an analytic creativity assessment scale of DGSD (CAS-DGSD) based on literature and original ideas was developed in our previous work. This study aims to statistically examine its construct validity, internal consistency reliability, and interrater reliability to verify its effectiveness. Three commercial games of 3 different game genres (action, puzzle, and role-play) were rated by 32 student raters and 4 expert raters. Statistical results show acceptable construct validity, internal consistency reliability, and interrater reliability of the CAS-DGSD. The CAS-DGSD not only helps evaluators like teachers identify which aspects of DGSD are short of creativity, but also serves as a guideline for digital game story designers like design students and product developers to tailor creative and entertaining game stories.     

The Validity and Reliability of Value Measurements

Four scaling techniques for measuring human values were compared, using the measurement properties of test-retest reliability, discriminatory power, convergent validity, and achievement of substantive results consistent with theoretical expectations. The standard rank-order version of the Rokeach Value Survey was superior to a 100-point rating, magnitude estimation, and the handgrip scaling procedure with respect to each of these fundamental measurement requirements. Because it exhibited less measurement error than the other methods, rank-order scaling is shown to be the best technique for measuring human values even though it achieves only an ordinal level of measurement.     

Reliability and External Validity of Neurobiological Experiments

Reliability and external validity are two fundamental values that pose incompatible constraints on neurobiological experiments. The more reliability an experimental result achieves, the less external validity it earns, and vice versa. In this article, I propose an externalist interpretation of external validity: the external validity of an experimental result depends not only on how much complexity is built into an experimental design, but also on the relationship between the experimental result and other related experiments. This externalist interpretation, which explains how a neurobiological experiment can be reliable and externally valid simultaneously, suggests a new way to understand the epistemology of neurobiological experimentation.     

The Feeling-Tone Questionnaire: Reliability and Validity of a Direct Patient Assessment Screening Instrument for the Detection of Depressive Symptoms in Cases of Dementia

The purpose of this study was to develop a reliable and valid measure of affective disorder in elderly demented patients. The field lacks instruments which are sensitive to detecting depressive signs in severely as well as mildly and moderately demented subjects. Two samples of subjects were chosen for study. The first sample consisted of subjects from six institutions, which were chosen for study from a probability sample of 25 long-term care institutions in New York City. This sample was part of the Cross-National Institutional Study conducted in New York and London (Gurland et al., 1979; Mann et al., 1984). Thirty patients were selected at random within each institution. The second sample consisted of 52 inpatients at Willard Psychiatric Center, a traditional state psychiatric hospital in upstate New York. All subjects had a chart diagnosis of dementia and were 60 years old or older. The mean age of the sample was 82 years and 56% of the subjects were female. The Feeling-Tone Questionnaire, which was developed for these analyses, consists of 16 dichotomous items and 16 5-point Likert ratings of affect. The reliability of this scale using Cronbach's alpha is .91 for the long-term care institutional sample and .90 for the psychiatric hospital sample. Interrater reliability for two raters on ten cases is .99. Test–retest reliability on ten cases with a 1-day to 2-day interval between trials is .81. A short mood scale was developed from the observational data as a validity measure for the Feeling-Tone Questionnaire. Evidence for the validity of the Feeling-Tone Questionnaire is provided.     

Reliability and Validity Analyses of the Self-Management Questionnaire

The authors developed a parent-report measure to enhance findings for diagnoses and evaluations, and therefore, improve on the assessments currently being widely used for this purpose. Data were gathered on 322 clients (4–18 years old) seeking treatment at a private practice setting. Factor analysis revealed 18 subscales. Scores were internally consistent and distinct from other subscales. In addition, the questionnaire showed good diagnostic utility for depression, Oppositional Defiant Disorder (ODD), and Pervasive Developmental Disorder (PDD). This study provides sufficient evidence of the reliability and validity of the Self-Management Questionnaire (SMQ) to recommend its use for skills and needs assessment with children and adolescents.     

职业评定量表(VIESA)的信度和效度研究

根据D.E.Super的职业发展理论,职业选择的成熟与发展一般分为五个阶段,中学生正处在从探索到成熟的重要转变期。本研究以美国大学测验中心的职业分类系统为基础,以职业评定量表VIESA为工具和对象,对北京市第六十五中学的179名高中三年级学生和北京师范大学心理学系的55名二年级学生进行了施测,检验了量表的信度和效度。量表的信度指标基本在.70以上,构想效度良好,以专业满意感为效标的效标效度达到显著水平。     

Scale of Unpredictability Beliefs: Reliability and Validity

Experiencing unpredictability in the environment has a variety of negative outcomes. However, these are difficult to ascertain due to the lack of a psychometrically sound measure of unpredictability beliefs. This article summarizes the development of the Scale of Unpredictability Beliefs (SUB), which assesses perceptions about unpredictability in one's life, in other people, and in the world. In Study I, college students (N = 305) responded to 68 potential items as well as other scales. Exploratory factor analysis yielded three internally consistent subscales (Self, People, and World; 16 items total). Higher SUB scores correlated with more childhood family unpredictability, greater likelihood of parental alcohol abuse, stronger causal uncertainty, and lower self-efficacy. In Study II, a confirmatory factor analysis supported the three-factor solution (N = 186 college students). SUB scores correlated with personality, childhood family unpredictability, and control beliefs. In most instances the SUB predicted family unpredictability and control beliefs beyond existing unpredictability measures. Study III confirmed the factor structure and replicated family unpredictability associations in an adult sample (N = 483). This article provides preliminary support for this new multi-dimensional, self-report assessment of unpredictability beliefs, and ideas for future research are discussed.     

Reliability and Validity of the Spiritual Care-Giving Scale in a Turkish Population

This study aimed to adapt an English version of the survey tool Spiritual Care-Giving Scale for Turkish students and to evaluate its psychometric properties. Spiritual care is a central element of holistic nursing, but is not often made explicit in the theoretical and practical components of preregistration nursing programs. A composite scale will assist in identifying students’ perceptions and issues to be addressed in curricula and practice settings in Turkey. The scale was composed of 35 items and five subscales. Cronbach’s α reliability coefficient was .96, and item–total point correlations were between .37 and .77. In addition, split-half reliability coefficient was .88. The Spiritual Care-Giving Scale was found to be a valid and reliable instrument for measuring the multifaceted perspectives of spirituality and spiritual care in practice by students. Further testing of this scale is required with other student populations and clinicians.     

Reliability and Validity of a Brief Clinician-Report Scale for Screening Behavioral Inhibition

Given the implications of behavioral inhibition (BI) on social anxiety and depression, several cost-effective methods have been developed to detect this trait. However, there is no instrument for obtaining a systematic measurement of BI that uses the reports of mental health professionals. The present study aims to explore the psychometric properties of the Behavioral Inhibition Observation System (BIOS), a brief tool enabling clinicians to provide a quick measure of BI. Parents and teachers of 361 preschool children provided measures of BI and psychopathology. Clinicians completed the BIOS after an individual meeting with every child. Seventy-four randomly selected children were exposed to standardized observation. Two researchers completed the BIOS after standardized observation. The results show that the BIOS is structured in one dimension that explains 60 % of the total variance. It shows good internal consistency (Cronbach’s Alpha?=?.88) and good test-retest reliability. The BIOS scores moderately converge with observational measures of BI and adequately distinguish constructs not related to BI (such as aggressiveness, disruptive behavior, or peer conflict). The results support that the BIOS, an 8-item scale that takes 3 min to complete, is valid and reliable for capturing a clinician’s measure of BI. The BIOS may help to detect temperamental disposition to social anxiety.     

Reliability and Validity of a Brief Life Satisfaction Scale with a High School Sample

This study investigated the psychometric properties of a brief life satisfaction scale, the Brief Multidimensional Students’ Life Satisfaction Scale (Huebner et al., 2004a), with a high school age sample. The results revealed preliminary evidence of internal consistency, two-week test–retest reliability, construct validity, and criterion-related validity for research purposes. Expected demographic effects were also obtained. Overall, the findings suggest that the BMSLSS may provide a useful indicator of adolescent life satisfaction in studies in which brevity of measures is an important consideration.     

Research Validity Scales for the NEO-PI-R: Additional Evidence for Reliability and Validity

We examined the reliability and validity of the research validity scales (Schinka, Kinder, & Kremer, 1997) for the NEO-Personality Inventory-Revised (NEO-PI-R) in a clinical sample. The Negative Presentation Management (NPM) and Positive Presentation Management (PPM) scales were found to have satisfactory internal consistency reliability. Support for the validity of these scales was provided by the pattern of convergent and discriminant correlations with respective Personality Assessment Inventory (PAI) validity scales. Finally, PAI profiles of individuals with invalid NPM scores were found to differ significantly from those with valid NPM scores. Comparisons of the invalid profiles with profiles from other clinical samples provided additional support for the use of the NPM scale as a measure of negative impression management.     

The Body Awareness Questionnaire: Reliability and Validity

Norms and reliability and validity data are presented for an objectively scored Body Awareness Questionnaire (BAQ), which is suitable for use with college students and nonstudent adults. The BAQ is an 18-item scale designed to assess self-reported attentiveness to normal nonemotive body processes, specifically, sensitivity to body cycles and rhythms, ability to detect small changes in normal functioning, and ability to anticipate bodily reactions. Research applications, including investigation of sex-related differences in body awareness and its correlates, are discussed.