大学生过度消费量表编制

在文献分析和半结构化访谈的基础上编制大学生过度消费初始量表。以250名大学生为被试进行项目分析和探索性因素分析,以401名大学生为被试进行信效度检验,最终形成的正式量表包含透支、计划和享乐三个维度,共11个条目。结果发现,用于测量大学生过度消费的多维量表、各维单项目量表和单项目量表均具有较高的内部一致性信度以及良好的结构效度和效标效度,可以作为后续相关研究的测量工具。     

Probing item social desirability by correlating personality items with Balanced Inventory of Desirable Responding (BIDR): A validity examination

Researchers often include a social desirability measure in personality measures, commonly the Balanced Inventory of Desirable Responding (BIDR), and correlate it with personality items to probe for social desirability of the items. A strong correlation between BIDR scores and a personality item would indicate high item social desirability. The current research assesses the validity of this practice. Results showed that these correlations have high validity only when BIDR scores are calculated as a continuous variable rather than as dichotomized item scores. In addition, self-deception scores have higher validity for detecting item social desirability than do impression management scores. The current research supported the use of the self-deception scores, in particular, to detect highly desirable or undesirable items.     

Item‐level Frame‐of‐reference Effects in Personality Testing: An investigation of incremental validity in an organizational setting

A recent line of research has investigated the frame‐of‐reference effect on personality scale scores, in which self‐report personality items are contextualized to the specific performance setting (e.g., work, school) within which the performance criterion is gathered. Contextualization has been shown to increase both the reliability and the criterion‐related validity of the personality scale scores by facilitating the self‐presentation of respondents, and by more closely measuring the personality construct relevant to the performance domain. The current research extends this area of personality research in two ways. First, this study tests the generalizability of the effectiveness of item‐level contextualization within an organizational setting. Second, this study also provides the necessary test of the incremental validity of this contextualized approach to personality measurement above and beyond the traditional, noncontextualized approach. The results confirm that a work‐specific personality measure, contextualized at the item level, adds to the prediction of job performance above and beyond that obtained by a noncontextual measure of the same personality traits. Practical and theoretical implications are discussed.     

Examining the impact of unscorable item responses on the validity and interpretability of MMPI-2/MMPI-2-RF Restructured Clinical (RC) Scale scores

This article examined the impact of unscorable item responses on the psychometric validity and practical interpretability of scores on the Restructured Clinical (RC) Scales of the Minnesota Multiphasic Personality Inventory-2/Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2/MMPI-2-RF). In analyses conducted with five archival samples, we found that relatively large proportions of unscorable responses (defined as 10% or more of the items scored on a scale) were relatively uncommon, occurring most often in forensic samples. Simulated unscorable responses were inserted in varying proportions (10% to 90%) in place of the responses of participants in two of the archival samples for which criterion data were available. Analyses were conducted to gauge the impact of unscorable responses on the criterion validity of scores on these scales and their interpretability. Impact on validity was evaluated by examining correlations with extra-test variables as a function of increasing levels of unscorable responding. Interpretability was evaluated by examining the proportion of participants who produced clinically elevated RC Scale scores as a function of unscorable responding. Results indicate that whereas scale score validity was relatively robust up to a level of 50% unscorable responses, interpretability was substantially compromised at only 10% unscorable responding. This suggests that prorated scores may be used to correct for the impact of unscorable responses on the interpretability of RC Scale scores at levels as high as 50% unscorable responses. Classification analyses supported this possibility. Further steps needed to explore the feasibility of using prorated scores are discussed.     

Development and validation of the Intervention Skills Profile–Skills: A brief measure of student social-emotional and academic enabling skills

The purpose of this study was to support the development and initial validation of the Intervention Selection Profile (ISP)–Skills, a brief 14-item teacher rating scale intended to inform the selection and delivery of instructional interventions at Tier 2. Teacher participants (n = 196) rated five students from their classroom across four measures (total student n = 877). These measures included the ISP-Skills and three criterion tools: Social Skills Improvement System (SSIS), Devereux Student Strengths Assessment (DESSA), and Academic Competence Evaluation Scales (ACES). Diagnostic classification modeling (DCM) suggested an expert-created Q-matrix, which specified relations between ISP-Skills items and hypothesized latent attributes, provided good fit to item data. DCM also indicated ISP-Skills items functioned as intended, with the magnitude of item ratings corresponding to the model-implied probability of attribute mastery. DCM was then used to generate skill profiles for each student, which included scores representing the probability of students mastering each of eight skills. Correlational analyses revealed large convergent relations between ISP-Skills probability scores and theoretically-aligned subscales from the criterion measures. Discriminant validity was not supported, as ISP-Skills scores were also highly related to all other criterion subscales. Receiver operating characteristic (ROC) curve analyses informed the selection of cut scores from each ISP-Skills scale. Review of classification accuracy statistics associated with these cut scores (e.g., sensitivity and specificity) suggested they reliably differentiated students with below average, average, and above average skills. Implications for practice and directions for future research are discussed, including those related to the examination of ISP-Skills treatment utility.     

Comparative validity of the MMPI-2 Wiener-Harmon subtle-obvious scales in male prison inmates

The Wiener-Harmon subtle-obvious MMPI subscales (Wiener, 1948; Wiener & Harmon, 1946) have been the subject of considerable debate. In this study, we examined the intercorrelations among full clinical scale T scores and their subtle and obvious subscales in an offender population. Low subtle to full-scale correlations were observed, suggesting that these items contribute little to full-scale scores. Further, we explored the criterion validity of the MMPI-2 subtle-obvious scales in this forensic sample. The results demonstrated that the obvious scales of the MMPI-2 had greater criterion validity than the subtle scales when compared to crime history data. Scores on the subtle subscales were unrelated to crime history. The Ma-O subscale demonstrated the strongest association to crime history data. The findings from this study add to a mounting body of evidence indicating that when respondents are in a position to understand item content, and can therefore provide a direct self-appraisal, responses are most predictive of clinical criteria.     

Cognitive processes and computerized personality assessment

Several implications of the cognitive viewpoint on personality are tested and the predictive validity of cognitive processing variables is assessed with judgements of parents and friends as a criterion measure. Free recall of items was related to cognitive schemas but reaction time during score recall was not. Ease of faking as well as response latency during faking were not related to cognitive schemas. Intra-individual analysis revealed a consistent non-linear relationship between response latency and item score in all conditions of the experiment. Although some cognitive process variables were correlated with the criterion measures, adding these variables to item scores did not always increase the predictive validity.     

Incremental validity of the frame-of-reference effect in personality scale scores: a replication and extension

Context-specific personality items provide respondents with a common frame of reference unlike more traditional, noncontextual personality items. The common frame of reference standardizes item interpretation and has been shown to reduce measurement error while increasing validity in comparison to noncontextual items (M. J. Schmit, A. M. Ryan. S. L. Stierwalt. & S. L. Powell, 1995). Although the frame-of-reference effect on personality scales scores has been well investigated (e.g., M. J. Schmit et al., 1995), the ability of this innovation to obtain incremental validity above and beyond the well-established, noncontextual personality scale scores has yet to be examined. The current study replicates and extends work by M. J. Schmit et al. (1995) to determine the incremental validity of the frame-of-reference effect. The results indicate that context-specific personality items do indeed obtain incremental validity above and beyond both noncontextual items and cognitive ability, and in spite of socially desirable responding induced by applicant instructions. The implications of these findings for personnel selection are discussed.     

Normative Scoring of Multidimensional Pairwise Preference Personality Scales Using IRT: Empirical Comparisons With Other Formats

In this article, we offer some suggestions as to why tetrads and pentads have become the dominant formats for administering multidimensional forced choice (MFC) items but, in turn, raise questions regarding the underlying psychometric model and means of addressing item quality and scoring accuracy. We then focus our attention on multidimensional pairwise preference (MDPP) items and present an item response theory–based approach to constructing and modeling MDPP responses directly, assessing information at the item and scale levels, and a way of computing standard errors for trait scores and estimating scale reliability. To demonstrate the viability of this method for applied use, we show that the correspondence between MDPP scores derived from direct modeling with those obtained using single statement and unidimensional pairwise preference measures administered in a laboratory setting. Trait score correlations and criterion related validities are compared across testing formats and rating sources (i.e., self and other), and the usefulness of our model-based approach is further demonstrated by some illustrative results involving computerized adaptive tests (CAT).     

Structured personality test item characteristics and validity

Using the structured personality test item as the unit of analysis, the purpose of this research was to evaluate the relationship between validity and a variety of other test item parameters. Of particular interest was the relationship of test item criterion validity to negative keying and to negative wording. By drawing a distinction between negative keying and negative wording it was demonstrated that the use of balanced scales to control acquiescence need not result in a reduction in item criterion validity. Whereas the use of negative wording has in the past reduced validity, data demonstrated that positively worded, negatively keyed items did not. In addition, results indicated that clear, moderately short, relevant test items tended to be the most empirically valid.     

A successive approximation method of maximizing test validity

The ratio of item validity to item-total correlation can be used to select items which will tend to yield the maximum correlation with a criterion. Items to be retained are identified by comparing the ratio for each item with the validity of the original test. Further improvement of the validity in the experimental sample can be obtained by adding items to or removing items from the selected nucleus, according to recomputed ratios involving the correlations of the items with the nucleus and evaluated by means of a revised cut-off point. With slight variations, the method may be used for interest and personality tests as well as for aptitude material. The principal advantage over previous methods is that for any cycle of the analysis an exact cut-off point is provided.     

A measure of autonomy

This article reports the development of a measure of indicidual differences in autonomous rule compliance. The autonomy scale (a short, easily administered CPI based test) was developed within the framework of a multidimensional, role-theoretical model of moral development. Five samples were used in the construction of the scale. Two of the samples (total n = 111) were used to derive the autonomy scale. The items for the scale were derived through the sequential use of two common item selection stategies: criterion keying and factor analysis. An initial set of 55 CPI items were derived using an "ideal" autonomy Q-sort profile as a selection criterion, and an Alpha factor solution was used to reduce this initial pool to a final set of 25 items. Several analyses were conducted using three additional samples (total n = 245) to estimate the reliability of the scale and determine its validity. The results of these analyses provide initial evidence for the content, criterion-realted, and construct validity of the scale and indicate that the measure has an adequate reliability.     

True or False? Keying Direction and Acquiescence Influence the Validity of Socio-Emotional Skills Items in Predicting High School Achievement

What type of items, keyed positively or negatively, makes social-emotional skill or personality scales more valid? The present study examines the different criterion validities of true- and false-keyed items, before and after correction for acquiescence. The sample included 12,987 children and adolescents from 425 schools of the State of São Paulo Brazil (ages 11–18 attending grades 6–12). They answered a computerized 162-item questionnaire measuring 18 facets grouped into five broad domains of social-emotional skills, i.e.: Open-mindedness (O), Conscientious Self-Management (C), Engaging with others (E), Amity (A), and Negative-Emotion Regulation (N). All facet scales were fully balanced (3 true-keyed and 3 false-keyed items per facet). Criterion validity coefficients of scales composed of only true-keyed items versus only false-keyed items were compared. The criterion measure was a standardized achievement test of language and math ability. We found that coefficients were almost as twice as big for false-keyed items’ scales than for true-keyed items’ scales. After correcting for acquiescence coefficients became more similar. Acquiescence suppresses the criterion validity of unbalanced scales composed of true-keyed items. We conclude that balanced scales with pairs of true and false keyed items make a better scale in terms of internal structural and predictive validity.     

Development of a short form of the Treatment Evaluation Inventory for acceptability of psychological interventions

The Treatment Evaluation Inventory of Kazdin, French, and Sherick is a 19-item measure of the perceived acceptability of behavioural treatments. Development of two brief forms was based on data from two sources. For Study 1, data from 218 completed questionnaires were used to develop internally consistent brief scales. In Study 2 internal consistency and the validity of the brief forms were estimated for a set of 131 questionnaires. Item reduction was achieved by analysis of item-total minus item correlations. Brief forms with 3, 6, 9, and 12 items were proposed. Their internal consistency (Cronbach alpha) and construct validity were based on correlations of scores on each short form with the full scale scores and on comparing means of different forms. Discriminant validity was based on the difference between two groups (estimated effect size 0.7). Scores for all forms showed high internal consistency and correlated highly with total scale scores. Only the 12-item brief scale yielded mean scores similar to the full scale. The 3-item form could be used as a quick screen, and the 12-item form for more intensive purposes as it is most similar to the full-scale.     

Constructing personality scales under the assumptions of an ideal point response process: toward increasing the flexibility of personality measures

The main aim of this article is to explicate why a transition to ideal point methods of scale construction is needed to advance the field of personality assessment. The study empirically demonstrated the substantive benefits of ideal point methodology as compared with the dominance framework underlying traditional methods of scale construction. Specifically, using a large, heterogeneous pool of order items, the authors constructed scales using traditional classical test theory, dominance item response theory (IRT), and ideal point IRT methods. The merits of each method were examined in terms of item pool utilization, model-data fit, measurement precision, and construct and criterion-related validity. Results show that adoption of the ideal point approach provided a more flexible platform for creating future personality measures, and this transition did not adversely affect the validity of personality test scores.     

Figure-Ground Phenomena in the Perception of Physical and Social Stimuli

An instrument, the Grasha-Riechmann Student Learning Style Scales (GRSLSS), was developed to assess six student learning styles. These styles are Independent, Dependent, Avoidant, Participant, Collaborative, and Competitive. A “rational approach” was used to develop the GRSLSS and evaluate its construct validity. The process included professional and student inputs in special procedures for selecting scale items and designing criterion items. The utility of this approach is considered and problems critiqued. The rational approach yielded relatively high temporal reliability coefficients (range across scales r = .76 to r = .83; N = 269) and numerous meaningful correlations between criterion items and scale scores.     

The sensation seeking scale V (SSS-V): Still reliable and valid

This is a response to Gray and Wilson’s (2007) article: “A detailed analysis of the reliability and validity of the sensation seeking scale in a UK sample”. Gray and Wilson analysed the items in the four subscales of the SSS-V, using a Likert type response format and deconstructing the forced choice format of the original. However they used some anachronistic items from the old 1978 form rather than the revisions of these items in the newer form. But even excluding the 19 items from the 80 item test not meeting their internal reliability criterion did not improve the reliabilities of the old scales in their Likert format. Validity of the SSS is not really addressed despite the title of the article.     

Scale validation via quantifying item validity using the Dm index

While most validity indices are based on total test scores, this paper describes a method for quantifying the construct validity of items. The approach is based on the item selection technique originally described by Piazza in 1980. Unfortunately, Piazza's P2 index suffers from some substantial limitations. The Dm coefficient provides an alternative which can be used for item selection and provides a validity index for a set of items. The index is similar to that of traditional criterion-related validity indices. Criterion-related validity is used to demonstrate the accuracy of hypothesized relations of the measure with outcome variables of interest in research and practice. This method may be useful when the sample of items or persons is small, rendering more traditional approaches such as factor analysis or item response theory inappropriate. An example of how to use the technique is provided.     

Assessment of adolescents' learned helplessness in achievement situations

Three studies are reported that describe the development, reliability, and initial validation of the Mastery Orientation Inventory (MOI; Reynolds & Miller, in press) as a measure of generalized learned helplessness in adolescents. In Study 1, an initial version of 50 items was administered to a sample of 112 adolescents. A revised 40-item scale with an internal consistency reliability of .94 was then constructed, which correlated significantly with measures of locus of control and depression. Study 2 involved the administration of the 40-item MOI to 645 adolescents. In this study, the reliability of the MOI was .92, and MOI scores were significantly correlated with subjects' depression scores and with self-reported grade point average. Factor analysis of the MOI items produced a strong first factor with high loadings for every item. In Study 3, the 112 subjects who participated in Study 1 were, 3 months later, readministered the MOI, locus of control, and depression measures. As an external criterion variable, 13 teachers provided global ratings of learned helpless/mastery-oriented behaviors for 99 of these subjects. The MOI demonstrated high internal consistency (r alpha = .95) and adequate test-retest (rtt = .77) reliability. Validity was supported by significant correlations between the MOI and the three criterion variables (/rs/ = .49-.58). The results of these investigations provide initial support for the reliability and validity of the MOI as a measure of learned helplessness.     

Using item response theory to study the convergent and discriminant validity of three questionnaires measuring cigarette dependence

To determine whether the Cigarette Dependence Scale, the Fagerstr?m Test for Nicotine Dependence, and the Nicotine Dependence Syndrome Scale (NDSS) reliably and correctly assessed both weakly and severely dependent individuals, the authors collected data via Internet from 2,435 current smokers, from 2004 to 2007. They used a 2-parameter item response model to determine the difficulty and discrimination of each question and used correlations between latent scores to assess convergent and discriminant validity. The reliability of all scales was close to or exceeded .70. Both the Cigarette Dependence Scale and the Fagerstr?m Test for Nicotine Dependence had 1 misfitting item. Each NDSS scale had at least 2 misfitting items. The information curve of each of the questionnaires peaked between -2 and 2 and was low at both extremes. All questionnaires had adequate reliability and were more informative for a medium level of the underlying cigarette dependence continuum than for both extremes of this continuum. The correlations between latent scores indicated good convergent validity between questionnaires and low discriminant validity between NDSS subscales, except for Tolerance. This result suggests that nicotine dependence may not be composed of 5 dimensions but may be unidimensional and distinct from reduced sensitivity to the effects of smoking (Tolerance).