Self‐reported attitude scales: current practice in adequate assessment of reliability,validity, and dimensionality

The development of methods to create self‐reported attitude scales has lost momentum, in part because of increased research focused on implicit measures. This paper reviews 162 papers on methodological approaches applied to the validation and assessment of attitude scales. Assessment of methodological approaches applied indicates that neither reliability, validity, nor dimensionality assessments are consistently used according to standard operating procedures or in accordance with best practice. Within current practices in the field of attitude scale development, the full potential of self‐report scales is not met, in part because of such methodological issues. The improvement of existing practices and adoption of promising new developments in attitude scale construction and evaluation are discussed, together with recommendations for best practice in scale validation.     

The Eysenckian personality factors and their correlations with academic performance

Background. The relationship between personality and academic performance has long been explored, and a recent meta‐analysis established that measures of the five‐factor model (FFM) dimension of Conscientiousness have similar validity to intelligence measures. Although currently dominant, the FFM is only one of the currently accepted models of personality, and has limited theoretical support. In contrast, the Eysenckian personality model was developed to assess a specific theoretical model and is still commonly used in educational settings and research. Aims. This meta‐analysis assessed the validity of the Eysenckian personality measures for predicting academic performance. Sample. Statistics were obtained for correlations with Psychoticism, Extraversion, and Neuroticism (20–23 samples; N from 8,013 to 9,191), with smaller aggregates for the Lie scale (7 samples; N= 3,910). Methods. The Hunter–Schmidt random effects method was used to estimate population correlations between the Eysenckian personality measures and academic performance. Moderating effects were tested using weighted least squares regression. Results. Significant but modest validities were reported for each scale. Neuroticism and Extraversion had relationships with academic performance that were consistent with previous findings, while Psychoticism appears to be linked to academic performance because of its association with FFM Conscientiousness. Age and educational level moderated correlations with Neuroticism and Extraversion, and gender had no moderating effect. Correlations varied significantly based on the measurement instrument used. Conclusions. The Eysenckian scales do not add to the prediction of academic performance beyond that provided by FFM scales. Several measurement problems afflict the Eysenckian scales, including low to poor internal reliability and complex factor structures. In particular, the measurement and validity problems of Psychoticism mean its continued use in academic settings is unjustified.     

A double law of comparative judgment for the analysis of preferential choice and similarities data

By virtue of certain modifications in the Law of Comparative Judgment, equations are developed which (i) permit the construction of a joint scale of individuals and items, as in the case of attitude measurement, directly from their pair-comparison preferences, and (ii) take into account the variable of laterality which is significant for the construction of group preference scales.This work was supported by grant NSF-G5820 from the National Science Foundation.     

Quantitative approaches in environmental studies

Psychological scaling methods can be applied and can bring substantial knowledge to bear upon problems within the realm of environmental hygiene. Methodological issues related to the possibility of obtaining calibrated scales of perceptual variables such as discomfort, annoyance, and unpleasant odors are critical when the scale values are to be entered into a physical pollution index or a perceived environmental quality index, both being applied in the form of norms or recommendations. The level of measurement (ordinal, interval, or ratio scale) is related to the question of calibration of scales. From a practical point of view, the effects of observer-environment dependence are important to disguise in the scale values. Problems having to do with calibration of scales and with application of knowledge about perceptual processes to the scales are being explored in ongoing research projects. These include the measurement of temperature discomfort in different climates and the measurement of annoyance in areas with different noise exposures. Attempts have been made to solve an odorous air pollution problem with psychological measurements.     

Relationship between the Millon Behavioral Health Inventory and the minnesota multiphasic personality inventory (MMPI) in low-back pain patients

The Millon Behavioral Health Inventory (MBHI) is being used with increasing frequency for the assessment of chronic pain, although there is a relative lack of evidence as to its utility, and prior studies have not examined low back pain. This investigation compared the MBHI to the MMPI in a sample of low-back pain patients and analyzed subgroups of pain patients based upon their MBHI responses. Subjects were 60 patients who had been admitted to outpatient multidisciplinary pain clinics of two Chicago-area hospitals. Patients completed both the MMPI and the MBHI and provided demographic information. Results of correlational analyses indicated strong relationships between the MBHI psychogenic attitude, psychosomatic correlate, and prognostic index scales and the validity scales of the MMPI. The MBHI Pain Treatment Responsivity scale (PP) correlated with 16 of the other 19 MBHI scales. PP did not demonstrate specificity with low back pain patients. The results of both the scale comparisons and the exploratory two-group cluster subgroup analysis support the notion that responses to the MBHI are largely affected by the respondent's tendency to deny psychopathology or to admit emotional distress.     

On the evaluation and application of different scales for quantifying pairwise comparisons in fuzzy sets

One of the most critical issues in many applications of fuzzy sets is the successful evaluation of membership values. A method based on pairwise comparisons provides an interesting way of evaluating membership values. That method was proposed by Saaty, almost 20 years ago, and since then has captured the interest of many researchers around the world. However, recent investigations reveal that the original scale may cause severe inconsistencies in many decision-making problems. Furthermore, exponential scales seem to be more natural for humans to use in many decision-making problems. In this paper two evaluative criteria are used to examine a total of 78 scales which can be derived from two widely used scales. The findings in this paper reveal that there is no single scale that can outperform all the other scales. Furthermore, the same findings indicate that a few scales are very efficient under certain conditions. Therefore, for a successful application of a pairwise-comparison-based method the appropriate scale needs to be selected and applied.     

Assessing relationship quality across cultures: An examination of measurement equivalence

Researchers are increasingly studying close relationships across cultural contexts. One issue that arises when applying scales originally developed in Western countries to a different cultural context is measurement invariance. Researchers often do not examine whether scales show invariance across cultures and thus can be used with confidence. The goal of this article is to discuss the importance of measurement invariance, to discuss what testing invariance involves, and to test the measurement properties of scales of relationship satisfaction, commitment, intimacy, and trust across 4 samples (United States, Canada, Indonesia, and China). Analyses indicated that weak measurement invariance was met for all 4 scales, and assumptions of strong measurement invariance had to be relaxed for only a few items in each scale. Findings are discussed and recommendations are made regarding using these or other scales that have been shown to meet assumptions of invariance across different cultural groups.     

Polytomous Latent Scales for the Investigation of the Ordering of Items

We propose three latent scales within the framework of nonparametric item response theory for polytomously scored items. Latent scales are models that imply an invariant item ordering, meaning that the order of the items is the same for each measurement value on the latent scale. This ordering property may be important in, for example, intelligence testing and person-fit analysis. We derive observable properties of the three latent scales that can each be used to investigate in real data whether the particular model adequately describes the data. We also propose a methodology for analyzing test data in an effort to find support for a latent scale, and we use two real-data examples to illustrate the practical use of this methodology.     

Measurement scales and scale independence in the analytic hierarchy process

One approach to evaluate the relative performance of decision alternatives with respect to multiple criteria is provided by the analytic hierarchy process. The method is based on pairwise comparisons between attributes, and several numerical measurement scales for the ratio statements have been proposed. The choice of measurement scale is re‐examined, and new arguments supporting the measurement scale of geometric progression are derived. Separately from the measurement scale considerations, the effects of the scale parameter in geometric measurement scale are also studied. By using a regression model for pairwise comparisons data, it is shown that the statistical inference does not depend on the value of the scale parameter in the case of a single pairwise comparison matrix. It is also shown when the scale independence of statistical inference can be achieved in a decision hierarchy. This requires the use of the geometric‐mean aggregation rule instead of the traditional arithmetic‐mean aggregation. The results of the case study demonstrate that the measurement scale and the aggregation rule have potentially large impacts on decision support. Copyright © 2000 John Wiley & Sons, Ltd.     

Confirming satisfaction as an attitude within the service‐buying process

This paper presents research that uses cognitive and affective semantic differential scales to construct an attitude measuring instrument to be used in the services domain in the context of long applications to buy a service. The scales measure ‘Attitude to the Application Process’ (AAP) which represents satisfaction, as an internal ‘buying process’ attitude, which moderates the buying intention. Data collected over a three‐year period between 1996‐9, was analysed using exploratory and confirmatory factor analyses. The results provide evidence of a robust and useful cognitive scale. Although there were anomalies in the exploratory and confirmatory analysis of the affective scale, a robust scale is presented which represents either one or two factors. The whole measuring instrument of 14 items was developed as a means to access applicant attitudes during the application process to university — an extended and extreme application situation for first‐time consumers. The results identify that the instrument is sound and can be used in its original or modified form, depending on the context of the service‐process monitoring situation. The scales represent aspects of attitude not dealt with extensively in the literature and measure the perception of the application process as simple, open, friendly and helpful. These aspects of satisfaction mirror three attributes of service quality: assurance, responsiveness and empathy identified in previous research. Copyright © 2001 Henry Stewart Publications.     

A measure of autonomy

This article reports the development of a measure of indicidual differences in autonomous rule compliance. The autonomy scale (a short, easily administered CPI based test) was developed within the framework of a multidimensional, role-theoretical model of moral development. Five samples were used in the construction of the scale. Two of the samples (total n = 111) were used to derive the autonomy scale. The items for the scale were derived through the sequential use of two common item selection stategies: criterion keying and factor analysis. An initial set of 55 CPI items were derived using an "ideal" autonomy Q-sort profile as a selection criterion, and an Alpha factor solution was used to reduce this initial pool to a final set of 25 items. Several analyses were conducted using three additional samples (total n = 245) to estimate the reliability of the scale and determine its validity. The results of these analyses provide initial evidence for the content, criterion-realted, and construct validity of the scale and indicate that the measure has an adequate reliability.     

Mathematical structures and psychological measurements

The nature of psychological measurements in relation to mathematical structures and representations is examined. Some very general notions concerning algebras and systems are introduced and applied to physical and number systems, and to measurement theory. It is shown that the classical intensive and extensive dimensions of measurements with their respective ordinal and additive scales are not adequate to describe physical events without the introduction of the notions of dimensional units and of dimensional homogeneity. It is also shown that in the absence of these notions, the resulting systems of magnitudes have only a very restricted kind of isomorphism with the real number system, and hence have little or no mathematical representations. An alternative in the form of an extended theory of measurements is developed. A third dimension of measurement, the supra-extensive dimension, is introduced; and a new scale, the multiplicative scale, is associated with it. It is shown that supra-extensive magnitudes do constitute systems isomorphic with the system of real numbers and that they alone can be given mathematical representations. Physical quantities are supra-extensive magnitudes. In contrast, to date, psychological quantities are either intensive or extensive, but never of the third kind. This, it is felt, is the reason why mathematical representations have been few and without success in psychology as contrasted to the physical sciences. In particular, the Weber-Fechner relation is examined and shown to be invalid in two respects. It is concluded that the construction of multiplicative scales in psychology, or the equivalent use of dimensional analysis, alone will enable the development of fruitful mathematical theories in this area of investigation.The editors of this journal should perhaps point out that unanimous agreement with the arguments and points of view expressed in this article is not anticipated. They believe, however, that its publication may stimulate needed thinking and clarification of problems basic to psychological measurement and thus serve the purpose for which the journal was founded.     

Two subjective scales of number

The way in which the apparent magnitude of numbers grows with their absolute magnitude was measured with a modified version of the direct technique Marks and Slawson (1966) used to determine the psychophysical exponent for loudness. This modified technique required subjects to estimate how evenly and randomly a sequence of integers appeared to sample the numerical continuum. The results indicate that the apparent magnitude of numbers increases with a decelerated power function of their arithmetic magnitude when a series samples from an open-ended range. However, when an upper boundary of the range is specified, the subjective scale seems to be linear. Random productions of numbers parallel the results found with judgments of presented sequences. The two scales of number provide the basis for an interpretation of the difference between magnitude and category scales: that subjects use numbers differently when the response scale is open-ended Imagnitude estimation than when it has a fixed upper limit tcategory scale. Given the assumption that subjects use numbers in this way in the two tasks, the qualitative relation between magnitude and category scales is predicted.     

Assessing the Validity of Self Construal Scales

We disagree with Levine et al.'s (2003) conclusion that the 3 self construal scales currently in use have “severe” or “fatal” flaws. We argue that the results of Levine et al.'s meta‐analysis and priming studies do not raise problems with the validity of self construal scales, in part, because their results are compatible with theorizing about self construals. We also contend that Levine et al.'s measurement studies do not necessarily lead to the conclusion that there are problems with the validity of the scales (e.g., there may be multiple dimensions that form independent and interdependent self construals in second‐order factor analyses). We summarize other evidence for the construct validity of self construal scales which indicates that there are theoretically consistent findings across approximately 50 studies using the 3 scales. This would not be possible if there were major problems with the 2‐dimensional model of self construals or the scales used to measure them. We conclude that the 2‐dimensional model of self construals and the current scales are viable for use in future research.     

Acquiescence as a source of bias and model and person misfit: A theoretical and empirical analysis

In personality and attitude measurement, the presence of acquiescent responding can have an impact on the whole process of item calibration and test scoring, and this can occur even when sensible procedures for controlling acquiescence are used. This paper considers a bidimensional (content acquiescence) factor‐analytic model to be the correct model, and assesses the effects of fitting unidimensional models to theoretically unidimensional scales when acquiescence is in fact operating. The analysis considers two types of scales: non‐balanced and fully balanced. The effects are analysed at both the calibration and the scoring stages, and are of two types: bias in the item/respondent parameter estimates and model/person misfit. The results obtained theoretically are checked and assessed by means of simulation. The results and predictions are then assessed in an empirical study based on two personality scales. The implications of the results for applied personality research are discussed.     

Measurement of Trust in Salesperson–Customer Relationships in Direct Selling

No readily available scales exist that are appropriate to the measurement of trust in direct selling. This paper presents a conceptualization of trust that focuses on its nature and presents it as an affect—that is, a combination of attitudes and emotions. To develop a scale that reflects this approach, a two-country study is conducted looking at the perceptions of buyers regarding the trust they place in direct selling salespeople. Three distinct measures of trust and two related measures, based on scales developed for use in contexts other than direct selling, are utilized. The measurement properties of all five are shown to be adequate; however, differences are apparent in their robustness and in the pattern of responses in the two countries. Moreover, one measure (Trust 3) clearly is inferior to the other measures. The paper concludes with a discussion of the effectiveness of these measures of trust and suggestions for future refinements to them.     

Language Measurement Equivalence of the Ethnic Identity Scale With Mexican American Early Adolescents

The current study considers methodological challenges in developmental research with linguistically diverse samples of young adolescents. By empirically examining the cross-language measurement equivalence of a measure assessing three components of ethnic identity development (i.e., exploration, resolution, and affirmation) among Mexican American adolescents, the study both assesses the cross-language measurement equivalence of a common measure of ethnic identity and provides an appropriate conceptual and analytical model for researchers needing to evaluate measurement scales translated into multiple languages. Participants are 678 Mexican-origin early adolescents and their mothers. Measures of exploration and resolution achieve the highest levels of equivalence across language versions. The measure of affirmation achieves high levels of equivalence. Results highlight potential ways to correct for any problems of nonequivalence across language versions of the affirmation measure. Suggestions are made for how researchers working with linguistically diverse samples can use the highlighted techniques to evaluate their own translated measures.