Reversed better-than-average effect in direct comparisons of nonsocial stimuli depends on the set size

Studies on direct comparative judgments typically show that, for items that are positively evaluated, a single item randomly drawn from a larger set of similar items tends to be judged as better than average (the BTA effect). However, Windschitl, Conybeare, and Krizan (2008) demonstrated that, under timing conditions that do not favor focusing attention on the single item, the reversal of the BTA effect occurs. We report two experiments showing that the magnitude of the reversed BTA effect increases as a function of the size of a multiitem referent with which a single item target is compared. Specifically, in direct comparative judgments of the attractiveness of positively evaluated objects (nice-looking cloth buttons, attractive buildings, or cupcakes), underestimation of the attractiveness of singletons, as compared with a multiitem set (reversed BTA effect), increased with the increased set size. Analysis of absolute judgments obtained for singletons and for small and large multiitem sets suggests that, for attractive stimuli, both the reversed BTA effect in comparative judgments and its sensitivity to set size occur as a result of a positive relationship between set size and perceived attractiveness in absolute judgments.     

Intuitive (in)coherence judgments are guided by processing fluency, mood and affect

Recently proposed accounts of intuitive judgments of semantic coherence assume that processing fluency results in a positive affective response leading to successful assessment of semantic coherence. The present paper investigates whether processing fluency may indicate semantic incoherence as well. In two studies, we employ a new paradigm in which participants have to detect an incoherent item among semantically coherent words. In Study 1, we show participants accurately indicating an incoherent item despite not being able to provide an accurate solution to coherent words. Further, this effect is modified by affective valence of solution words that are not retrieved from memory. Study 2 replicates those results and extend them by showing that mood moderates incoherence judgments independently of affective valence of solutions. The results support processing fluency account of intuitive semantic coherence judgments and show that it is not fluency per se but fluency variations that drive judgments.     

Variation of Preference Inconsistency When Applying Ratio and Interval Scale Pairwise Comparisons

Several studies on numerical rating in discrete choice problems address the tendency of inconsistencies in decision makers' measured preferences. This is partly due to true inconsistencies in preferences or the decision makers' uncertainty on what he or she really wants. This uncertainty may be reflected in the elicited preferences in different ways depending on the questions asked and methods used in deriving the preferences for alternatives. Some part of the inconsistency is due to only having a discrete set of possible judgments. This study examined the variation of preference inconsistency when applying different pairwise preference elicitation techniques in a five‐item discrete choice problem. The study data comprised preferences of five career alternatives elicited applying interval scale and numerically and verbally anchored ratio scale pairwise comparisons. Statistical regression technique was used to analyse the differences of inconsistencies between the tested methods. The resulting relative residual variances showed that the interval ratio scale comparison technique provided the greatest variation of inconsistencies between respondents, thus being the most sensitive to inconsistency in preferences. The numeric ratio scale comparison gave the most uniform preferences between the respondents. The verbal ratio scale comparison performed between the latter two when relative residual variances were considered. However, the verbal ratio scale comparison had weaker ability to differentiate the alternatives. The results indicated that the decision recommendation may not be sensitive to the selection between these preference elicitation methods in this kind of five‐item discrete choice problem. The numeric ratio scale comparison technique seemed to be the most suitable method to reveal the decision makers' true preferences. However, to confirm this result, more studying will be needed, with an attention paid to users' comprehension and learning in the course of the experiment. Copyright © 2013 John Wiley & Sons, Ltd.     

Scale values derived from the method of choices

In many a psychometric problem, particularly in practical problems, the only data conveniently obtainable are the first choices given to certain stimuli among a list of stimuli that are available for selection. By assuming that such choices represent real comparative judgments, we may apply Thurstone's law of comparative judgments, extract experimental proportions from the numbers of first choices, and compute psychological scale values for the stimuli. Two procedures are proposed for estimating such proportions and examples of their applications are given. A procedure for allocating a meaningful zero point on the scale by the use of absolute judgments is explained and demonstrated. Suggestions are added for overcoming certain weaknesses and limitations of the method of choices.     

JOB CLASSIFICATION APPROACHES AND THE IMPLEMENTATION OF VALIDITY GENERALIZATION RESULTS

This paper compares two job classification methods for showing the appropriateness of cognitive tests in settings that were not involved in supplying data for a validity generalization analysis. One approach was an elaborate quantitative procedure that involved a lengthy job inventory and a multivariate item analysis. This approach was shown to be highly successful when applied to the responses from 1179 job inventories collected in 54 petroleum-petrochemical plants from 30 different companies. The other procedure involved simple job classification judgments by supervisors and incumbents. This latter approach was shown to be as effective, but was much less time consuming and costly. Professional and legal implications of these findings are discussed.     

Bisection of loudness

In a loudness bisection task, subjects varied one sound to lie halfway between two given sounds in terms of loudness. The two given sounds were varied from 30 to 90 dB in a 4 by 9 factorial design. Functional measurement methods based on monotone analysis provided good support for the bisection model, and yielded a loudness scale with an exponent of about .3, except for a falloff at lower intensities. Two other tasks, judging average loudness and difference in loudness of the two given sounds, yielded mixed results. In Experiment 2, in particular, the differencing judgments were not additive, even under monotone transformation. These analyses also indicated that previous applications of monotone analysis have typically lacked adequate power to allow any conclusion about the operative model. Overall, the present bisection scale agrees with Garner’s lambda scale, and the present theoretical approach agrees with that of Garner in its emphasis on algebraic models as a foundation for psychological measurement.     

Development and Testing of an Abbreviated Numeracy Scale: A Rasch Analysis Approach

Research has demonstrated that individual differences in numeracy may have important consequences for decision making. In the present paper, we develop a shorter, psychometrically improved measure of numeracy—the ability to understand, manipulate, and use numerical information, including probabilities. Across two large independent samples that varied widely in age and educational level, participants completed 18 items from existing numeracy measures. In Study 1, we conducted a Rasch analysis on the item pool and created an eight‐item numeracy scale that assesses a broader range of difficulty than previous scales. In Study 2, we replicated this eight‐item scale in a separate Rasch analysis using data from an independent sample. We also found that the new Rasch‐based numeracy scale, compared with previous measures, could predict decision‐making preferences obtained in past studies, supporting its predictive validity. In Study, 3, we further established the predictive validity of the Rasch‐based numeracy scale. Specifically, we examined the associations between numeracy and risk judgments, compared with previous scales. Overall, we found that the Rasch‐based scale was a better linear predictor of risk judgments than prior measures. Moreover, this study is the first to present the psychometric properties of several popular numeracy measures across a diverse sample of ages and educational level. We discuss the usefulness and the advantages of the new scale, which we feel can be used in a wide range of subject populations, allowing for a more clear understanding of how numeracy is associated with decision processes. Copyright © 2012 John Wiley & Sons, Ltd.     

Constructing personality scales under the assumptions of an ideal point response process: toward increasing the flexibility of personality measures

The main aim of this article is to explicate why a transition to ideal point methods of scale construction is needed to advance the field of personality assessment. The study empirically demonstrated the substantive benefits of ideal point methodology as compared with the dominance framework underlying traditional methods of scale construction. Specifically, using a large, heterogeneous pool of order items, the authors constructed scales using traditional classical test theory, dominance item response theory (IRT), and ideal point IRT methods. The merits of each method were examined in terms of item pool utilization, model-data fit, measurement precision, and construct and criterion-related validity. Results show that adoption of the ideal point approach provided a more flexible platform for creating future personality measures, and this transition did not adversely affect the validity of personality test scores.     

Direct-comparison judgments: when and why above- and below-average effects reverse

Above-average and below-average effects appear to be common and consistent across a variety of judgment domains. For example, several studies show that individual items from a high- (low-) quality set tend to be rated as better (worse) than the other items in the set (e.g., E. E. Giladi & Y. Klar, 2002). Experiments in this article demonstrate reversals of these effects. A novel account is supported, which describes how the timing of the denotation of the to-be-judged item influences attention and ultimately affects the size or direction of comparative biases. The authors discuss how this timing account is relevant for many types of referent-dependent judgments (e.g., probability judgments, resource allocations) and how it intersects with various accounts of comparative bias (focalism, generalized-group, compromise between local and general standards [LOGE]).     

Do different metamemory judgments tap the same underlying aspects of memory?

We compared the predictions from several kinds of metamemory judgments (on the same set of items), both in terms of their predictive accuracy and in terms of the commonality of predictions. Undergraduates made judgments about the ease with which they could learn each item in a list (ease-of-learning judgments); then they learned every item, either to a minimal criterion of learning or with overlearning, and made judgments about how well they knew each item (judgments of knowing); finally, they returned 4 weeks later for a retention session and made feeling-of-knowing judgments on every time they could not recall, after which a recognition test assessed predictive accuracy. Ease-of-learning judgments had the least predictive accuracy. Surprisingly, however, the recognition of nonrecalled items was predicted equally well by judgments of knowing (made 4 weeks earlier) as by feeling-of-knowing judgments (made immediately prior to recognition). Moreover, those two kinds of judgments were only weakly correlated with each other, which implies that they do not tap memory in the same way.     

Schematic knowledge changes what judgments of learning predict in a source memory task

Source monitoring can be influenced by information that is external to the study context, such as beliefs and general knowledge (Johnson, Hashtroudi, & Lindsay, 1993). We investigated the extent to which metamnemonic judgments predict memory for items and sources when schematic information about the sources is or is not provided at encoding. Participants made judgments of learning (JOLs) to statements presented by two speakers and were informed of the occupation of each speaker either before or after the encoding session. Replicating earlier work, prior knowledge decreased participants’ tendency to erroneously attribute statements to schematically consistent but episodically incorrect speakers. The origin of this effect can be understood by examining the relationship between JOLs and performance: JOLs were equally predictive of item and source memory in the absence of prior knowledge, but were exclusively predictive of source memory when participants knew of the relationship between speakers and statements during study. Background knowledge determines the information that people solicit in service of metamnemonic judgments, suggesting that these judgments reflect control processes during encoding that reduce schematic errors.     

Item Characteristic Curve Estimation of Signal Detection Theory-Based Personality Data: A Two-Stage Approach to Item Response Modeling

Signal Detection Theory (SDT; MacMillan & Creelman, 1991) is a method of data collection that has been used for several years, which describes the decision-making strategies of individuals. However, its use has been largely restricted to experiments involving sensation and perception. The Overclaiming Questionnaire (OCQ; Paulhus & Bruce, 1990) is a scale that has been developed to measure intellectual ability and personality, using SDT as a guideline. Although the scale has been successful in measuring human characteristics such as narcissism and intelligence, it is still unclear how to measure the characteristics of the various stimuli used (e.g., item difficulty, item discrimination, etc.). In some ways, this is a direct consequence of the general lack of research involved in item parameter estimation in the field of SDT. Using the OCQ, this article presents a graphical and nonparametric form of item response modeling to address this issue. In many ways, the approach is influenced by and structured around item response theory (IRT; Hambleton, Swaminathan, & Rogers, 1991). The general features of both SDT and IRT are described. Results suggest that this method is indeed a reasonable approach to describing item functioning, and there are several advantages to using this method over traditional IRT methods. Furthermore, SDT appears to be a fruitful approach to assessing intelligence, ability, and other psychological constructs, with advantages over traditional approaches. Overall, the results provide interesting implications for item selection and test development in several scientific and academic fields.     

Test context affects recollection and familiarity ratings: implications for measuring recognition experiences

The binary remember/know task requires participants to dichotomize their subjective recognition experiences into those with recollection and those only with familiarity. Many variables have produced dissociative effects on remember/know judgments. In contrast, having participants make independent recollection/familiarity ratings has consistently produced parallel effects, suggesting the dissociations may be artifacts of using binary judgments. Bodner and Lindsay (2003) reported a test-list context effect with binary judgments: Increased remembering but decreased knowing for a set of critical items tested with a set of less-memorable (vs. more-memorable) items. Here we report a parallel effect of test-list context on recollection and familiarity ratings, induced by a shift in response bias. We argue that independent ratings are preferable to binary judgments because they allow participants to directly report the co-occurrence of recollection and familiarity for each item. Implications for the measurement of self-reported recognition experiences, and for accounts of recognition memory, are discussed.     

Developmental changes in memorial comparisons: the effects of stimulus presentation mode.

First graders, fifth graders, and college students made comparative size judgments of either pictures (line drawings) or names (spoken words) of common objects by designating the "bigger" item in real life. Care was taken to equate the picture and word conditions on a number of critical parameters including method of item-pair presentation and activation of response-time intervals. All groups exhibited a symbolic distance effect. While judgments were faster with pictures than words, the magnitude of the difference did not change with age. Previous research suggesting a marked developmental decline in the magnitude of the "pictorial superiority effect" may have confounded reduced memory demands with stimulus presentation mode for young children. Finally, slopes of the symbolic distance functions were found to decrease with increasing grade level, at least from first to fifth grade. This is the first demonstration of an age-related decline in slopes for magnitude comparisons of concrete objects.     

ESTIMATING CONTENT-ACQUIESCENCE CORRELATION BY COVARIANCE STRUCTURE ANALYSIS

A topic of continuing interest in the measurement area is response acquiescence. A recent study has demonstrated the feasibiliy of studying acquiescence or, more importantly, content/acquiescence correlation in the MMPI. Utilizing the components of variance approach, this study found that the variance due to acquiescence in scores on the Pt and Hg scales was small relative to content variance, but that the correlation between acquiescence and content may be substantial for the Pt scale. The present paper describes a general statistical procedure for investigating content variance, variance due to non-content characteristics of items, and the covariances of content and various item characteristics. The data from a previous paper are reanalyzed, using alternative covariance structure models. Maximum likelihood procedures which allow for a statistical test for parameters of interest are used. The results point to the significance of the content- acquiescence correlation in the Pt scale, but not in the Hy scale. The previous findings are verified statistically, and procedures which hold promise for other investigation into the properties of behavioral tests are described.     

A Factor Analysis Of Individual Social Desirability Scale Values

The factorial structure of individual judgments of social desirability scale value (SDSV) was studied by factor analyzing a 112 (college students) by 100 (MMPI items) data matrix of individual judgments of SDSV in two ways. One analysis was factoring by item (R technique) while the second analysis was factoring by subject (Q technique). A large general SD factor appeared when subject intercorrelations were factored, but no such general factor was identified when item intercorrelations were factored. Evidence was also presented that the large general SD factor was highly correlated with mean or normative SDSVs.     

Perceived differences between chimpanzee (Pan troglodytes) and human (Homo sapiens) facial expressions are related to emotional interpretation

Human face perception is a finely tuned, specialized process. When comparing faces between species, therefore, it is essential to consider how people make these observational judgments. Comparing facial expressions may be particularly problematic, given that people tend to consider them categorically as emotional signals, which may affect how accurately specific details are processed. The bared-teeth display (BT), observed in most primates, has been proposed as a homologue of the human smile (J. A. R. A. M. van Hooff, 1972). In this study, judgments of similarity between BT displays of chimpanzees (Pan troglodytes) and human smiles varied in relation to perceived emotional valence. When a chimpanzee BT was interpreted as fearful, observers tended to underestimate the magnitude of the relationship between certain features (the extent of lip corner raise) and human smiles. These judgments may reflect the combined effects of categorical emotional perception, configural face processing, and perceptual organization in mental imagery and may demonstrate the advantages of using standardized observational methods in comparative facial expression research.     

Anchor effects using numerical estimates of simple dot patterns

Shifts due to anchor stimuli using absolute judgments have been ascribed to “semantic” and “scale modulus” changes. To test this explanation, random dot patterns of 10, 12, 14, 16 and 18 dots were exposed for 0.30 sec with anchors of 4, 13 or 32 dots preceding each of the stimuli. In the control only the series of stimuli were judged. It was found that numerical estimates of the number of dots increased with small anchor, decreased with large anchor, and were not significantly changed with anchor in the vicinity of AL. These results exactly parallel those found using methods of absolute and comparative ratings and hence cannot be ascribed to semantic shifts especially in view of the fact that independent groups of Ss served in the four anchor and control conditions. Since the anchor effects were significant but not so large that they can be ascribed to change of scale modulus this theory must also be rejected. The AL model seems to offer the simplest and most unitary explanation of these as well as many other results obtained with different types of stimuli and different methods of judging.     

Toward a New Method of Detecting Deliberately Faked Personality Tests: The use of idiosyncratic item responses

This study presents a new method for developing faking detection scales based on idiosyncratic item‐response patterns. Two scoring schemes based on this approach strongly differentiated between scores obtained under honest vs directed faking conditions in cross‐validation samples (r_pb=.45 and .67). This approach is shown to successfully classify between 20% and 37% of faked personality measures with only a 1% false positive rate in a sample comprised of 56% honest responses. Of equal importance, this method does not result in a scale that meaningfully correlates with personality or cognitive ability tests. This study raises many questions about both the source and generalizabiltiy of the effect. Key directions for future research and improved scale development that may limit or enhance the utility of the idiosyncratic item‐response method are discussed.     

学习和测验经验对学习判断中UWP效应的影响

针对重复学习判断中出现的练习伴随低估效应（UWP效应）,目前存在多种理论解释。本研究基于过去测验记忆假说,在学习阶段和测验阶段引入两种判断：学习判断准确性的判断和回溯性信心判断,通过两个实验考察学习经验和测验经验对UWP效应的影响。结果发现：在学习判断中学习判断准确性的判断和回溯性信心判断均消除了UWP效应,间接证明了学习和测验经验均影响到UWP效应的出现。