The contribution of judgment scale to the unskilled-and-unaware phenomenon: How evaluating others can exaggerate over- (and under-) confidence

The unskilled-and-unaware phenomenon occurs when low performers tend to overestimate their performance on a task, whereas high performers judge their performance more accurately (and sometimes underestimate it). In previous research, this phenomenon has been observed for a variety of cognitive tasks and judgment scales. However, the role of judgment scale in producing the unskilled-and-unaware phenomenon has not been systematically investigated. Thus, we present four studies in which all participants judged their performance on both a relative scale (percentile rank) and an absolute scale (number correct). The studies included a variety of performance tasks (general knowledge questions, math problems, introductory psychology questions, and logic questions) and test formats (multiple-choice, recall). Across all tasks and formats, the percentile-rank judgments were less accurate than the absolute judgments, particularly for low and high performers. Furthermore, in Studies 1–3, the absolute judgments were highly accurate, even when the percentile-rank judgments were not. Thus, differences in the accuracy of percentile-rank judgments across skill levels do not always represent differences in self-awareness, but rather they may arise from difficulties that performers have at evaluating how well others are performing. Most importantly, the unskilled-and-unaware phenomenon on a relative scale does not guarantee inaccurate self-evaluations of absolute performance.     

Influence of angle of view on judges' evaluations of inverted cross in men's rings

The present study focuses on two factors that may influence the accuracy and precision of a judge's evaluation of the angular deviation error in "inverted cross" (angle of the arms away from the horizontal axis). 16 Greek judges of competitive gymnastics participated in an evaluation of the magnitude of the error in the skill. Photographs of the performance of 8 gymnasts were shown to the judges in triplicates in random order from three angles of observation (0 degrees, 45 degrees, and 90 degrees) on a computer screen. The precision in the judges' judgments was best at the 45 degrees angle of observation and at angular deviations of 16 degrees to 30 degrees. Accuracy was influenced both by the judge's angle of observation and the gymnast's angular deviation, as well as by the interaction. Overall, judges may need to improve their ability to evaluate performance through training. Training may incorporate a variety of exercises in judging the inverted cross through augmented feedback of knowledge of results.     

Fast Mapping in Healthy Young Adults: The Influence of Metamemory

Several research studies suggest the significant role played by metamemory in lexical abilities of both adults and children. To our knowledge, there have been no studies to date that have explored the role of metamemory (Judgments of Learning) in fast mapping of novel words by adults. One hundred and twelve undergraduate students were given tasks of fast mapping and judgments of learning. A one-way Multivariate Analysis of Variance revealed that participants who performed well on fast mapping had superior global and item-by-item absolute prediction scores compared to poor performers. However, the good and poor performers did not significantly differ in their item-by-item relative predictive accuracy (Goodman–Kruskal gamma correlations) in the immediate as well as the delayed judgments of learning conditions. The possible reasons for inferior gamma correlations, the methodological issues for future research and the clinical implications for the assessment and treatment of adults with lexical deficits are discussed.     

Enough skill to kill: Intentionality judgments and the moral valence of action

Extant models of moral judgment assume that an action’s intentionality precedes assignments of blame. Knobe (2003b) challenged this fundamental order and proposed instead that the badness or blameworthiness of an action directs (and thus unduly biases) people’s intentionality judgments. His and other researchers’ studies suggested that blameworthy actions are considered intentional even when the agent lacks skill (e.g., killing somebody with a lucky shot) whereas equivalent neutral actions are not (e.g., luckily hitting a bull’s-eye). The present five studies offer an alternative account of these provocative findings. We suggest that people see the morally significant action examined in previous studies (killing) as accomplished by a basic action (pressing the trigger) for which an unskilled agent still has sufficient skill. Studies 1 through 3 show that when this basic action is performed unskillfully or is absent, people are far less likely to view the killing as intentional, demonstrating that intentionality judgments, even about immoral actions, are guided by skill information. Studies 4 and 5 further show that a neutral action such as hitting the bull’s-eye is more difficult than killing and that difficult actions are less often judged intentional. When difficulty is held constant, people’s intentionality judgments are fully responsive to skill information regardless of moral valence. The present studies thus speak against the hypothesis of a moral evaluation bias in intentionality judgments and instead document people’s sensitivity to subtle features of human action.     

Olympic medals as fruits of comparison? Assimilation and contrast in sequential performance judgments

The authors investigated the evaluative consequences of sequential performance judgments. Recent social comparison research has suggested that performance judgments may be influenced by judgments about a preceding performance. Specifically, performance judgments may be assimilated to judgments of the preceding performance if judges focus on similarities between the two. If judges focus on differences, however, contrast may ensue. The authors examined sequential performance judgments, using data gathered from the 2004 Olympic Games as well as data gathered in the laboratory with students or experienced gymnastics judges as participants. Sequential performance judgments were influenced by previously judged performances, and the direction of this influence depended on the degree of perceived similarity between the successive performances.     

Overconfidence in Initial Self-Efficacy Judgments: Effects on Decision Processes and Performance

Judgments of task-specific, expected performance (i.e., self-efficacy) can affect the activities one chooses to pursue and the extent of effort devoted to these activities. However, relatively little is known about the accuracy of self-efficacy judgments or their effects on behavior, performance, and perceptions of performance in complex cognitive tasks. The results of a pilot study and experiment indicate that initial, "first-impression" self-efficacy judgments made in cognitively complex tasks are biased toward overestimates of personal ability (i.e., "overconfidence"). The experiment manipulated performance expectations to illuminate how overestimates of initial self-efficacy affect decision making. Inducing positive expectations produced overconfidence in choice accuracy, but did not increase effort, attention to strategy, or performance relative to mildly negative and strongly negative expectations. In contrast, inducing mildly negative expectations increased effort, attention to strategy, and performance relative to strongly negative expectations. The results suggest that the demotivational effects of initial negative expectations are more robust than the motivational effects of initial positive expectations. ln addition, inducing mildly negative expectations may improve performance more than positive expectations in at least some tasks and settings.     

EVALUATION OF TASK AND JOB SKILL LINKAGE JUDGMENTS USED TO DEVELOP TEST SPECIFICATIONS

The research on job analysis judgments, such as "time spent," has been relatively limited, particularly with reference to external criteria remote from the job analysis operation. The more complex job analysis judgment linking a job skill to specific tasks or duties has not been systematically examined. While it would appear that a simple scaling of importance of a skill for a task or duty or a retranslation judgment would suffice, the fact is that a single job skill may be a prerequisite for performance in a variety of tasks, and any one task may require multiple skills of varying levels for effective performance. With a multiple assignment of tasks to job skills, the evaluation becomes considerably more difficult. In the present study, a sequence of statistical evaluations was conducted to examine, first, the reliability of the subject matter expert (SME) panel's association of tasks and job skills and, second, the factor structure of the task by job skill relationship. The results are discussed with reference to developing selection test specifications and test budgets.     

Different working memory capacity in normal young adults for visual and tactile letter recognition task

Thirty-nine young adult participants performing the visual and tactile n-back working memory task were compared. In the visual task, letters were presented on a computer screen and in the tactile task, plastic letters embedded on a board were explored tactually. The amount of incorrect responses increased with increasing memory load in both tasks, but was significantly lower in the visual task. Subgrouping the participants with extreme performances into "skilled" and "poor" performers showed that the best performance was found among "skilled" visual performers, and the worst one among "poor" tactile performers. There was more interindividual variation among tactile than visual performance. We conclude that tactile working memory capacity, measured here by letter recognition and letter memory, is generally more limited and shows more variability than visual memory in normal sighted participants.     

Challenge Point: A Framework for Conceptualizing the Effects of Various Practice Conditions in Motor Learning

The authors describe the effects of practice conditions in motor learning (e.g., contextual interference, knowledge of results) within the constraints of 2 experimental variables: skill level and task difficulty. They use a research framework to conceptualize the interaction of those variables on the basis of concepts from information theory and information processing. The fundamental idea is that motor tasks represent different challenges for performers of different abilities. The authors propose that learning is related to the information arising from performance, which should be optimized along functions relating the difficulty of the task to the skill level of the performer. Specific testable hypotheses arising from the framework are also described.     

Challenge point: a framework for conceptualizing the effects of various practice conditions in motor learning

The authors describe the effects of practice conditions in motor learning (e.g., contextual interference, knowledge of results) within the constraints of 2 experimental variables: skill level and task difficulty. They use a research framework to conceptualize the interaction of those variables on the basis of concepts from information theory and information processing. The fundamental idea is that motor tasks represent different challenges for performers of different abilities. The authors propose that learning is related to the information arising from performance, which should be optimized along functions relating the difficulty of the task to the skill level of the performer. Specific testable hypotheses arising from the framework are also described.     

Learning to calibrate: Providing standards to improve calibration accuracy for different performance levels

This experimental study explores whether feedback in the form of standards helps students in giving more accurate performance estimates not only on current tasks but also on new, similar tasks and whether performance level influences the effect of standards. We provided 122 first‐year psychology students with seven texts that contained key terms. After reading each text, participants recalled the correct definitions of the key terms and estimated the quality of their recall. Half of the participants subsequently received standards and again estimated their own performance. Results showed that providing standards led to better calibration accuracy, both on current tasks and on new, similar tasks, when standards were not available yet. Furthermore, with or without standards, high performers calibrated better than low performers. However, results showed that especially low performers' calibration accuracy benefitted from receiving standards.     

Problem–Problem Solver Characteristics Affectingthe Calibration of Judgments

The purpose of this paper is to examine factors that affect the calibration of judgments by systematically comparing experts’ judgments to novices’ when solving a complex, real-world problem that varies in its initial characteristics. Calibration in this context refers to the proportion of times decision makers provide a range about their best estimates that includes the actual outcome. We found that experts specify a narrower range and provide more accurate best estimates than novices. But their tighter ranges are not justified by their greater accuracy: they are less likely to encompass the actual outcome than are novices. However, this effect is attenuated when solving more complex problems. Novices apparently underestimate the complexity of difficult problems, hence the accuracy of their best estimates decreases as does the width of their ranges, resulting in worse calibration. The performance of experts was not significantly different across problem solving conditions. Both groups provided asymmetrical ranges about their best estimates, which suggests they account for the effect of subproblem dependencies.     

Age-related similarities and differences in monitoring spatial cognition

Spatial cognitive performance is impaired in later adulthood but it is unclear whether the metacognitive processes involved in monitoring spatial cognitive performance are also compromised. Inaccurate monitoring could affect whether people choose to engage in tasks that require spatial thinking and also the strategies they use in spatial domains such as navigation. The current experiment examined potential age differences in monitoring spatial cognitive performance in a variety of spatial domains including visual–spatial working memory, spatial orientation, spatial visualization, navigation, and place learning. Younger and older adults completed a 2D mental rotation test, 3D mental rotation test, paper folding test, spatial memory span test, two virtual navigation tasks, and a cognitive mapping test. Participants also made metacognitive judgments of performance (confidence judgments, judgments of learning, or navigation time estimates) on each trial for all spatial tasks. Preference for allocentric or egocentric navigation strategies was also measured. Overall, performance was poorer and confidence in performance was lower for older adults than younger adults. In most spatial domains, the absolute and relative accuracy of metacognitive judgments was equivalent for both age groups. However, age differences in monitoring accuracy (specifically relative accuracy) emerged in spatial tasks involving navigation. Confidence in navigating for a target location also mediated age differences in allocentric navigation strategy use. These findings suggest that with the possible exception of navigation monitoring, spatial cognition may be spared from age-related decline even though spatial cognition itself is impaired in older age.     

Why people fail to recognize their own incompetence

Successful negotiation of everyday life would seem to require people to possess insight about deficiencies in their intellectual and social skills. However, people tend to be blissfully unaware of their incompetence. This lack of awareness arises because poor performers are doubly cursed: Their lack of skill deprives them not only of the ability to produce correct responses, but also of the expertise necessary to surmise that they are not producing them. People base their perceptions of performance, in part, on their preconceived notions about their skills. Because these notions often do not correlate with objective performance, they can lead people to make judgments about their performance that have little to do with actual accomplishment.     

相对到达时间任务中飞行员对客体特征与运动特征的分离

相对到达时间任务(RAT)是判断两个运动客体哪个先到达指定目标, 可用来评估个体动态空间能力。采用RAT任务对飞行员与普通被试进行对照研究, 寻求发现两组在运动客体特征和视觉空间运动特征及其相互关系上的处理差异。设计了3个实验分别考察客体颜色、客体大小、运动方向、速率大小、视线方向以及背景特征对判断的影响。结果显示：(1)客体颜色不影响运动客体的相对时间判断, 客体大小、运动方向、速率大小、视线方向以及背景特征影响判断; (2)控制组对显示屏上从左到右的运动客体的相对时间判断好于从右到左任务, 大速率任务判断更好, 对大客体快速行驶而小客体低速行驶时的相对到达时间更易区分, 且与两眼视线方向不一致的运动方向会使控制组判断更难, 运动背景中的目标线特征改变使控制组判断绩效降低; (3)和控制组比, 飞行员反应快正确率高, 其快速判断优势集中体现在从右到左运动以及小速率任务上, 且在不同运动方向和不同速率上的反应时均无差异, 飞行员的处理优势还表现在不受客体大小、视线方向改变和目标线特征改变的影响。结论：飞行员能在变化的空间中准确处理相对速度、相对距离、相对时间等运动信息, 能分离客体大小、背景、运动方向等因素对相对到达时间判断的影响, 在运动空间中飞行员具有较高场独立性认知特征和动态空间处理能力。     

The distance effect and level of expertise: Is the optimal external focus different for low-skilled and high-skilled performers?

Focusing attention on a movement effect that is farther away from the body (distal external focus) has been shown to result in more effective motor performance or learning than focusing on an effect that is in greater proximity to the body (proximal focus). The present study examined whether the distance of the external focus impacts the performance of relatively inexperienced and experienced performers differently. Low-skilled and high-skilled volleyball players passed a volleyball continuously to a target. In the proximal focus condition they were asked to concentrate on the “platform,” whereas in the distal focus condition they were instructed to concentrate on the target. The high-skilled group's accuracy scores were higher in the distal relative to proximal focus condition. However, low-skilled players' accuracy scores was greater in the proximal relative to distal focus condition. We argue that the optimal distance of the external focus depends on the level of expertise when the skill requires a specific movement technique. An external focus on that technique seems to be more advantageous for low-skilled performers. In contrast, when the movement pattern has become more automatic (high-skilled performers), a focus on the overall movement effect is more beneficial.     

Relative and absolute duration judgments under prospective and retrospective paradigms

A dual-process contingency model of short duration judgment is proposed and tested. The first process, or P(t), is a timer that uses cognitive capacity to keep track of units of time. If capacity is directed toward other tasks, P(t) will record fewer units and produce lower time judgments than when capacity is not directed toward other tasks. This timing process is most likely to affect performance when people know in advance (prospective judgments) that time judgments will be required and when absolute, rather than relative, judgments are made. The second process, or P(m), which is used for retrospective and relative judgments, judges duration on the basis of the number of remembered high priority events (HPEs) occurring during the interval. When this process is used, time judgments increase with the amount of HPEs that can be retrieved at the moment of judgment. Two experiments are reported. Tactual stimuli were presented, and nontemporal information processing load (simple or complex stimuli), type of judgment (absolute or relative), and judgment paradigm (prospective or retrospective)were manipulated. The results obtained support the proposed dual-process contingency model.     

Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments

People tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it. Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. Several analyses linked this miscalibration to deficits in metacognitive skill, or the capacity to distinguish accuracy from error. Paradoxically, improving the skills of participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities.     

Metamemory for faces, names, and common nouns

This study examined the metacognitive aspects of face–name learning with the goal of providing a comprehensive profile of monitoring performance during this task. Four types of monitoring judgments were solicited during encoding and retrieval of novel face–name associations. Across all of the monitoring judgments, relative accuracy was significantly above chance for face and name targets. Furthermore, metamemory performance was similar between both target conditions, even though names were more difficult to recognize than faces. As a preliminary test of the stability of monitoring accuracy across different categories of stimuli, we also compared metamemory performance between face–name pairs and noun–noun pairs. Prospective monitoring accuracy was similar across the categories of stimuli, but retrospective monitoring accuracy was superior for noun targets compared with face or name targets. Altogether, our results indicate that participants can monitor their memory for face–name associations at a level above chance, and retrospective monitoring is more accurate with nouns compared with faces and names.     

Social curiosity and interpersonal perception: a judge x trait interaction

The present study examined the impact of social curiosity on the utilization of social information and the accuracy of personality judgments. In total, 182 individuals who never met each other before were asked to interact for 10 minutes and afterwards to evaluate the personality (Big Five) of their interaction partner. High socially curious judges were more accurate in evaluating the degree of Extraversion and Openness of their interaction partners. Interestingly, high and low curious judges differed significantly in the utilization of verbal and nonverbal cues displayed by their interaction partner. Specifically, high socially curious judges more often used valid cues for inferring Extraversion and Openness. No differences in interpersonal accuracy and cue utilization were found for Neuroticism, Conscientiousness, and Agreeableness. The results suggest that high socially curious individuals are more accurate in judging visible traits and that this higher accuracy is grounded in a more comprehensive utilization of valid cues.