Centrality ratings,forensic relevance,and production frequency: Which one best identifies central and peripheral items?

Production frequency has often been used to identify central and peripheral information, under the assumption that high frequency implies that the item is central. However, no research to date has tested the relationship between centrality and frequency. Participants watched a video of a bank robbery and completed a free recall test, from which frequency for recalled items was computed. Two groups then watched the same video and rated centrality and forensic relevance for each item. Results showed that most, but not all, items with high frequency were rated as central and forensically relevant but that low frequency items were not diagnostic of either item centrality or forensic relevance. Forensic relevance was a better indicator of item centrality than frequency. We concluded that frequency measures should be avoided to determine centrality. Also, if centrality ratings cannot be collected, forensic relevance ratings may be more appropriate for this purpose.     

Do memorability ratings affect study-time allocation?

This research addresses the relation between predicting future memory performance (judgment of learning, or JOL) and subsequent self-paced study-time allocation. The results of three experiments support the main hypotheses: (1) recall increases with increasing JOL, (2) restudy increases JOL accuracy, and (3) study time is related to JOL. This last relation depends on the length of initial presentation time of the items. When the initial exposure trials were short, the most restudy time was allocated to the items judged hard to recall, but when the initial exposure times were long, the most restudy time was allocated to the uncertain items. Items studied longer were recalled equally well (Experiments 1 and 3) or to a lesser extent (Experiment 2) than items studied for a shorter time. It is hypothesized that during study time, subjects refine their JOLs for the items initially less well discriminated.     

Should assessment center dimension ratings be mechanically-derived?

This study examined the relationship between consensusderived and mechanically-derived assessment center ratings using a sample of 110 police candidates. It was determined that the two sets of dimension ratings were highly correlated (medianr=.83). Further, the mechanically-derived and the consensus-derived assessment ratings both correlated with the overall assessment ratings (OAR) at .71. It was estimated that the organization in this study could save $6137.92 a year, and $36.89 per assessee by eliminating the team meeting (consensus decision-making).     

Do desirability ratings moderate the validity of probability ratings on the Smoking Consequences Questionnaire-Adult? A reanalysis using regression

Smokers' responses to the Smoking Consequences Questionnaire-Adult (A. L. Copeland, T. H. Brandon, & E. P. Quinn, 1995) were used to determine whether smoking outcome expectancies are moderated by subjective evaluations (desirability ratings) in their effect on smoking-related variables. Hierarchical regression in a reanalysis of data from A. L. Copeland et al. indicated that the product of probability and desirability ratings accounted for a significant amount of variance in smoking rate, nicotine dependence, saliva cotinine, and posttreatment smoking status above that accounted for by probability and desirability ratings alone. Results indicate that desirability ratings serve as moderators to probability ratings in explaining current smoking, nicotine dependence, and continued smoking. Effects were modest in magnitude but suggest that it may be important to address subjective evaluations of outcomes in smoking cessation and relapse prevention efforts.     

What mediates gender bias in work behavior ratings?

This study explored possible mediators of gender bias in work behavior ratings. However, we believed it important to do so by first attempting to create a decision-making environment that better reflected the cognitive demands imposed on raters in work settings. Accordingly, 74 participants, mostly White and middle class students, read a vignette that depicted the work behavior of a male or female police officer. All participants attended to another task while reading the vignette and did so while under a perceived time limit; then, immediately or 5 days later, they completed a work behavior questionnaire. As expected, more effective work behaviors were attributed to men than women—but only when ratings were delayed. Further analyses revealed that a systematic response bias was responsible, whereas selective memory played no role. Specifically, participants adopted a more liberal decision criterion when attributing effective work behaviors to men than women. These results help to illuminate the dynamics of discrimination and provide direction for future research efforts.     

Body—object interaction ratings for 1,618 monosyllabic nouns

Body-object interaction (BOI) assesses the ease with which a human body can physically interact with a word’s referent. Recent research has shown that BOI influences visual word recognition processes in such a way that responses to high-BOI words (e.g., couch) are faster and less error prone than responses to low-BOI words (e.g., cliff ). Importantly, the high-BOI words and the low-BOI words that were used in those studies were matched on imageability. In the present study, we collected BOI ratings for a large set of words. BOI ratings, on a 1–7 scale, were obtained for 1,618 monosyllabic nouns. These ratings allowed us to test the generalizability of BOI effects to a large set of items, and they should be useful to researchers who are interested in manipulating or controlling for the effects of BOI. The body-object interaction ratings for this study may be downloaded from the Psychonomic Society’s Archive of Norms, Stimuli, and Data, www.psychonomic.org/archive.     

Students' ratings of teaching effectiveness: a laughing matter?

Gump in 2004 identified a positive significant relationship between awareness of daily class objectives and ratings of the instructor's overall teaching effectiveness. The idea that rating of teaching effectiveness can be related to other non-teaching related attributes of the lecturer was further examined. Correlations based on ratings of teaching effectiveness from 453 undergraduate students (M=21 yr., SD= 5.5; 73% women) showed that another nonteaching related variable, namely, how funny the instructor was perceived, was significantly related to indicators of teaching effectiveness.     

How well do ratings of academic performance by mothers and their sons correspond to grades,achievement test scores,and teachers' ratings?

Accurate perceptions of their child's academic performance are pivotal for parents' encouragement of their child's academic efforts. In research studies, parents' reports of their child's academic achievement, as well as the child's own reports, are often used as a proxy for teachers' ratings or grades. However, are parents' reports or their child's reports accurate and unbiased? We investigated whether ratings of academic performance made by mothers and their sons corresponded more closely to final grades and achievement test scores than did similar ratings made by teachers for three large samples of boys. Mothers' ratings correlated with final grades as well or better than teachers' ratings but more poorly with achievement test scores than did teachers' ratings. The seventh-grade boys' ratings correlated more poorly with both final grades and achievement test scores than did teachers' or mothers' ratings. Across all three samples combined, mothers were more likely to over-estimate rather than under-estimate the final grade. The same was true of the seventh grade sample of boys. Once the effect of final grade had been controlled, little evidence of a consistent pattern of associations between maternal demographic, parenting, and child behavior variables and either over- or underestimation was found.     

Do the effects of subjective frequency and age of acquisition survive better word frequency norms?

Megastudies with processing efficiency measures for thousands of words allow researchers to assess the quality of the word features they are using. In this article, we analyse reading aloud and lexical decision reaction times and accuracy rates for 2,336 words to assess the influence of subjective frequency and age of acquisition on performance. Specifically, we compare newly presented word frequency measures with the existing frequency norms of Ku?era and Francis (1967) Ku?era, H. and Francis, W. 1967. Computational analysis of present-day American English, Providence, RI: Brown University Press.  [Google Scholar], HAL (Burgess & Livesay, 1998 Burgess, C. and Livesay, K. 1998. The effect of corpus size in predicting reaction time in a basic word recognition task: Moving on from Kucera and Francis. Behavior Research Methods, Instruments, & Computers, 30: 272–277. [Crossref] , [Google Scholar]), Brysbaert and New (2009) Brysbaert, M. and New, B. 2009. Moving beyond Kucera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41: 977–990. [Crossref], [PubMed], [Web of Science ®] , [Google Scholar], and Zeno, Ivens, Millard, and Duvvuri (1995) Zeno, S. M., Ivens, S. H., Millard, R. T. and Duvvuri, R. 1995. The educator's word frequency guide, Brewster, NY: Touchstone Applied Science.  [Google Scholar]. We show that the use of the Ku?era and Francis word frequency measure accounts for much less variance than the other word frequencies, which leaves more variance to be “explained” by familiarity ratings and age-of-acquisition ratings. We argue that subjective frequency ratings are no longer needed if researchers have good objective word frequency counts. The effect of age of acquisition remains significant and has an effect size that is of practical relevance, although it is substantially smaller than that of the first phoneme in naming and the objective word frequency in lexical decision. Thus, our results suggest that models of word processing need to utilize these recently developed frequency estimates during training or setting baseline activation levels in the lexicon.     

Are performance appraisal ratings from different rating sources comparable?

The purpose of this study was to test whether a multisource performance appraisal instrument exhibited measurement invariance across different groups of raters. Multiple-groups confirmatory factor analysis as well as item response theory (IRT) techniques were used to test for invariance of the rating instrument across self, peer, supervisor, and subordinate raters. The results of the confirmatory factor analysis indicated that the rating instrument was invariant across these rater groups. The IRT analysis yielded some evidence of differential item and test functioning, but it was limited to the effects of just 3 items and was trivial in magnitude. Taken together, the results suggest that the rating instrument could be regarded as invariant across the rater groups, thus supporting the practice of directly comparing their ratings. Implications for research and practice are discussed, as well as for understanding the meaning of between-source rating discrepancies.     

What is this thing called frequency?

When researchers are interested in the influence of long-term knowledge on performance, printed word frequency is typically the variable of choice. Despite this preference, we know little about what frequency norms measure. They ostensibly index how often and how recently words are experienced, but words appear in context, so frequency potentially reflects an influence of connections with other words. This paper presents the results of a large free association study as well as the results of experiments designed to evaluate the hypothesis that common words have stronger connections to other words. The norms indicate that common words tend to be more concrete but they do not appear to have more associates, stronger associates, or more connections among their associates. Two extralist cued recall experiments showed that, with other attributes being equal, high- and low-frequency words were equally effective as test cues. These results suggest that frequency does not achieve its effects because of stronger or greater numbers of connections to other words, as implied in SAM. Other results indicated that common words have more connections from other words, including their associates, and that free association provides a valid index of associative strength.     

Visual long-term memory for spatial frequency?

It has been suggested that a visual long-term memory based on a sensory representation of the stimulus accounts for discrimination performance when the reference and the test stimuli are separated in time. Decision processes involved in setting response criteria, however, may also contribute to discrimination performance. In the present study, it is shown that under proper control, spatial frequency discrimination thresholds from a group of observers, each performing on a single trial, are significantly higher for a 2-h than for a 5-sec retention interval, whereas thresholds from individual observers performing in repeated trials with a 2-h retention interval are considerably lower. The results suggest that discrimination performance may depend on the retention of task-relevant information, such as a response criterion, rather than on visual memory of the stimulus. It is concluded that it is risky to postulate ahigh-fidelity long-term visual memory for spatial frequency on the basis of psychophysical group discrimination thresholds.     

Imageability and body–object interaction ratings for 599 multisyllabic nouns

We collected imageability and body–object interaction (BOI) ratings for 599 multisyllabic nouns. We then examined the effects of these variables on a subset of these items in picture-naming, word-naming, lexical decision, and semantic categorization. Picture-naming latencies were taken from the International Picture-Naming Project database (Szekely, Jacobsen, D’Amico, Devescovi, Andonova, Herron, et al. Journal of Memory and Language, 51, 247–250, 2004), word-naming and lexical decision latencies were taken from the English Lexicon Project database (Balota, Yap, Cortese, Hutchison, Kessler, Loftis, et al. Behavior Research Methods, 39, 445–459, 2007), and we collected semantic categorization latencies. Results from hierarchical multiple regression analyses showed that imageability and BOI separately accounted for unique latency variability in each task, even with several other predictor variables (e.g., print frequency, number of syllables and morphemes, age of acquisition) entered first in the analyses. These ratings should be useful to researchers interested in manipulating or controlling for the effects of imageability and BOI for multisyllabic stimuli in lexical and semantic tasks.     

Pregnancy in the workplace: Does pregnancy affect performance appraisal ratings?

Archival data were collected to examine bias effects in performance evaluation related to employee pregnancy. Prior survey research has suggested that many supervisors and co-workers hold negative views of pregnant employees. Performance appraisal ratings were collected before, during, and after pregnancy for employees taking maternity leave over a four year time frame. In addition, data was also collected from randomly selected control subjects, which were matched on job title and observation period. Contrary to expectations, performance appraisal ratings were found to increase during pregnancy when compared to before ratings and control group ratings. Potential explanations as well as suggestions for further research are discussed.     

Age-related differences in valence and arousal ratings of pictures from the International Affective Picture System (IAPS): Do ratings become more extreme with age?

The International Affective Picture System (IAPS) has been widely used in aging-oriented research on emotion. However, no ratings for older adults are available. The aim of the present study was to close this gap by providing ratings of valence and arousal for 504 IAPS pictures by 53 young and 53 older adults. Both age groups rated positive pictures as less arousing, resulting in a stronger linear association between valence and arousal, than has been found in previous studies. This association was even stronger in older than in young adults. Older adults perceived negative pictures as more negative and more arousing and positive pictures as more positive and less arousing than young adults did. This might indicate a dedifferentiation of emotional processing in old age. On the basis of a picture recognition task, we also report memorability scores for individual pictures and how they relate to valence and arousal ratings. Data for all the pictures are archived at www.psychonomic.org/archive/.     

Is there low frequency vibrotactile temporal summation?

Reaction times of subjects to threshold vibratory stimuli of various durations and frequencies were measured simultaneously with the thresholds in order to test whether more time is needed for the detection of stimuli of long than short duration at threshold intensities. Vibratory stimuli of 20 or 150 Hz frequency and 50, 150, or 300 ms duration were applied to the back of the hand. Lower detection thresholds and longer reaction times were obtained for both 20 and 150 Hz vibration with increase of stimulus duration. The results suggest that temporal summation of high frequencies is due to energy integration , whereas at low frequencies probability summation explains better the threshold decrement.     

Gender differences in supervisors’ multidimensional performance ratings: Large sample evidence

We examined gender differences in supervisor ratings of overall job performance and 37 performance dimensions. Based on data from a large, diverse sample of managers (N = 3,367) and nonmanagers (N = 9,670), we found that gender had only minimal effects on ratings for a small number of specific job performance dimensions. This was generally true regardless of whether the job performance dimension was more agentic or communal in nature, whether the job was a managerial or a nonmanagerial position, and regardless of the proportion of men or women that traditionally occupy a specific job. Overall, our results are more consistent with the gender similarities hypothesis than the agency/communion paradigm, role congruity theory, and the lack of fit model. We discuss future research avenues and implications.     

Motivation to inflate performance ratings: Perceptual bias or response bias?

In a laboratory experiment support was found for the idea that inflated performance evaluations may be due to a self-serving bias — the tendency for raters to highly evaluate subordinates' performance so that they may themselves appear to be successful. Two hundred sixteen students evaluating another's work gave higher ratings of performance quantity and quality to employees they counseled about their work than to others with whom they had either only social contact or no contact at all. This effect remained even when raters were not being evaluated by an authority figure, suggesting that the inflated ratings were the result of an interest in enhancing one's own self-image (a perceptual bias) rather than cultivating the favorable impression of another (a response bias). The conceptual and practical implications of these findings are discussed.The author acknowledges the helpful comments of Robert Heneman on an earlier draft of this article. A preliminary report of this experiment was presented at the meeting of the Academy of Management, San Diego, August 1985. This work was supported, in part, by grant INT-8304375 from the National Science Foundation.     

Reducing the performance-cue bias in work behavior ratings: can groups help?

The authors examined whether the performance-cue bias can be reduced by relying on groups as raters. Study participants (N = 333) were provided with feedback regarding the performance of a workgroup and, after observing the group, assigned to an individual or group rater condition to complete a behavioral rating instrument. Results revealed that when provided with positive (vs. negative) feedback, individuals attributed more effective and fewer ineffective behaviors to the workgroup; however, group ratings were unaffected by the feedback. In addition, feedback biased the decision criteria and false alarm rates of individuals but not of groups. Discussion of when groups may attenuate versus amplify bias in performance appraisal judgments emphasizes 2 key elements-bias magnitude and task perception.     

Do observer ratings validate self-reports of intergroup contact?: A round-robin analysis

Two studies used a round-robin design to examine whether the observers made consensual judgments of targets' degree and quality of intergroup contact, and whether these consensual judgments were correlated with the targets' own self ratings, and moderated by the observability of the contact. Study 1 revealed projection/assumed similarity, with participants rating others as similar to themselves to a large extent, but also yielded evidence for the validity of whites' self-reports of direct, but not extended, intergroup contact with Asians, even when controlling for extraversion and perceived attitudes. Study 2 replicated the main results, using both Asians and Gay men as outgroups, and showed that participants' ratings discriminated between the two discrete outgroups, with measures of contact and attitude being only meaningfully related within, but not between, outgroups. Overall, these findings help to validate self-report measures of direct intergroup contact.