PROGRESS IN PSYCHOMETRICS: CAN INDUSTRIAL AND ORGANIZATIONAL PSYCHOLOGY CATCH UP?

Murphy and DeShon (2000) show that interrater correlations do not provide reasonable estimates of the reliability of job performance ratings, and suggest that better estimates can be obtained by applying the methods of generalizability theory. Schmidt, Viswesvaran, and Ones (2000) criticize our suggestions as radical, and argue that: (a) the reliability of ratings should be evaluated using the parallel test model rather than the more general and more realistic generalizability model, (b) reliability and validity are distinct concepts that should not be confused, and (c) measurement models have little to do with substantive models of the processes that generate scores on a test or measure. All three of these ideas were once part of the psychometric mainstream, but progress in psychometrics over the last 3 decades has moved the field well beyond these assumptions and approaches. Modern psychometric theory calls for close linkages between measurement models and substantive models of the phenomena being measured.     

The Creation Process of Brazilian Musicians

This study investigated the creative process of well‐known musicians in Brasília, Brazil and the conditions that facilitated the development of their creative abilities. A sample of 14 professional musicians (11 composers and 3 performers) participated in semi‐structured interviews to discuss the following topics: musical formation, dedicating time to music, working conditions, creativity in music, the creative process, and the importance of music in their lives. The results indicated that the majority of the musicians view creativity as a complex process supported by many factors, especially the environment. Four stages of the creative process were described by these musicians: process triggering, preparation, product elaboration, and product validation.     

Receiver operating characteristics for the brief symptom inventory depression, paranoid ideation, and psychoticism scales in a large sample of clinical inpatients

A commonly used screening tool for psychopathology, the Brief Symptom Inventory, provides normative data for assessing current mental functioning across multiple domains. Using data from 654 psychiatric inpatients, receiver operating characteristic (ROC) analyses were conducted for three scales, Depression, Paranoid Ideation, and Psychoticism. t ratios identified significant group differences on the Depression scale between patients diagnosed with or without depression but no differences on the Paranoid Ideation and Psychoticism scales between patients diagnosed with or without schizophrenia. Area under the curve for Depression was .65, indicating that the scale improved diagnostic prediction somewhat beyond chance; for Paranoid Ideation, the area was .52 and for Psychoticism, the area was .53, indicating that these two scales did not significantly improve diagnostic prediction beyond chance.     

Test context affects recollection and familiarity ratings: implications for measuring recognition experiences

The binary remember/know task requires participants to dichotomize their subjective recognition experiences into those with recollection and those only with familiarity. Many variables have produced dissociative effects on remember/know judgments. In contrast, having participants make independent recollection/familiarity ratings has consistently produced parallel effects, suggesting the dissociations may be artifacts of using binary judgments. Bodner and Lindsay (2003) reported a test-list context effect with binary judgments: Increased remembering but decreased knowing for a set of critical items tested with a set of less-memorable (vs. more-memorable) items. Here we report a parallel effect of test-list context on recollection and familiarity ratings, induced by a shift in response bias. We argue that independent ratings are preferable to binary judgments because they allow participants to directly report the co-occurrence of recollection and familiarity for each item. Implications for the measurement of self-reported recognition experiences, and for accounts of recognition memory, are discussed.     

Relations of psychological distress with objective individual,family, and neighborhood characteristics of urban adolescents

This paper describes the relations of psychological distress with 11 ecological factors (5 community, 4 family, and 2 individual) among 423 older urban adolescents who are predominantly people of color. Data were obtained by means of self-report via group-administered questionnaires given between 1994 and 1996; most scales are multi-item. There is wide variation in distress. There are very few statistically significant relationships between the ecological factors and distress, and those are negligible. Girls whose parents have less education and live in extended families have higher distress; family and individual domains are correlated .04 and .03 with distress and the community domain .00, so this ecological model accounts for a mere 7%, of the common total variance in distress.     

What kind of love is love at first sight? An empirical investigation

Love at first sight (LAFS) is a commonly known phenomenon, but has barely been investigated scientifically. Major psychological theories of love predict that LAFS is marked by high passion. However, it could also be a memory confabulation construed by couples to enhance their relationship. We investigated LAFS empirically by assessing feelings of love at the moment participants met potential partners for the first time. Data were collected from an online study, a laboratory study, and three dating events. Experiences of LAFS were marked neither by high passion, nor by intimacy, nor by commitment. Physical attraction was highly predictive of reporting LAFS. We therefore suggest that LAFS is not a distinct form of love, but rather a strong initial attraction that some label as LAFS, either in the moment of first sight or retrospectively.     

The Influence of Child-Directed Speech on Word Learning and Comprehension

This paper describes an investigation into the function of child-directed speech (CDS) across development. In the first experiment, 10–21-month-olds were presented with familiar words in CDS and trained on novel words in CDS or adult-directed speech (ADS). All children preferred the matching display for familiar words. However, only older toddlers in the CDS condition preferred the matching display for novel words. In Experiment 2, children 3–6 years of age were presented with a sentence comprehension task in CDS or ADS. Older children performed better overall than younger children with 5- and 6-year-olds performing above chance regardless of speech condition, while 3- and 4-year-olds only performed above chance when the sentences were presented in CDS. These findings provide support for the theory that CDS is most effective at the beginning of acquisition for particular constructions (e.g. vocabulary acquisition, syntactic comprehension) rather than at a particular age or for a particular task.     

Predicting Successful Responses to Emergencies: the Emergency Responsiveness Scale

People respond to disasters in a variety of ways. Some individuals respond to disasters or emergencies with panicked activity while others mobilize resources without panicking. In this article, we propose that the individual response to emergencies is determined by Emergency Responsiveness, the manner in which an individual appraises an emergency situation. We believe Emergency Responsiveness consists of four factors: Risk Taking, Experience with Emergencies, Emergency Anxiety, and Emergency Preparation. In Study One, each of the four subscales of Emergency Responsiveness correlated with personality traits in the hypothesized directions, providing evidence of construct validity. In Study Two, we establish criterion-related validity by demonstrating that a sample of first-responders scored significantly higher than a control sample on all four dimensions. Our results support our hypothesis that Emergency Responsiveness is a measurable individual difference. Implications for first responders and emergency services are discussed.     

Engagement Attentional Bias toward Value-Associated Stimuli

Recent evidence has shown that learning to associate the value of behavioral outcomes with specific stimuli (i.e., value learning/association) can induce attentional capture in a subsequent visual search task. Here we investigated the component of attentional bias toward value-associated stimuli in the dot-probe paradigm. In the training phase, participants learned to associate nonsense shape outlines with monetary wins or losses. The shapes were then used as cues in an explicit no feedback dot-probe task. Probe appeared at either the salient (congruent trial) or nonsalient location (incongruent trial). Results indicate attentional bias toward value-associated stimuli, futher more this attentional bias is driven by an engagement bias.     

Easing the Inferential Leap in Competency Modelling: The Effects of Task‐related Information and Subject Matter Expertise*

Despite the rising popularity of the practice of competency modeling, research on competency modeling has lagged behind. This study begins to close this practice–science gap through 3 studies (1 lab study and 2 field studies), which employ generalizability analysis to shed light on (a) the quality of inferences made in competency modeling and (b) the effects of incorporating elements of traditional job analysis into competency modeling to raise the quality of competency inferences. Study 1 showed that competency modeling resulted in poor interrater reliability and poor between‐job discriminant validity amongst inexperienced raters. In contrast, Study 2 suggested that the quality of competency inferences was higher among a variety of job experts in a real organization. Finally, Study 3 showed that blending competency modeling efforts and task‐related information increased both interrater reliability among SMEs and their ability to discriminate among jobs. In general, this set of results highlights that the inferences made in competency modeling should not be taken for granted, and that practitioners can improve competency modeling efforts by incorporating some of the methodological rigor inherent in job analysis.