The use of the psychological stress evaluator with stutterers

A lie detection instrument, the Psychological Stress Evaluator, has been reported by the manufacturer as being capable of quantifying changes in the degree of “stress” as reflected by a subject's voice. Speech pathologists might consider the use of the instrument in clinical and research activities concerning the stutterer's emotional (stress) reaction to stuttering.To test the validity of the instrument, four stutterers and four normal speakers were recorded while making consecutive telephone calls. Randomized recordings were analyzed by a company trained technical to determine the degree of stress for each speech sample.Results indicated that both groups had a significant reduction of stress from the first to the fourth telephone call. However, the groups did not differ in the degree of stress during the ordered series of calls. Due to the inability of the Psychological Stress Evaluator to differentiate between the groups, the validity of the instruments is questioned.     

INTERRATER CORRELATIONS DO NOT ESTIMATE THE RELIABILITY OF JOB PERFORMANCE RATINGS

Interrater correlations are widely interpreted as estimates of the reliability of supervisory performance ratings, and are frequently used to correct the correlations between ratings and other measures (e.g., test scores) for attenuation. These interrater correlations do provide some useful information, but they are not reliability coefficients. There is clear evidence of systematic rater effects in performance appraisal, and variance associated with raters is not a source of random measurement error. We use generalizability theory to show why rater variance is not properly interpreted as measurement error, and show how such systematic rater effects can influence both reliability estimates and validity coefficients. We show conditions under which interrater correlations can either overestimate or underestimate reliability coefficients, and discuss reasons other than random measurement error for low interrater correlations.     

Channels and order of report in dichotic memory*

Experiments were conducted in which Ss received to-be-remembered sequences of two, three, or four simultaneous pairs of digits. Both digits of each pair were recorded by the same male speaker and both were presented binaurally, thus eliminating cues of spatial location and voice by which Ss could “channel” their reports. Even in the absence of these stimulus channels, Ss reported the digits sequentially. High bias ratings in the first experiment suggested the possibility that sequential reports were induced by uncontrolled stimulus characteristics (e.g., temporal synchrony, intensity, and pitch). Pulse-coded speech stimuli, which provided greater control over nonlinguistic stimulus features, were used in the terminal experiment. Bias ratings were reduced, but the majority of Ss continued to report sequentially. These results suggest that the presence of stimulus channels is not a necessary condition for the occurrence of sequential reporting.     

Fatigue estimation using voice analysis

In the present article, we present a means to remotely and transparently estimate an individual's level of fatigue by quantifying changes in his or her voice characteristics. Using Voice analysis to estimate fatigue is unique from established cognitive measures in a number of ways: (1) speaking is a natural activity requiring no initial training or learning curve, (2) voice recording is a unobtrusive operation allowing the speakers to go about their normal work activities, (3) using telecommunication infrastructure (radio, telephone, etc.) a diffuse set of remote populations can be monitored at a central location, and (4) often, previously recorded voice data are available for post hoc analysis. By quantifying changes in the mathematical coefficients that describe the human speech production process, we were able to demonstrate that for speech sounds requiring a large average air flow, a speaker's voice changes in synchrony with both direct measures of fatigue and with changes predicted by the length of time awake.     

A Cross-Modal Comparison of Telephone and Face-to-Face Selection Interviews in Graduate Recruitment

Although there has been an increase in the use of telephone interviews for graduate recruitment by companies in the UK, there is little evidence attesting to their equivalence with traditional face-to-face selection interviews. A total of 70 candidates applying to a multinational oil corporation received both face-to-face and telephone interviews as the first stage of the 1996 graduate recruitment milkround. Group A (N = 41) received an initial face-to-face interview followed by a telephone interview and group B (N = 29) a telephone interview followed by a face-to-face interview. Findings indicate that candidates received significantly lower ratings when interviewed by telephone than when interviewed face-to-face (p ≤ 0.001). A significant interaction was also found (p ≤ 0.05) with candidates who received face-to-face interviews following telephone interviews demonstrating improved performance in their face-to-face interviews. The practical implications of these findings for companies wishing to use telephone interviews are discussed.     

The voice of confidence: Paralinguistic cues and audience evaluation

A standard speaker read linguistically confident and doubtful texts in a confident or doubtful voice. A computer-based acoustic analysis of the four tapes showed that paralinguistic confidence was expressed by increased loudness of voice, rapid rate of speech, and infrequent, short pauses. Under some conditions, higher pitch levels and greater pitch and energy fluctuations in the voice were related to paralinguistic confidence. In a 2 × 2 design, observers perceived and used these cues to attribute confidence and related personality traits to the speaker. Both text and voice cues are related to confidence ratings; in addition, the two types of cue are related to differing personality attributes.     

Estimating Coefficients of Equivalence and Stability for Job Performance Ratings: The importance of controlling for transient error on criterion measurement

Previous research on measurement error in job performance ratings estimated reliability using coefficients: alpha, test–retest, and interrater correlation. None of these three coefficients control for the four main sources of error in performance ratings. For this reason, coefficient of equivalence and stability (CES) has been suggested as the ideal estimate of reliability. This article presents the estimates of CES for a time interval of 1, 2, and 3 years. The values obtained for a single rater were .51, .48, and .44, respectively. For two raters, the values were .59, .55, and .51. The findings suggest that previous reliability estimates based on alpha, test–retest, and interrater coefficients overestimated the reliability of job performance ratings. In the present study, the interrater coefficient overestimates reliability by 13.6–25.4% for an interval time of 1–3 years, as it does not control for transient error. Results also showed that the importance of transient error increases as the length of the interval between the measures increases. Based on the results, it is suggested that corrected validities based on interrater reliability underestimate the magnitude of the validity. The implications of these findings for future efforts to estimate criterion reliability and predictor validity are discussed.     

Technologies,Multitasking, and Driving: Attending to and Preparing for a Mobile Phone Conversation in a Car

This article investigates mobile phone calls initiated or received by drivers and passengers in cars and focuses on the participants' actions before the telephone conversation proper. Drawing on video‐recorded data of real driving situations, and building on conversation analysis and multimodal interaction analysis, this article discusses how participants temporally and sequentially coordinate situations that require multitasking, that is, use a phone while on the move. This article shows how participants draw on the current social‐interactional, material context to handle the call as relevant at that point and how they, through their vocal and bodily conduct, manage the prebeginning as a collaborative effort. The findings have relevance for research both on driving and on human–human and human–technology interaction.     

Utility of Behavior Ratings by Examiners During Assessments of Preschool Children With Attention-Deficit/Hyperactivity Disorder

This study examines the clinical utility of behavior ratings made by nonclinician examiners during assessments of preschool children with Attention-Deficit/Hyperactivity Disorder (AD/HD). Matched samples of children with (n = 127) and without (n = 125) AD/HD were utilized to test the internal, convergent, concurrent, and incremental validity of ratings completed by examiners on the Hillside Behavior Rating Scale (HBRS). Results indicated that HBRS ratings were internally consistent, possessed sufficient interrater reliability, and were significantly associated with parent and teacher reports of AD/HD when controlling for age, gender, intelligence, and symptoms of other psychopathology. HBRS ratings also were significantly associated with other measures of functioning, and provided a significant increment in the prediction of impairment over parent and teacher report alone. These findings suggest that behavioral ratings during testing provide a unique source of clinical information that may be useful as a supplement to parent and teacher reports.     

Earwitnesses: effects of accent,retention and telephone

An experiment was conducted to investigate the effects of accent, telephone and a relatively long retention interval (3 or 8 weeks) on speaker identification. Three‐hundred and sixty participants heard the target's voice and were asked to identify the target by means of a line‐up consisting of 6 voices. Half of the participants were given a target‐present line‐up and the other half a target‐absent line‐up. The results showed that 24% of participants correctly identified the target in the target‐present condition (hits), whereas 50% of participants incorrectly identified a person as the target in the target‐absent condition (false alarms). The speaker with the standard‐accented voice was more often correctly recognized than the speaker with the non‐standard‐accented voice. No difference was found between identification accuracy after one, three or eight weeks and between the telephone and non‐telephone conditions. It can be concluded that there is a relatively high probability that an innocent defendant is identified as the perpetrator, even in a procedurally correct voice line‐up (in this experiment 8%). Furthermore, reliability may be drastically reduced when the perpetrator has a strong accent, unfamiliar to the listener. On the other hand, reliability of a voice line‐up seems not to be affected by a presentation over the telephone, as well as by a retention interval of at least 8 weeks. Copyright © 2006 John Wiley & Sons, Ltd.     

Examining the reliability and predictive validity of performance assessments by soccer coaches and scouts: The influence of structured collection and mechanical combination of information

Soccer coaches and scouts typically assess in-game soccer performance to predict players’ future performance. However, there is hardly any research on the reliability and predictive validity of coaches’ and scouts’ performance assessments, or on strategies they can use to optimize their predictions. In the current study, we examined whether robust principles from psychological research on selection – namely structured information collection and mechanical combination of predictor information through a decision-rule – improve soccer coaches’ and scouts’ performance assessments. A total of n = 96 soccer coaches and scouts participated in an elaborate within-subjects experiment. Participants watched soccer players’ performance on video, rated their performance in both a structured and unstructured manner, and combined their ratings in a holistic and mechanical way. We examined the inter-rater reliability of the ratings and assessed the predictive validity by relating the ratings to players’ future market values. Contrary to our expectations, we did not find that ratings based on structured assessment paired with mechanical combination of the ratings showed higher inter-rater reliability and predictive validity. In contrast, unstructured-holistic ratings yielded the highest reliability and predictive validity, although differences were marginal. Overall, reliability was poor and predictive validities small-to-moderate, regardless of the approach used to rate players’ performance. The findings provide insights into the difficulty of predicting future performance in soccer.     

The impact of job complexity and performance measurement on the temporal consistency, stability, and test-retest reliability of employee job performance ratings

Although research has shown that individual job performance changes over time, the extent of such changes is unknown. In this article, the authors define and distinguish between the concepts of temporal consistency, stability, and test-retest reliability when considering individual job performance ratings over time. Furthermore, the authors examine measurement type (i.e., subjective and objective measures) and job complexity in relation to temporal consistency, stability, and test-retest reliability. On the basis of meta-analytic results, the authors found that the test-retest reliability of these ratings ranged from .83 for subjective measures in low-complexity jobs to .50 for objective measures in high-complexity jobs. The stability of these ratings over a 1-year time lag ranged from .85 to .67. The analyses also reveal that correlations between performance measures decreased as the time interval between performance measurements increased, but the estimates approached values greater than zero.     

MEASUREMENT ERROR IN RESEARCH ON THE HUMAN RESOURCES AND FIRM PERFORMANCE RELATIONSHIP: FURTHER EVIDENCE AND ANALYSIS

Our earlier article in Personnel Psychology demonstrated how general-izability theory could be used to obtain improved reliability estimates in the human resource (HR) and firm performance literature and that correcting for unreliability using these estimates had important implications for the magnitude of the HR and firm performance relationship. In their comment, Huselid and Becker both raise criticisms specific to our study and broad issues for the field to consider. In our present article, we argue, using empirical evidence whenever possible, that the issues and criticisms raised by Huselid and Becker do not change our original conclusions. We also provide new evidence on how the reliability of HR-related measures may differ at different levels of analysis. Finally, we build on Huselid and Becker's helpful discussion of broad research design and strategy issues in the HR and firm performance literature in an effort to help researchers make better informed choices regarding their own research designs and strategies in the area.     

Reliability of heterosocial skills measurement with sex offenders

The heterosocial skills of 79 sex offenders were assessed using two self-report inventories, the Social Anxiety and Distress Scale (SAD) and the S-R Inventory of Anxiousness (SRI). In addition, staff members rated the offenders on modified versions of the SAD and the Heterosocial Skills Behavior Checklist for Males (HSB). Twenty-one offenders were also videotaped during a brief interaction with a female confederate. The tapes were rated using the HSB. All ratings and videotapings were conducted twice within a two-week period. The two self-report scales showed good test-retest reliability (r=.88) and correlatedr=.63 with each other. However, the SAD correlated significantly with the Crowne-Marlowe Social Desirability Scale and showed significant change from first to second administrations. The SRI showed less reactivity to social desirability and changed less over time. The staff ratings showed higher test-retest (r=.72–.76) than interrater (r=.36–.43) reliability. They correlatedr=–.56 with each other and did not show a significant relationship to the Crowne-Marlowe. However, both staff-ratings changed significantly from test to retest. Inspection of correlations between staff and self-report measures indicated that these instruments may be strongly affected by rater variance. Different measures completed by the same person correlated higher than the same measure completed by different persons. The HSB showed high levels of interrater agreement (90–100%) and split-half reliability (r=.95–.97). However, test-retest reliability was poor for the overall score and for the category measuring form of conversation. Categories measuring voice and affect were moderately reliable over time (r=.53–.74). However, the voice category scores changed substantially from first to second videotaping. On almost all measures used in this study, the sex offenders' scores were similar to those derived from normal populations. Only the category measuring affect on the HSB seemed to differentiate this group from normals.     

Situation and person-centered approaches to promoting leadership behavior in low performance self-esteem women

In each of 84 mixed-sex dyads, a low performance self-esteem woman was appointed to the role of leader in the dyad's decision-making task. Situation and person-centered approaches to promoting the woman's leadership behavior were tested by providing information regarding the women's task-relevant competence to one, both, or neither dyad member prior to the day of the task. Decision outcome scores and ratings of audio tapes served as measures of dyadic influence. When women were given the leadership information beforehand, they were more leaderlike and their partners were more acquiescent; informing men beforehand had little effect on the interaction, especially when the women had not been informed. Subjects' postdiscussion self-ratings, partner ratings, and perceived partner ratings suggested subtle forms of male defensiveness and more obvious forms of female discomfort with the leader role.     

Norms of subjective frequency of use and imagery for a sample of 1,916 French nouns.

Norms of rated subjective frequency of use and imagery on seven-point scales are reported for 1,916 French nouns. Subjective frequency was defined as the rated frequency of occurrence of words in spoken French, and imagery was defined as the rated case with which a word aroused a mental image. The mean, standard deviation, and percentile rank of the frequency and imagery ratings for each item are presented in the Appendix together with their objective frequency of occurrence in Baudot's (1992) dictionary. Interjudge reliability was assessed by calculating the correlation between the mean ratings of items repeated in the booklet, between the mean ratings obtained from odd-numbered and even-numbered respondents, and by computing the Cronbach alpha statistic for each page of the booklet. These reliability estimates were equal to or greater than .92 for frequency and for imagery, confirming the high level of interjudge consistency. Although the estimates provided by female and male participants were highly correlated (r = .97), the former gave a slightly higher frequency rating to the word sample but a slightly lower imagery rating than the latter did. Moreover, female respondents gave slightly more extreme ratings on the frequency and imagery scales. An analysis of the absolute difference between female and male ratings revealed a discrepancy of one half point or more on 20% of the word sample for frequency and 13% for imagery. On both scales, the mean absolute difference between male and female ratings was larger than that obtained by chance alone. This finding highlights the possibility that some words may not be equally familiar to women and men or may not evoke imagery with the same ease in these groups. Validity estimates for the frequency and imagery ratings were derived from correlations with scale values drawn from other normative studies. These correlation coefficients were equal to or greater than .78 for frequency and .86 for imagery, confirming the high level of consistency between this and other studies. An analysis of the relationship between subjective frequency and imagery ratings indicated that these variables are generally uncorrelated but exceptions occur. In the present study the coefficient of the correlation between subjective frequency and imagery was .24. However, when items with extreme mean frequency were excluded from the calculation, the correlation coefficient dropped to .04 and was no longer significant. Imagery ratings from five independent studies were all positively and significantly correlated with Vikis-Freibergs's (1974) frequency estimates, which were obtained from a free-association task. This finding suggests that word association, as a form of cued recall, may be influenced by several stimulus attributes including prior frequency of association and imagery-evoking value. The pattern of correlation between imagery ratings and text-based frequency estimates is not coherent. It reveals significant correlations only in select cases and no consistent polarity of linear relationship. The main contribution of this research is to provide reliable estimates of subjective frequency and imagery value for a word sample that is larger than those included in previous studies. A close examination of the linear relationship among the various sources of frequency and imagery data underscores the risk of confounding these variables in the selection of lexical stimuli for research.     

Reasons for Gaps in Crime Reporting: The Case of White-Collar Criminals Investigated by Private Fraud Examiners in Norway

Recent years have seen an increasing use of private internal investigations assessing financial irregularities that often imply economic crime. When fraud examiners discover evidence of white-collar crime, they often leave it to their clients whether the issue should be reported to the police. We examine the gaps in white-collar crime reporting after fraud examination and reasons behind such decisions. In Norway, non-reporting could be as high as 96%, as calculated in this article. We apply techniques of neutralization to private fraud examiners’ reasoning for non-reporting of suspected or detected white-collar crime. We also offer possible policy-based solutions to reduce the identified gaps in reporting.     

IVR Test & Survey: A computer program to collect data via computerized telephonic applications

Several studies have demonstrated the advantages of using interactive voice response (IVR) technology to collect self-report data from research participants and recipients of psychological/medical services. IVR allows participants to phone a computer and respond to recorded questions by pressing the appropriate touch-tone keys on their telephone. Because this technology offers substantial benefits in terms of cost and efficiency, it is surprising that it has not been more widely utilized by researchers and practitioners. Along with the automation of the administration and scoring of tests or surveys, IVR provides for questioning to be adapted to the participants’ responses. One possible explanation for the failure to exploit this technology is the absence of easy-to-use software that does not require programming knowledge. This article describes IVR Test & Survey, a program to facilitate the administration, scoring, and analysis of information collected with the use of IVR technology.