Evaluating correlations in studies of personality and behavior: Beyond the number of significant findings to be expected by chance

When large numbers of statistical tests are computed, such as in broad investigations of personality and behavior, the number of significant findings required before the total can be confidently considered beyond chance is typically unknown. Employing modern software, specially written code, and new procedures, the present article uses three sets of personality data to demonstrate how approximate randomization tests can evaluate (a) the number of significant correlations between a single variable and a large number of other variables, (b) the number of significant correlations between two large sets of variables, and (c) the average size of a large number of effects. Randomization tests can free researchers to fully explore large data sets and potentially have even wider applicability.     

Survey of New Zealand truck driver fatigue and fitness for duty

This paper presents recent research into compliance with current driving hours regulations, the effectiveness of using driving hours to predict fatigue, and alternative compliance and enforcement options. The paper describes results of a major survey of truck driver fatigue in New Zealand, a review of international compliance and enforcement procedures, and research focussing on the social forces and influences that affect truck drivers. The survey of truck drivers was based on interviews and performance tests collected from 600 truck drivers at depots, wharves, markets, and other locations throughout the North Island of New Zealand. The interviews included: questions on driver demographic and work/rest patterns, drivers' attitudes towards fatigue, propensity towards daytime sleepiness, and a self-assessment of the driver's momentary level of fatigue. In addition, a simulator-based performance test of driving was undertaken. The performance test included a combination of a standard driving task, a dual-axis sub-critical tracking task (maintaining speed and steering in a controlled but unstable environment, a virtual roadway affected by the appearance of random wind gusts requiring steering correction), and a tertiary or side-task requiring driver monitoring and periodic responses. The initial results from the first 100 drivers have found a sizable number of drivers exceeding the allowable driving hours, high levels of fatigue and sleepiness, and interesting differences between line-haul and local delivery drivers. A related research project into the social processes and relationships that affect truck drivers has resulted in a good understanding of the social conditions that influence cultural change and the actions of truck drivers and fleet managers. In this paper we will have particular regard to these processes in the construction of ideas concerning safety. This includes an understanding of the role of major stakeholders, such as freight forwarders and the enforcement agencies with respect to drivers and their conditions, actions and understanding of the road transport industry. This knowledge coupled with the survey results and an understanding of compliance and enforcement alternatives will be used to explore potential fatigue management options.     

Text anxiety on cognitive ability test can result in differential predictive validity of academic performance

This paper investigates whether test anxiety leads to differential predictive validity in academic performance. Our results show that the predictive validity of a cognitive ability test, using final exam performance as a criterion, decreased a small amount as Worry (the cognitive aspect of anxiety) increased but was unaffected by Emotionality (the physiological aspect of anxiety). These results suggest that cognitive ability tests may be more useful as predictors of performance for low anxiety test-takers. These findings are discussed in the context of the interference and deficit perspectives of test anxiety.     

Measurement Issues in High Stakes Testing

SUMMARY

Validity and reliability of the new high stakes testing systems initiated in school systems across the United States in recent years in response to the accountability features mandated in the No Child Left Behind Legislation largely depend on item response theory and new rules of measurement. Reliability and validity in item response theory and classical test theory are reviewed. Additionally, practices in the states are considered. The conclusion of the paper is that the new test technology is theoretically better suited to assess achievement than classical test theory, but has not been shown to be valid and reliable enough for use as the sole criterion for determination of what was learned in school. Further, there is no evidence that they will ever be found to be valid and reliable enough for that purpose. Areas of additional needed research are considered.     

Thinking Styles and Academic Achievement Among Filipino Students

The authors' objective in this study was to determine whether the precepts of R. J. Sternberg's (1988, 1997) theory of mental self-government apply to a non-Western culture. They administered R. J. Sternberg and R. K. Wagner's (1992) Thinking Styles Inventory, which is based on the theory of mental self-government, to 429 Filipino university students. The results of item analysis, scale intercorrelations, and factor analysis were consistent with the general provisions of the theory. Correlational analysis between thinking styles and grade point average showed that thinking styles are related to academic achievement. The results are explained with respect to the concepts and practices of Philippine culture and schools and discussed in relation to the developmental assumptions of the theory of mental self-government.     

The Link Between Cerebellar Dominance and Skilled Hand Performance in 8–10-Year-Old Right-Handed Children

A focus of attention on the step-by-step control of a skill has been shown to be detrimental to experts' performance but to have no significant effect on novices' performance (e.g., S. L. Beilock, T. H. Carr, C. MacMahon, &; J. L. Starkes, 2002), contrary to the results of manipulations of the direction of attentional focus (e.g., G. Wulf, M. Hös, &; W. Prinz, 1998). In previous studies, researchers have not separated the focus of attention from the nature of the instruction provided or the skill level of the participants. In the present experiment, 10 skilled and 10 less skilled soccer players dribbled a ball after receiving instructions directing attention to an internal, skill-relevant feature (foot); an internal, skill-irrelevant feature (arm); or a skill-irrelevant task (word-monitoring). Performance was evaluated in relation to a no-attentional-focus control condition. For skilled performers, an internal focus on the arms and feet interfered with performance. For less skilled performers, an internal, yet skill-relevant, focus of attention (foot) did not degrade performance, whereas attention to the arms and word monitoring had a detrimental effect. No significant differences were observed across the three attentional manipulations when the skilled performers used the nondominant foot. The results generally supported the deautomization of skills hypothesis.     

A supposedly objective thing I'll never use again: Word association and the quest for validity and reliability in emotional adjustment research from Carl Jung to Carl Rogers (1898–1927)

As the first two decades of the 20th century unfolded, clinical psychologists, who had until then been mainly associated with intelligence testing, attempted to implement a specific psychological method—Carl Gustav Jung's (1875–1961) word-association “test”—in individual personality assessments. As one of the early clinical psychologists who attempted to use the method, Carl Ransom Rogers (1902–1987) is conspicuously absent from the historiography of clinical psychological testing. In fact, historians have recently suggested that we are lacking narratives about Rogers' early ideas and techniques in the context of both the development of clinical psychology and the emergence of psychological testing as clinicians' foremost scholarly activity. In light of the above, this paper pursues two main goals. First, it attempts to reconstruct Rogers' first original research project on emotional adjustment testing in young children in the broader context of the development of word-association tests as carried out by Jung and Whately Smith (1892–1947). Second, it aims to reconstruct Rogers' earliest theoretical ideas as well as his epistemological assumptions regarding test objectivity, validity and reliability. By drawing on unpublished documents and heretofore overlooked primary sources I show that although Rogers initially drew from Jung and Smith's complex and refined tradition, he ultimately rejected it as well as the tests themselves. At first drawn to Smith's quantitative, empiricist and experimental philosophy of psychology, Rogers was deterred when the data gathered through his own research in 1927 suggested that word association tests had no real, effective clinical value when used in children. By showcasing the complex process of test construction and validation undertaken by 1920s clinical psychologists, Rogers' case illustrates the research practices, the methodological problems and the epistemological dilemmas faced by most if not all of his contemporaries.     

Subjective cognitive complaint in healthy older adults: Identification of major domains and relation to objective performance

This study analyzed complaints across a range of memory domains in middle-aged and older adults, as well as assessed their relationship with objective cognitive performance and personal characteristics. A total of 115 community-dwelling, healthy adults between the ages of 45 and 87 completed the Self-Evaluation Questionnaire (QAM) and were tested with a comprehensive neuropsychological battery. A principal component analysis identified seven dimensions of complaint. Difficulty inhibiting internal and external sources of interference when learning or remembering something was the main area of complaint and it was correlated with word-list recall and Digit Symbol. We identified complaints related to a set of memory failures that appear to be more detrimental to autonomy and safety. These were correlated with a measure of functional autonomy and might, therefore, indicate more severe difficulties. Overall, this study provides indications regarding the main domains of memory complaints and identifies which complaints are most reflective of objective cognitive problems.