Development of a method for increasing the utility of multiple correlations by considering both testing time and test validity

A modification of the Wherry-Doolittle test selection method is presented by which tests are included in a multiple correlation (obtained for a given battery of tests) in the sequence in which the rate of return in validity per unit of testing time is greatest, rather than in the order of the size of their contribution to the multiple correlation. It is proposed that the modified method can be utilized profitably when there are economic or practical limits on the time available for test administration.The major portion of this article is based upon a thesis by W. F. Long directed by Dr. Joseph Tiffin with the counsel of Dr. Irving W. Burr. This thesis was submitted in partial fulfillment of the requirements for the degree of Master of Science in Psychology, Purdue University, June, 1947.     

A multiple group method of factoring the correlation matrix

There are a number of methods of factoring the correlation matrix which require the calculation of a table of residual correlations after each factor has been extracted. This is perhaps the most laborious part of factoring. The method to be described here avoids the computation of residuals after each factor has been computed. Since the method turns on the selection of a set of constellations or clusters of test vectors, it will be calleda multiple group method of factoring. The method can be used for extracting one factor at a time if that is desired but it will be considered here for the more interesting case in which a number of constellations are selected from the correlation matrix at the start. The result of this method of factoring is a factor matrixF which satisfies the fundamental relationFF'=R.This study is one of a series of investigations in the development of multiple factor analysis and application to the study of primary mental abilities. We wish to acknowledge the financial assistance from the Social Science Research Committee of The University of Chicago which has made possible the work of the Psychometric Laboratory.     

The multidimensional nature of ageism: construct validity and group differences

The authors investigated the factor structure and construct validity of the Fraboni Scale of Ageism and the age and gender differences in ageism scores. Confirmatory factor analyses supported the multidimensional nature of FSA scores and generally corroborated the initial factor structure reported by M. Fraboni, with some notable exceptions. Essentially, the present findings were aligned with theoretical models of ageism that emphasize both cognitive facets and affective facets. That is, on the basis of their factor analytic findings, the authors redefined Fraboni's original factors of Antilocution, Avoidance, and Discrimination as Stereotypes, Separation, and Affective Attitudes, respectively, because of the clustering of items within factors. The revised 3-factor structure accounted for 36.4% of the variance in FSA scores. FSA factor scores significantly related to other scores from other measures of age-related attitudes, with higher correlations among factors that were similar in terms of their cognitive nature versus their affective nature. Finally, younger individuals and men had significantly higher ageism scores on the FSA than older individuals and women. The authors discussed the importance of adequately assessing ageism, with particular emphasis devoted to the understanding of age bias.     

Toward an understanding of situational judgment item validity and group differences

This paper evaluates 2 adjustments to common scoring approaches for situational judgment tests (SJTs). These adjustments can result in substantial improvements to item validity, reductions in mean racial differences, and resistance to coaching designed to improve scores. The first adjustment, applicable to SJTs that use Likert scales, controls for elevation and scatter (Cronbach & Gleser, 1953). This adjustment improves item validity. Also, because there is a White-Black mean difference in the preference for extreme responses on Likert scales (Bachman & O'Malley, 1984), these adjustments substantially reduce White-Black mean score differences. Furthermore, this adjustment often eliminates the score elevation associated with the coaching strategy of avoiding extreme responses (Cullen, Sackett, & Lievens, 2006). Item validity is shown to have a U-shaped relationship with item means. This holds both for SJTs with Likert score response formats and for SJTs where respondents identify the best and worst response option. Given the U-shaped relationship, the second adjustment is to drop items with midrange item means. This permits the SJT to be shortened, sometimes dramatically, without necessarily harming validity.     

An examination of gender differences in the construct validity of the silencing the self scale

Jack’s (1991) theory of self-silencing was originally designed to explain higher rates of depression in women in comparison to men. However, research finding that men score equal or even higher than women on measures of self-silencing has lead theorists to speculate that self-silencing tendencies may be driven by different motivations and have different consequences for women versus men (Jack & Ali, 2010). Using a sample of 247 college students, we examined gender differences in the construct validity of the Silencing the Self Scale (STSS; Jack & Dill, 1992). We hypothesized that women would score higher on the Externalized Self-Perception subscale, but not the other three subscales. Gender differences in the relationship between the STSS subscales and theoretically-relevant constructs were also explored. The results indicated that women on average scored higher than men on the Externalized Self-Perception subscale, whereas men scored higher on the Care as Self-Sacrifice subscale. Further, there was a significant Gender × Care as Self-Sacrifice subscale interaction in the prediction of depression, such that this subscale was negatively correlated to depression in men, and uncorrelated in women. These results clarify how self-silencing might translate into different mental health outcomes for women and men.     

Note about the multiple group method

This note directs attention to the basic similarity between a factor analysis method described by Holzinger in 1944 and what Thurstone has called the multiple group method. With minor modifications and the application of Holzinger's method in several successive cycles until the residuals vanish, the methods are essentially the same.     

Recruitment modeling applied to longitudinal studies of group differences in intelligence

In longitudinal studies data is collected in a series of waves. Each wave after the first suffers from attrition. Therefore it can be difficult to discriminate between changes in sample parameters due to a longitudinal process (e.g. ageing) and changes due to attrition. The problem is particularly vexing if one of the purposes is to compare differences between two or more groups over time, because there may be differential attrition across groups. We refer to this as a recruitment problem, because the sample studied in any wave after the first can be thought of as having been recruited from a preceding wave. We present a mathematical model of the recruitment process. We then apply it to two large longitudinal studies, the British Cohort Survey 1970 and the National Child Development Study (1958), to demonstrate how we can identify effects expected from recruitment alone. As an illustration we show that differential recruitment would predict increasing differences between estimates of men's and women's intelligence as the studied sample ages.     

A method for eliciting scale errors in preschool classrooms

Six preschool classrooms, containing children between the ages of 4.0 and 40.0 months, were seeded with miniature items to examine how easily scale errors could be elicited. Observations occurred over a period of 3 months, with a total of 280 min of observations per classroom. A concealed observer documented all interactions with the miniature items. A total of 93 scale errors occurred, with a higher frequency committed by the younger children. These results confirm findings from previous research and suggest that scale errors can be effectively elicited in a preschool classroom. Our approach may provide a useful method for systematically studying the factors that influence scale errors.     

A validity scale sign calling for caution in the interpretation of MMPIs among psychiatric inpatients

Three expert MMPI judges classified 100 psychiatric inpatients as psychotic or non-psychotic on the basis of their MMPIs. Validity scale data, as well as clinical scale scores, were included for 50 of the profiles, while the validity scale scores were withheld from the judges for the remaining 50 profiles. Within each of the above two groups, half had a "positive" validity scale sign (a defensive validity scale configuration defined as L or K greater than or equal to 70, or both greater than or equal to 60) and half had a negative validity scale sign, indicating a lack of defensiveness. Using actual diagnosis as the external criterion, results indicated that the majority of defensive psychotic patients produced clinical scale configurations which appeared nonpsychotic to the judges. Conversely, the majority of nondefensive nonpsychotics produced psychotic-appearing clinical scale configurations. These two types of test misses suggest that K corrections on MMPI scales relating to psychosis are not optimal for psychiatric inpatients. Guidelines were developed for interpreting defensive profiles.     

Testing problem-solving capacities: differences between individual testing and social group setting

Testing animals individually in problem-solving tasks limits distractions of the subjects during the test, so that they can fully concentrate on the problem. However, such individual performance may not indicate the problem-solving capacity that is commonly employed in the wild when individuals are faced with a novel problem in their social groups, where the presence of a conspecific influences an individual’s behaviour. To assess the validity of data gathered from parrots when tested individually, we compared the performance on patterned-string tasks among parrots tested singly and parrots tested in social context. We tested two captive groups of orange-winged amazons (Amazona amazonica) with several patterned-string tasks. Despite the differences in the testing environment (singly vs. social context), parrots from both groups performed similarly. However, we found that the willingness to participate in the tasks was significantly higher for the individuals tested in social context. The study provides further evidence for the crucial influence of social context on individual’s response to a challenging situation such as a problem-solving test.     

A method for obtaining an ordered metric scale

A method is presented for collecting data which will yield a scale on which the entities are ranked in preference (ordinality), the distances between the entities on the scale are ranked (ordered metric), and all combinations of the distances are ranked (higher-ordered metric). The sources drawn upon are von Neumann and Morgenstern (9), and lattice theory. An empirical example is given in which a higher-ordered metric scale is derived.I am grateful to Professor William L. Lepley (Department of Psychology) and Professor Jack R. Tessman (Department of Physics) for their critical reading of this paper. Paul Hurst and Robert Radlow participated in many discussions on the form of measurement discussed in this paper, and assisted in collecting data. I am also grateful to Professor T. C. Benton (Department of Mathematics) for certain source materials.     

A factor-adjusted multiple testing procedure for ERP data analysis

Event-related potentials (ERPs) are now widely collected in psychological research to determine the time courses of mental events. When event-related potentials from treatment conditions are compared, often there is no a priori information on when or how long the differences should occur. Testing simultaneously for differences over the entire set of time points creates a serious multiple comparison problem in which the probability of false positive errors must be controlled, while maintaining reasonable power for correct detection. In this work, we extend the factor-adjusted multiple testing procedure developed by Friguet, Kloareg, and Causeur (Journal of the American Statistical Association, 104, 1406-1415, 2009) to manage the multiplicity problem in ERP data analysis and compare its performance with that of the Benjamini and Hochberg (Journal of the Royal Statistical Society B, 57, 289-300, 1995) false discovery rate procedure, using simulations. The proposed procedure outperformed the latter in detecting more truly significant time points, in addition to reducing the variability of the false discovery rate, suggesting that corrections for mass multiple testings of ERPs can be much improved by modeling the strong local temporal dependencies.