Interval estimation for the intraclass correlation in dirichlet-multinomial data

When the underlying distribution is discrete with a limited number of categories, methods for interval estimation of the intraclass correlation which assume normality are theoretically inadequate for use. On the basis of large sample theory, this paper develops an asymptotic closed-form interval estimate of the intraclass correlation for the case where there is a natural score associated with each category. This paper employs Monte Carlo simulation to demonstrate that when the underlying intraclass correlation is large, the traditional interval estimator which assumes normality can be misleading. We find that when the number of classes is 20, the interval estimator proposed here can generally perform reasonably well in a variety of situations. This paper further notes that the proposed interval estimator is invariant with respect to a linear transformation. When the data are on a nominal scale, an extension of the proposed method to account for this case, as well as a discussion on the relationship between the intraclass correlation and a kappa-type measure defined here and on the limitation of the corresponding kappa-type estimator are given.The authors wish to thank the Editor, the Associate Editor, and the three referees for many valuable comments and suggestions to improve the clarity of this paper. The works for the first, the third, and the fourth authors were partially supported by grant #R01AR43025-01 from the National Institute of Arthritis and Musculoskeletal and Skin Diseases.     

A note on the unbiased estimation of the intraclass correlation

The intraclass correlation,, is a parameter featured in much psychological research. Two commonly used estimators of, the maximum likelihood and least squares estimators, are known to be negatively biased. Olkin and Pratt (1958) derived the minimum variance unbiased estimator of the intraclass correlation, but use of this estimator has apparently been impeded by the lack of a closed form solution. This note briefly reviews the unbiased estimator and gives a FORTRAN 77 subroutine to calculate it.The first author was supported by an All-University Fellowship from the University of Southern California.     

Sex differences in the estimated intelligence of school children

This study investigated sex differences in estimated general and multiple intelligence in school children, their parents, and their teachers. There were three groups of participants: 285 (149 female, 136 male) pupils of a mixed government‐run comprehensive school, between the ages of 13 and 16 years; 93 mothers and 58 fathers of the pupils; and five female and eight male teachers. Children estimated their own and their parents' IQ, whilst the parents estimated their own and their children's IQ; the teachers estimated only the children's intelligence. The aims of this study were firstly to assess whether perceptions of male intellectual superiority were observable in school age children and school teachers, and to make direct comparisons between the children's self‐estimations and those of the parents and the teachers. Secondly, this study aimed to replicate previous literature on adult self‐estimations of overall and multiple intelligences, and to compare these to estimations by children of these adults (their parents). Fewer sex differences were observed than expected. Teachers' estimations did not follow conceptions of male superiority. The patterns of sex differences in mother and teacher estimations of children were similar to each other, as were those of fathers and children. Verbal and numerical abilities were found to be most closely related to estimations of overall IQ in all three groups. Most striking was the lack of correlation between father and daughter estimations of each other. Reasons why this study failed to replicate findings on adult samples are discussed. Copyright © 2002 John Wiley & Sons, Ltd.     

Age differences in the distractor-ratio effect as a function of contrast level

Latencies (RT) and eye movement measures were used to examine the effects of age and contrast on the distractor-ratio effect (DRE) in visual search. Younger and older adults performed a contrast x orientation conjunction search task where the ratios of white to black distractors and luminance contrast levels were varied. The distractor-ratio manipulation had similar effects for older and younger adults on both RT and the number of fixations required to find the target. Both measures were largely independent of distractor ratio on target-present trials, while both RTs and the fixation number increased with the number of items sharing the target's contrast polarity on target-absent trials. A more detailed analysis of eye movements suggested that younger adults were a bit more adept at attending to the smaller set of distractors, which presumably facilitated both overt and covert search. Generalized slowing can account for the age differences in RT, but the fixation number data speak to another mechanism, perhaps increased cautiousness on the part of the elderly when signal strength is low.     

Biases of success rate differences shown in binomial effect size displays

The intent of a binomial effect size display (BESD) is to show "the [real-world] importance of [an] effect indexed by a correlation [r]" (R. Rosenthal, 1994, p. 242) by reexpressing this correlation as a success rate difference (SRD) (e.g., treatment group success rate-control group success rate). However, SRDs displayed in BESDs generally overestimate real-world SRDs implied by correlations of (a) dichotomous X and Y variables (phi coefficients), (b) dichotomous X and continuous Y variables (point-biserial coefficients [rpbs]). and (c) continuous X and Y variables (rxys). Furthermore, overestimation biases are larger for rxys than for rpbs. Differences in the sizes of biases linked to different correlations suggest that BESD SRDs reported for different correlations are not comparable. The stochastic difference index (N. Cliff, 1993: A. Vargha & H. D. Delaney, 2000) is recommended as an alternative to the BESD.     

Sex differences in human brain size and the general meaning of differences in brain size.

Contrary to commonly held convictions, there is no clear association between brain size and body parameters in humans. Within sexes, once age and health status are controlled for, there is no significant association between brain size and body height for females. For males, body height accounts for no more than .04% of the variance in brain size. The relation between brain weight and body weight is even less clearly defined. Nevertheless, there are large and significant differences in brain size between the sexes. If no adequate body parameters can be found that scale to brain size within the sexes, the marked dimorphism between males and females makes it even more difficult to find a common set of parameters that allow evaluation of brain size differences between sexes. Within and across sexes, there is no convincing link between a limited measure of behavioural capacity (IQ) and brain size. This leads to the more general question: Why would one expect such a link, and, if it is not found, what does this mean in the context of general theories of cortical function?     

Masking-level differences in older adults: The effect of the level of the masking noise

In Experiment 1, masking-level differences (MLDs) for a 500-Hz tone at five masker levels were obtained from younger and older adults. For both age groups, there were no reliable increases in MLD once the spectrum level of the masker exceeded 27 dB SPL. MLDs were larger for younger than for older adults over the range of masker levels tested. In Experiment 2, the levels of both the signal and the masker in one ear were attenuated by either 15 or 30 dB relative to their level in the other ear, which was fixed at a spectrum level of 47 dB SPL. MLDs for both age groups declined with increasing IAA and age-related differences were observed in all conditions. The findings of these experiments indicate that (1) age-related differences in MLDs exist even when the level of the masker is sufficiently high that older adults achieve their plateau performance, and (2) older listeners are not disadvantaged more than younger listeners by interaural differences in the level of the input.     

Examining the reliability of interval level data using root mean square differences and concordance correlation coefficients

This article introduces new statistics for evaluating score consistency. Psychologists usually use correlations to measure the degree of linear relationship between 2 sets of scores, ignoring differences in means and standard deviations. In medicine, biology, chemistry, and physics, a more stringent criterion is often used: the extent to which scores are identically equal. For each test taker (or other unit of measurement), the difference between the 2 scores is calculated. The root mean square difference (RMSD) represents the average change from 1 set of scores to the other, and the concordance correlation coefficient (CCC) rescales this coefficient to have a maximum value of 1. This article shows the relationship of the RMSD and CCC to the intraclass correlation coefficients, product-moment correlation, and standard error of measurement. Finally, this article adapts the RMSD and the CCC for linear, consistency, and absolute definitions of agreement.     

Mapping and correcting the influence of gaze position on pupil size measurements

Pupil size is correlated with a wide variety of important cognitive variables and is increasingly being used by cognitive scientists. Pupil data can be recorded inexpensively and non-invasively by many commonly used video-based eye-tracking cameras. Despite the relative ease of data collection and increasing prevalence of pupil data in the cognitive literature, researchers often underestimate the methodological challenges associated with controlling for confounds that can result in misinterpretation of their data. One serious confound that is often not properly controlled is pupil foreshortening error (PFE)—the foreshortening of the pupil image as the eye rotates away from the camera. Here we systematically map PFE using an artificial eye model and then apply a geometric model correction. Three artificial eyes with different fixed pupil sizes were used to systematically measure changes in pupil size as a function of gaze position with a desktop EyeLink 1000 tracker. A grid-based map of pupil measurements was recorded with each artificial eye across three experimental layouts of the eye-tracking camera and display. Large, systematic deviations in pupil size were observed across all nine maps. The measured PFE was corrected by a geometric model that expressed the foreshortening of the pupil area as a function of the cosine of the angle between the eye-to-camera axis and the eye-to-stimulus axis. The model reduced the root mean squared error of pupil measurements by 82.5 % when the model parameters were pre-set to the physical layout dimensions, and by 97.5 % when they were optimized to fit the empirical error surface.     

Sex differences in the activity level of infants1

A gender difference in motor activity level (AL) is well established for children, but questions about the existence and nature of an infant sex difference remain. To assess these questions, we applied meta‐analytic procedures to summarize 46 infancy studies comprising 78 male–female motor activity comparisons. Our results showed that, as with children, male infants were more active than females. Objective measures of infant AL estimated the size of this difference to be 0.2 standard deviations, though subjective parent‐report measures estimated the difference to be smaller. We argue that this early sex difference in activity level is biologically based. However, socialization processes, such as gender‐differentiated expectations and experiences, in conjunction with further sex‐differentiated biological developments, amplify this early difference to produce the larger gender differences in activity found during childhood. Copyright © 1999 John Wiley & Sons, Ltd.     

Sex and cross-cultural differences in the estimated multi-faceted intelligence quotient score for self, parents and siblings

Over four hundred young people from Britain, Hawaii and Singapore estimated their own, their parents and their siblings IQ score on each of Gardner (1983) fundamental human intelligences: verbal (linguistic), logical (mathematical), spatial, musical, body-kinesthetic, interpersonal and intrapersonal. They also answered six simple questions concerning intelligence tests. There were both cultural and sex differences in the estimation of overall own intelligence score. Males gave higher scores than females (109 vs 107) while the British gave the highest score (109) compared with the Singaporeans (106) and Hawaiians (104). Factor analysis of the seven dimensions yielded either a two or three factor solution, the latter being verbal (verbal, inter-intrapersonal), mathematical (mathematical and spatial), and musical (musical, body-kinesthetic). There were consistent sex differences in the estimations of the three factors for self, but not of parents, and only marginally of sisters. Males more than females, and the British more than the other groups, were more likely to believe in sex and race difference in intelligence.     

Cultural differences in the primacy effect for person perception

Previous work has shown there are robust differences in how North Americans and East Asians form impressions of people. The present research examines whether the tendency to weigh initial information more heavily—the primacy effect—may be another component of these cultural differences. Specifically, we tested whether Americans would be more likely to use first impressions to guide person perception, compared to Japanese participants. In this experiment, participants read a vignette that described a target person's behaviour, then rated the target's personality. Before reading the vignette, some trait information was given to create an expectation about the target's personality. The data revealed that Americans used this initial information to guide their judgments of the target, whereas the Japanese sample based their judgments on all the information more evenly. Thus, Americans showed a stronger primacy effect in their impression formation than Japanese participants, who engaged in more data‐driven processing.     

Quantifying the reliability of image replication studies: The image intraclass correlation coefficient (I2C2)

This article proposes the image intraclass correlation (I2C2) coefficient as a global measure of reliability for imaging studies. The I2C2 generalizes the classic intraclass correlation (ICC) coefficient to the case when the data of interest are images, thereby providing a measure that is both intuitive and convenient. Drawing a connection with classical measurement error models for replication experiments, the I2C2 can be computed quickly, even in high-dimensional imaging studies. A nonparametric bootstrap procedure is introduced to quantify the variability of the I2C2 estimator. Furthermore, a Monte Carlo permutation is utilized to test reproducibility versus a zero I2C2, representing complete lack of reproducibility. Methodologies are applied to three replication studies arising from different brain imaging modalities and settings: regional analysis of volumes in normalized space imaging for characterizing brain morphology, seed-voxel brain activation maps based on resting-state functional magnetic resonance imaging (fMRI), and fractional anisotropy in an area surrounding the corpus callosum via diffusion tensor imaging. Notably, resting-state fMRI brain activation maps are found to have low reliability, ranging from .2 to .4. Software and data are available to provide easy access to the proposed methods.     

Using individual differences to predict job performance: correcting for direct and indirect restriction of range

The present study investigates the relationship between individual differences, indicated by personality (FFM) and general mental ability (GMA), and job performance applying two different methods of correction for range restriction. The results, derived by analyzing meta-analytic correlations, show that the more accurate method of correcting for indirect range restriction increased the operational validity of individual differences in predicting job performance and that this increase primarily was due to general mental ability being a stronger predictor than any of the personality traits. The estimates for single traits can be applied in practice to maximize prediction of job performance. Further, differences in the relative importance of general mental ability in relation to overall personality assessment methods was substantive and the estimates provided enables practitioners to perform a correct utility analysis of their overall selection procedure.     

Confidence intervals for the overall effect size in random-effects meta-analysis

One of the main objectives in meta-analysis is to estimate the overall effect size by calculating a confidence interval (CI). The usual procedure consists of assuming a standard normal distribution and a sampling variance defined as the inverse of the sum of the estimated weights of the effect sizes. But this procedure does not take into account the uncertainty due to the fact that the heterogeneity variance (tau2) and the within-study variances have to be estimated, leading to CIs that are too narrow with the consequence that the actual coverage probability is smaller than the nominal confidence level. In this article, the performances of 3 alternatives to the standard CI procedure are examined under a random-effects model and 8 different tau2 estimators to estimate the weights: the t distribution CI, the weighted variance CI (with an improved variance), and the quantile approximation method (recently proposed). The results of a Monte Carlo simulation showed that the weighted variance CI outperformed the other methods regardless of the tau2 estimator, the value of tau2, the number of studies, and the sample size.