Statistical inferences about the error variance

This paper is a presentation of an essential part of the sampling theory of the error variance and the standard error of measurement. An experimental assumption is that several equivalent tests with equal variances are available. These may be either final forms of the same test or obtained by dividing one test into several parts. The simple model of independent and normally distributed errors of measurement with zero mean is employed. No assumption is made about the form of the distributions of true and observed scores. This implies unrestricted freedom in defining the population. First, maximum-likelihood estimators of the error variance and the standard error of measurement are obtained, their sampling distributions given, and their properties investigated. Then unbiased estimators are defined and their distributions derived. The accuracy of estimation is given special consideration from various points of view. Next, rigorous statistical tests are developed to test hypotheses about error variances on the basis of one and two samples. Also the construction of confidence intervals is treated. Finally, Bartlett's test of homogeneity of variances is used to provide a multi-sample test of equality of error variances.     

The estimation of true score variance and error variance in the classical test theory model

The use of one-way analysis of variance tables for obtaining unbiased estimates of true score variance and error score variance in the classical test theory model is discussed. Attention is paid to both balanced (equal numbers of observations on each person) and unbalanced designs, and estimates provided for both homoscedastic (common error variance for all persons) and heteroscedastic cases. It is noted that optimality properties (minimum variance) can be claimed for estimates derived from analysis of variance tables only in the balanced, homoscedastic case, and that there they are essentially a reflection of the symmetry inherent in the situation. Estimates which might be preferable in other cases are discussed. An example is given where a natural analysis of variance table leads to estimates which cannot be derived from the set of statistics which is sufficient under normality assumptions. Reference is made to Bayesian studies which shed light on the difficulties encountered. Work on this paper was carried out at the headquarters of the American College Testing Program, Iowa City, Iowa, while the author was on leave from the University College of Wales.     

Some thoughts regarding the sensitivity of t to error in estimated within-group variance

A. J. Riopelle (2003) has eloquently demonstrated that the null hypothesis assessed by the t test involves not only mean differences but also error in the estimation of the within-group standard deviation, s. He is correct in his conclusion that the precision of the interpretation of a significant t and the null hypothesis tested is complex, particularly when sample sizes are small. In this article, the author expands on Riopelle's thoughts by comparing t with some equivalent or closely related tests that make the reliance of t on the accurate estimation of error perhaps more salient and by providing a simulation that may address more directly the magnitude of the interpretational problem.     

The analysis of variance and covariance techniques in relation to the conventional formulas for the standard error of a difference

In this paper it is demonstrated that the analysis of variance techniques yield results equivalent to the calculation oft by means of expressions based on the short or the long formula. It is also shown that the covariance technique gives results equivalent to those obtained by means of the formula fort which should be used with matched groups. The conditions necessary for equivalent results are such that the conventional formulas fort would normally be used rather than the variance or covariance techniques. However, a knowledge of the relationships described in this paper should contribute to one's understanding of the variance and covariance techniques.The relationships described in this paper were brought to the attention of the author by the excellent article of Eugene Shen (8). The proofs given here and the examples are the work of the present author.     

The error in the error theory

Moral error theory of the kind defended by J. L. Mackie and Richard Joyce is premised on two claims: (1) that moral judgements essentially presuppose that moral value has absolute authority, and (2) that this presupposition is false, because nothing has absolute authority. This paper accepts (2) but rejects (1). It is argued first that (1) is not the best explanation of the evidence from moral practice, and second that even if it were, the error theory would still be mistaken, because the assumption does not contaminate the meaning or truth-conditions of moral claims. These are determined by the essential application conditions for moral concepts, which are relational rather than absolute. An analogy is drawn between moral judgements and motion judgements.     

Estimation of true score and error variance for tests under various equivalence assumptions

Maximum-likelihood estimators of true score variance and error variance for mental tests are derived for six different models of equivalent measurements. Statistical properties of the estimators are examined. Main emphasis is placed upon essentially-equivalent measurements. A statistical criterion for this type of measurement is given. The solution of the comparatively simple maximum-likelihood equations is effected by means of a rapid Newton-Raphson procedure. Two different initial estimators are considered and their relative merits in terms of second moments evaluated. Four numerical examples are appended by way of illustration.Paper read at the Mathematical Psychology Meeting, August 28, 1968, at Stanford, Calfornia.This research was supported in part by the National Institute of Child Health and Human Development, Research Grant 1 P01 HD01762.     

P50 sensory gating deficits in schizotypy

Sensory gating is the ability to filter out irrelevant stimuli from the environment. Individuals with schizophrenia consistently demonstrate deficits in this ability leading to sensory overload and cognitive fragmentation. This dysfunction has also been found in schizotypy, which is defined as a manifestation of nonclinical symptoms qualitatively similar to those found in schizophrenia. In the present study, auditory sensory gating was assessed in healthy individuals by testing the attenuation of the P50 event-related potential. The degree of suppression was then correlated with schizotypy by using the O-LIFE questionnaire. Relative to the low-scoring individuals, P50 suppression was significantly reduced in those with high levels of schizotypy. Furthermore, the degree of deficit in P50 gating correlated with both cognitive disorganisation and impulsive nonconformity dimensions of schizotypy. These results suggest that schizotypal individuals may have early sensory gating deficits similar to schizophrenia patients, especially if they display a disorganised or impulsive profile. As they do not exhibit overt psychotic symptoms, it is likely that such deficits represent an underlying core cognitive dysfunction within the schizophrenia spectrum.     

Modeling error variance in job specification ratings: the influence of rater, job, and organization-level factors

The authors modeled sources of error variance in job specification ratings collected from 3 levels of raters across 5 organizations (N=381). Variance components models were used to estimate the variance in ratings attributable to true score (variance between knowledge, skills, abilities, and other characteristics [KSAOs]) and error (KSAO-by-rater and residual variance). Subsequent models partitioned error variance into components related to the organization, position level, and demographic characteristics of the raters. Analyses revealed that the differential ordering of KSAOs by raters was not a function of these characteristics but rather was due to unexplained rating differences among the raters. The implications of these results for job specification and validity transportability are discussed.     

The variance reaction time model

The variance reaction time model (VRTM) is proposed to account for various recognition data on reaction time, the mirror effect, receiver-operating-characteristic (ROC) curves, etc. The model is based on simple and plausible assumptions within a neural network: VRTM is a two layer neural network where one layer represents items and one layer represents contexts. The recognition decision is based on a random walk of nodes activated at recognition. VRTM suggests theoretical constraints on the distributions of nodes activated at recognition and the noise in the random walk. The variability in the net inputs to nodes depends on the item frequency (the number of times that the item has been encoded) and the list length. The essential mechanism that accounts for the empirical data is a non-linear activation function. The mean activation threshold in the non-linear activation function is placed to achieve efficient discriminability between new and old items and there is variability in the activation threshold. VRTM predicts the mirror effect for low and high frequency words, a strength based mirror effect between conditions but not within one condition, appropriate ROC-curves for old/new and high/low frequency items, and list-length effects. Furthermore, it predicts appropriate means and distributions of reaction times for old/new, correct/incorrect, and high/low frequency items as well as speed/accuracy tradeoffs. VRTM has an explicit mathematical solution, it is simulated in a neural network, and it is fitted to a number of datasets.     

The variance of the number of mutual choices in sociometry

The variance of the number of mutual dyads in a sociometric situation where each member of a group chooses independently and at random is derived for unrestricted numbers of choices per group member, as well as for a fixed number of choices. The distribution of the number of mutuals is considered.We are indebted to Robert Bush and Hartley Rodgers of Harvard University for helpful criticisms.     

The analysis of variance and pairwise scaling

Variance analyses are presented for two data layouts—each corresponding to the class of all ordered pairs from a single finite set. The analysis of the dominance layout is in terms of a fixed effects linear model which includes parameters representing the scale values of the elements of the set, response bias, and pairwise interactions. A parallel parametrization is carried out for the composition layout for which corresponding point estimates and hypothesis tests are given. A joint treatment of concurrently observed dominance and composition layouts is suggested and illustrative data are presented.This research was supported in part by National Institutes of Health Grant MH-04439-06. The author would like to express his appreciation to Richard Beatty of the University of Toronto, James Baker and William Carroll of Oregon Research Institute, and J. E. Keith Smith of the University of Michigan, for their helpful comments concerning aspects of this work. Computing assistance was obtained from the Health Sciences Computing Facility, UCLA, sponsored by NIH Grant FR-3.     

The variance theory of the mirror effect in recognition memory

The mirror effect refers to a rather general empirical finding showing that, for two classes of stimuli, the class with the higher hit rates also has a lower false alarm rate. In this article, a parsimonious theory is proposed to account for the mirror effect regarding, specifically, high- and low-frequency items and the associated receiver-operating curves. The theory is implemented in a recurrent network in which one layer represents items and the other represents contexts. It is shown that the frequency mirror effect is found in this simple network if the decision is based on counting the number of active nodes in such a way that performance is optimal or near optimal. The optimal performance requires that the number of active nodes is low, only nodes active in the encoded representation are counted, the activation threshold is set between the old and the new distributions, and normalization is based on the variance of the input. Owing to the interference caused by encoding the to-be-recognized item in several preexperimental contexts, the variance of the input to the context layer is greater for highthan for low-frequency items, which yields lower hit rates and higher false alarm rates for high- than for low-frequency items. Although initially the theory was proposed to account for the mirror effect with respect to word frequency, subsequent simulations have shown that the theory also accounts for strength-based mirror effects within a list and between lists. In this case, consistent with experimental data, the variance theory suggests that focusing attention to the more difficult class within a list affects the hit rate, but not the false alarm rate and not the standard deviations of the underlying density, leading to no mirror effect.     

The ubiquity of common method variance: The case of the Big Five

The factor structures of the International Personality Item Pool (IPIP) and NEO-FFI Big Five questionnaires were examined via confirmatory factor analyses. Analyses of IPIP data for five samples and NEO data for one sample showed that a CFA model with three method bias factors, one influencing all items, one influencing negatively worded items, and one influencing positively worded items fit the data significantly better than models without method factors or models with only one method factor . With the method factors estimated, our results indicated that the Big Five dimensions may be more nearly orthogonal than previously demonstrated. Implications of the presence of method variance in Big Five scales are discussed.     

The role of correlation in analysis of variance

A test of the significance of a row or column agent in an analysis of variance may be expressed in the form of correlation between the agent and the variate. A test of the significance of interaction variance may be expressed in the form of correlation between the agents. These expressions are principally of theoretical interest in that the degree of significance in anF test or the value of a correlation coefficient may be controlled at will, or inadvertently, within certain limits.This study was made possible by the Bureau of Psychological Services, Institute for Human Adjustment, Horace H. Rackham School of Graduate Studies, University of Michigan.     

Reduced P50 auditory sensory gating response in professional musicians

Evoked potential studies have demonstrated that musicians have the ability to distinguish musical sounds preattentively and automatically at the temporal, spectral, and spatial levels in more detail. It is however not known whether there is a difference in the early processes of auditory data processing of musicians. The most emphasized and studied early process, especially for neuropsychiatric purposes, is sensory gating. The suppression percentage of the midlatency auditory evoked potential P50, and rarely the N100, wave is used for sensory gating studies. Our aim in this study was to investigate whether there was a difference in the auditory P50 and N100 suppression of control subjects who were professional musicians with no psychiatric problems. 34 professional musicians and 19 non-musicians (the control group) were included in this study. P50 and N100 measurements were taken, the suppression percentage of P50 and N100 was calculated and the results compared. Musicians showed significantly less P50 suppression when compared to non-musicians. There was no significant difference for N100 suppression. What the decreased P50 suppression in musicians when compared to non-musician subjects means, when we also take into account that N100 suppression is not decreased, and how it may contribute to the music perception and production processes of these persons is discussed.     

Detection of interactions between a dichotomous moderator and a continuous predictor in moderated multiple regression with heterogeneous error variance

Moderated multiple regression (MMR) has been widely used to investigate the interaction or moderating effects of a categorical moderator across a variety of subdisciplines in the behavioral and social sciences. In view of the frequent violation of the homogeneity of error variance assumption in MMR applications, the weighted least squares (WLS) approach has been proposed as one of the alternatives to the ordinary least squares method for the detection of the interaction effect between a dichotomous moderator and a continuous predictor. Although the existing result is informative in assuring the statistical accuracy and computational ease of the WLS-based method, no explicit algebraic formulation and underlying distributional details are available. This article aims to delineate the fundamental properties of the WLS test in connection with the well-known Welch procedure for regression slope homogeneity under error variance heterogeneity. With elaborately systematic derivation and analytic assessment, it is shown that the notion of WLS is implicitly embedded in the Welch approach. More importantly, extensive simulation study is conducted to demonstrate the conditions in which the Welch test will substantially outperform the WLS method; they may yield different conclusions. Welch’s solution to the Behrens-Fisher problem is so entrenched that the use of its direct extension within the linear regression framework can arguably be recommended. In order to facilitate the application of Welch’s procedure, the SAS and R computing algorithms are presented. The study contributes to the understanding of methodological variants for detecting the effect of a dichotomous moderator in the context of moderated multiple regression. Supplemental materials for this article may be downloaded from brm.psychonomic-journals.org/content/supplemental.