Comparing one-step M-estimators of location when there are more than two groups

Methods for comparing means are known to be highly nonrobust in terms of Type II errors. The problem is that slight shifts from normal distributions toward heavy-tailed distributions inflate the standard error of the sample mean. In contrast, the standard error of various robust measures of location, such as the one-step M-estimator, are relatively unaffected by heavy tails. Wilcox recently examined a method of comparing the one-step M-estimators of location corresponding to two independent groups which provided good control over the probability of a Type I error even for unequal sample sizes, unequal variances, and different shaped distributions. There is a fairly obvious extension of this procedure to pairwise comparisons of more than two independent groups, but simulations reported here indicate that it is unsatisfactory. A slight modification of the procedure is found to give much better results, although some caution must be taken when there are unequal sample sizes and light-tailed distributions. An omnibus test is examined as well.     

Bootstrap standard error and confidence intervals for the correlations corrected for indirect range restriction

The standard Pearson correlation coefficient, r, is a biased estimator of the population correlation coefficient, ρ(XY) , when predictor X and criterion Y are indirectly range-restricted by a third variable Z (or S). Two correction algorithms, Thorndike's (1949) Case III, and Schmidt, Oh, and Le's (2006) Case IV, have been proposed to correct for the bias. However, to our knowledge, the two algorithms did not provide a procedure to estimate the associated standard error and confidence intervals. This paper suggests using the bootstrap procedure as an alternative. Two Monte Carlo simulations were conducted to systematically evaluate the empirical performance of the proposed bootstrap procedure. The results indicated that the bootstrap standard error and confidence intervals were generally accurate across simulation conditions (e.g., selection ratio, sample size). The proposed bootstrap procedure can provide a useful alternative for the estimation of the standard error and confidence intervals for the correlation corrected for indirect range restriction.     

Estimation of reliability for multiple‐component measuring instruments in test–retest designs

A covariance structure modelling method for the estimation of reliability for composites of congeneric measures in test–retest designs is outlined. The approach also allows an approximate standard error and confidence interval for scale reliability in such settings to be obtained. The procedure further permits measurement error components due to possible transient condition influences to be accounted for and evaluated, and is illustrated with a pair of examples.     

Assessing the informational value of parameter estimates in cognitive models.

Mathematical models of cognition often contain unknown parameters whose values are estimated from the data. A question that generally receives little attention is how informative such estimates are. In a maximum likelihood framework, standard errors provide a measure of informativeness. Here, a standard error is interpreted as the standard deviation of the distribution of parameter estimates over multiple samples. A drawback to this interpretation is that the assumptions that are required for the maximum likelihood framework are very difficult to test and are not always met. However, at least in the cognitive science community, it appears to be not well known that standard error calculation also yields interpretable intervals outside the typical maximum likelihood framework. We describe and motivate this procedure and, in combination with graphical methods, apply it to two recent models of categorization: ALCOVE (Kruschke, 1992) and the exemplar-based random walk model (Nosofsky & Palmeri, 1997). The applications reveal aspects of these models that were not hitherto known and bring a mix of bad and good news concerning estimation of these models.     

Assessing the informational value of parameter estimates in cognitive models

Mathematical models of cognition often contain unknown parameters whose values are estimated from the data. A question that generally receives little attention is how informative such estimates are. In a maximum likelihood framework, standard errors provide a measure of informativeness. Here, a standard error is interpreted as the standard deviation of the distribution of parameter estimates over multiple samples. A drawback to this interpretation is that the assumptions that are required for the maximum likelihood framework are very difficult to test and are not always met. However, at least in the cognitive science community, it appears to be not well known that standard error calculation also yields interpretable intervals outside the typical maximum likelihood framework. We describe and motivate this procedure and, in combination with graphical methods, apply it to two recent models of categorization: ALCOVE (Kruschke, 1992) and the exemplar-based random walk model (Nosofsky & Palmeri, 1997). The applications reveal aspects of these models that were not hitherto known and bring a mix of bad and good news concerning estimation of these models.     

Modeling error distributions of growth curve models through Bayesian methods

Growth curve models are widely used in social and behavioral sciences. However, typical growth curve models often assume that the errors are normally distributed although non-normal data may be even more common than normal data. In order to avoid possible statistical inference problems in blindly assuming normality, a general Bayesian framework is proposed to flexibly model normal and non-normal data through the explicit specification of the error distributions. A simulation study shows when the distribution of the error is correctly specified, one can avoid the loss in the efficiency of standard error estimates. A real example on the analysis of mathematical ability growth data from the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 is used to show the application of the proposed methods. Instructions and code on how to conduct growth curve analysis with both normal and non-normal error distributions using the the MCMC procedure of SAS are provided.     

Sources of variability and systematic error in mouse timing behavior

In the peak procedure, starts and stops in responding bracket the target time at which food is expected. The variability in start and stop times is proportional to the target time (scalar variability), as is the systematic error in the mean center (scalar error). The authors investigated the source of the error and the variability, using head poking in the mouse, with target intervals of 5 s, 15 s, and 45 s, in the standard procedure, and in a variant with 3 different target intervals at 3 different locations in a single trial. The authors conclude that the systematic error is due to the asymmetric location of start and stop decision criteria, and the scalar variability derives primarily from sources other than memory.     

Personnel Selection with Top‐Score‐Referenced Banding: On the Inappropriateness of Current Procedures

The method of selecting among job applicants using statistically based banding has been proposed over the last 10 years as a way to increase workforce diversity. The method continues to be reviewed by academics and considered by practitioners. Although the goal of increasing workforce diversity is important, statistical banding of scores remains controversial. We present a set of unique, statistically and theoretically based criticisms of a form of banding (top‐score‐referenced banding) that is widely used in hundreds of jobs in the public sector throughout the United States. We suggest that even within the premises of such banding, the wrong formula is used to estimate the standard error of measurement and standard error of the difference. One consequence is that too many individuals are labeled as essentially equal with respect to test scores. A related consequence is that test scores within a single band are statistically different and should therefore be treated as such for selection purposes. A more logically and statistically defensible procedure for responding to diversity concerns is to continue to attend to adverse impact issues at each step of the recruiting and test development process.     

Testing for robustness in Monte Carlo studies

Monte Carlo studies provide the information needed to help researchers select appropriate analytical procedures under design conditions in which the underlying assumptions of the procedures are not met. In Monte Carlo studies, the 2 errors that one could commit involve (a) concluding that a statistical procedure is robust when it is not or (b) concluding that it is not robust when it is. In previous attempts to apply standard statistical design principles to Monte Carlo studies, the less severe of these errors has been wrongly designated the Type I error. In this article, a method is presented for controlling the appropriate Type I error rate; the determination of the number of iterations required in a Monte Carlo study to achieve desired power is described; and a confidence interval for a test's true Type I error rate is derived. A robustness criterion is also proposed that is a compromise between W. G. Cochran's (1952) and J. V. Bradley's (1978) criteria.     

Constant errors occur in matched reproduction of angles even when likely biases are eliminated

The literature on the reproduction of visually perceived angles often seems to contain an explicit or implicit assumption that systematic errors are caused by misperception of the sizes of the angles. We argue that in the absence of procedural artifacts no consistent error should arise from purely perceptual causes. Accordingly, an experimental procedure was designed to eliminate or evaluate a number of artifacts that probably affected earlier experiments. A computer-controlled display generated both a standard angle and a variable one that could be adjusted by a subject to match the standard. The data analysis followed Cox’s (1970) procedure for the analysis of binary data to focus on the numbers of positive and negative errors rather than on their magnitudes and so eliminate the numerically distorting effects of asymmetrical error distributions. Nevertheless, the results showed what has usually been reported before: acute angles were matched with too large a comparison angle, and obtuse ones with too small an angle. Thus, there is a real effect to be explained, although the explanation must invoke response processes.     

A note concerning the creager—Valentine paper

TheU _i statistic, discussed by Creager and Valentine in a recent paper, is defined as a standard error of estimate for the regression of a composite, consisting ofn components, on then–1 components, exclusive of thejth component. It is shown that this standard error equals the standard error of estimate for the prediction of thejth component from the othern–1 components. This equality is used to suggest an efficient procedure to compute all primary uniquenessesU _i .     

An improved Hochberg procedure for multiple tests of significance

We propose a simple modification of Hochberg's step‐up Bonferroni procedure for multiple tests of significance. The proposed procedure is always more powerful than Hochberg's procedure for more than two tests, and is more powerful than Hommel's procedure for three and four tests. A numerical analysis of the new procedure indicates that its Type I error is controlled under independence of the test statistics, at a level equal to or just below the nominal Type I error. Examination of various non‐null configurations of hypotheses shows that the modified procedure has a power advantage over Hochberg's procedure which increases in relationship to the number of false hypotheses.     

Mimicking aphasic semantic errors in normal speech production: evidence from a novel experimental paradigm

Semantic errors are commonly found in semantic dementia (SD) and some forms of stroke aphasia and provide insights into semantic processing and speech production. Low error rates are found in standard picture naming tasks in normal controls. In order to increase error rates and thus provide an experimental model of aphasic performance, this study utilised a novel method- tempo picture naming. Experiment 1 showed that, compared to standard deadline naming tasks, participants made more errors on the tempo picture naming tasks. Further, RTs were longer and more errors were produced to living items than non-living items a pattern seen in both semantic dementia and semantically-impaired stroke aphasic patients. Experiment 2 showed that providing the initial phoneme as a cue enhanced performance whereas providing an incorrect phonemic cue further reduced performance. These results support the contention that the tempo picture naming paradigm reduces the time allowed for controlled semantic processing causing increased error rates. This experimental procedure would, therefore, appear to mimic the performance of aphasic patients with multi-modal semantic impairment that results from poor semantic control rather than the degradation of semantic representations observed in semantic dementia [Jefferies, E. A., & Lambon Ralph, M. A. (2006). Semantic impairment in stoke aphasia vs. semantic dementia: A case-series comparison. Brain, 129, 2132-2147]. Further implications for theories of semantic cognition and models of speech processing are discussed.     

Statistical inference for false positive and false negative error rates in mastery testing

This paper describes an asymptotic inferential procedure for the estimates of the false positive and false negative error rates. Formulas and tables are described for the computations of the standard errors. A simulation study indicates that the asymptotic standard errors may be used even with samples of 25 cases as long as the Kuder-Richardson Formula 21 reliability is reasonably large. Otherwise, a large sample would be required.This work was performed pursuant to Grant No NIE-G-78-0087 with the National Institute of Education, Department of Health, Education and Welfare, Huynh Huynh, Principal Investigator. Points of view or opinions stated do not necessarily reflect NIE position or policy and no official endorsement should be inferred. The editorial assistance of Joseph C. Saunders is gratefully acknowledged.     

Traditional and proposed tests of slope homogeneity for non-normal and heteroscedastic data

The purpose of this study was to develop and compare tests of independent groups' slopes for non-normal distributions and heteroscedastic variances, including slope tests based on least squares, Theil-Sen, and trimmed estimation approaches. A slope test based on jackknife standard error estimates is proposed that can utilize each of the traditional estimation methods while also addressing problematic aspects of methods' standard error estimates. Simulations demonstrate that the proposed jackknife-based slope tests can improve standard error estimation, Type I error, and power relative to the traditional slope tests.     

A Direct Method for Obtaining Approximate Standard Error and Confidence Interval of Maximal Reliability for Composites With Congeneric Measures

Unlike a substantial part of reliability literature in the past, this article is concerned with weighted combinations of a given set of congeneric measures with uncorrelated errors. The relationship between maximal coefficient alpha and maximal reliability for such composites is initially dealt with, and it is shown that the former is a lower bound of the latter. A direct method for obtaining approximate standard error and confidence interval for maximal reliability is then outlined. The procedure is based on a second-order Taylor series approximation and is readily and widely applicable in empirical research via use of covariance structure modeling. The described method is illustrated with a numerical example.     

Robust mean and covariance structure analysis through iteratively reweighted least squares

Robust schemes in regression are adapted to mean and covariance structure analysis, providing an iteratively reweighted least squares approach to robust structural equation modeling. Each case is properly weighted according to its distance, based on first and second order moments, from the structural model. A simple weighting function is adopted because of its flexibility with changing dimensions. The weight matrix is obtained from an adaptive way of using residuals. Test statistic and standard error estimators are given, based on iteratively reweighted least squares. The method reduces to a standard distribution-free methodology if all cases are equally weighted. Examples demonstrate the value of the robust procedure.The authors acknowledge the constructive comments of three referees and the Editor that lead to an improved version of the paper. This work was supported by National Institute on Drug Abuse Grants DA01070 and DA00017 and by the University of North Texas Faculty Research Grant Program.     

A test of the equality of standard errors of measurement

A procedure is proposed for testing the significance of group differences in the standard error of measurement of a psychological test. Wilks' criterion is used to assure that the tests used in ascertaining reliability and hence variance of errors of measurement may be assumed parallel for each group. Votaw's criterion may be used to check whether the test scores of all the groups have the same mean, variance, and covariance. It is possible, however, for the variance and reliability of the test to differ widely from group to group, so that Votaw's criterion is not satisfied even though the variance of errors of measurement stays relatively constant. For this case a modification of Neyman and Pearson's criterion is developed to test agreement among standard errors of measurement despite group differences in mean, variance, and reliability of the test.The author wishes to acknowledge the helpful criticisms of Dr. Harold Gulliksen, who suggested the problem.     

Basic statistics and the inconsistency of multiple comparison procedures.

This paper has two main themes. First, the various statistical measures used in this journal are summarized and their interrelationships described by way of a flow chart. These are the pooled standard deviation, the pooled variance or mean square error (MSE), the standard error of each treatment mean (SEM) and of the difference between two treatment means (SED), and the least difference between two means which is significant at (e.g.) the 5% level of significance (LSD(5%)). The last three measures can be displayed as vertical bars in graphs, and the relationship between the lengths of these bars is graphically illustrated. It is suggested that the LSD is the most useful of these three measures. Second, when the experimenter has no prior hypotheses to be tested using analysis of variance "contrasts," a multiple comparison procedure (MCP) that examines all pair-wise differences between treatment means, may be appropriate. In this paper a fictitious experimental data set is used to compare several well-known McPs by focussing on a particular operating characteristic, the consistency of the results between an overall analysis of all treatments and an analysis of a subset of the experimental treatments. The procedure that behaves best according to this criterion is the unrestricted least significant difference (LSD) procedure. The unrestricted LSD is therefore recommended with the proviso that it be used as a method of generating hypotheses to be tested in subsequent experimentation, not as a method that attempts to simultaneously formulate and test hypotheses.     

A one-step method for modelling longitudinal data with differential equations

Differential equation models are frequently used to describe non-linear trajectories of longitudinal data. This study proposes a new approach to estimate the parameters in differential equation models. Instead of estimating derivatives from the observed data first and then fitting a differential equation to the derivatives, our new approach directly fits the analytic solution of a differential equation to the observed data, and therefore simplifies the procedure and avoids bias from derivative estimations. A simulation study indicates that the analytic solutions of differential equations (ASDE) approach obtains unbiased estimates of parameters and their standard errors. Compared with other approaches that estimate derivatives first, ASDE has smaller standard error, larger statistical power and accurate Type I error. Although ASDE obtains biased estimation when the system has sudden phase change, the bias is not serious and a solution is also provided to solve the phase problem. The ASDE method is illustrated and applied to a two-week study on consumers’ shopping behaviour after a sale promotion, and to a set of public data tracking participants’ grammatical facial expression in sign language. R codes for ASDE, recommendations for sample size and starting values are provided. Limitations and several possible expansions of ASDE are also discussed.