Testing for independence between pairs of autocorrelated binomial data sequences

A problem arises in analyzing the existence of interdependence between the behavioral sequences of two individuals: tests involving a statistic such as chi-square assume independent observations within each behavioral sequence, a condition which may not exist in actual practice. Using Monte Carlo simulations of binomial data sequences, we found that the use of a chi-square test frequently results in unacceptable Type I error rates when the data sequences are autocorrelated. We compared these results to those from two other methods designed specifically for testing for intersequence independence in the presence of intrasequence autocorrelation. The first method directly tests the intersequence correlation using an approximation of the variance of the intersequence correlation estimated from the sample autocorrelations. The second method uses tables of critical values of the intersequence correlation computed by Nakamuraet al. (J. Am. Stat. Assoc., 1976,71, 214–222). Although these methods were originally designed for normally distributed data, we found that both methods produced much better results than the uncorrected chi-square test when applied to binomial autocorrelated sequences. The superior method appears to be the variance approximation method, which resulted in Type I error rates that were generally less than or equal to 5% when the level of significance was set at .05.     

ESTIMATING CONTENT-ACQUIESCENCE CORRELATION BY COVARIANCE STRUCTURE ANALYSIS

A topic of continuing interest in the measurement area is response acquiescence. A recent study has demonstrated the feasibiliy of studying acquiescence or, more importantly, content/acquiescence correlation in the MMPI. Utilizing the components of variance approach, this study found that the variance due to acquiescence in scores on the Pt and Hg scales was small relative to content variance, but that the correlation between acquiescence and content may be substantial for the Pt scale. The present paper describes a general statistical procedure for investigating content variance, variance due to non-content characteristics of items, and the covariances of content and various item characteristics. The data from a previous paper are reanalyzed, using alternative covariance structure models. Maximum likelihood procedures which allow for a statistical test for parameters of interest are used. The results point to the significance of the content- acquiescence correlation in the Pt scale, but not in the Hy scale. The previous findings are verified statistically, and procedures which hold promise for other investigation into the properties of behavioral tests are described.     

An algebraic solution for the communalities

Factorial analysis begins with ann ×n correlation matrixR, whose principal diagonal entries are unknown. If the common test space of the battery is under investigation, the communality of each test is entered in the appropriate diagonal cell. This value is the portion of the test's variance shared with others in the battery. The communalities must be so estimated thatR will maintain the rank determined by its side entries, after the former have been inserted. Previous methods of estimating the communalities have involved a certain arbitrariness, since they depended on selecting test subgroups or parts of the data inR. A theory is presented showing that this difficulty can be avoided in principle. In its present form, the theory is not offered as a practical computing procedure. The basis of the new method lies in the Cayley-Hamilton theorem: Any square matrix satisfies its own characteristic equation.     

Correcting an analysis of variance for clustering

A great deal of educational and social data arises from cluster sampling designs where clusters involve schools, classrooms, or communities. A mistake that is sometimes encountered in the analysis of such data is to ignore the effect of clustering and analyse the data as if it were based on a simple random sample. This typically leads to an overstatement of the precision of results and too liberal conclusions about precision and statistical significance of mean differences. This paper gives simple corrections to the test statistics that would be computed in an analysis of variance if clustering were (incorrectly) ignored. The corrections are multiplicative factors depending on the total sample size, the cluster size, and the intraclass correlation structure. For example, the corrected F statistic has Fisher's F distribution with reduced degrees of freedom. The corrected statistic reduces to the F statistic computed by ignoring clustering when the intraclass correlations are zero. It reduces to the F statistic computed using cluster means when the intraclass correlations are unity, and it is in between otherwise. A similar adjustment to the usual statistic for testing a linear contrast among group means is described.     

Genetic Influence on the Variance in Coincidence Timing and its Covariance with IQ: A Twin Study

Performance measures on a coincidence timing task have previously been associated with psychometric IQ suggesting that the ability of an individual to devote processing resources at the required time may account for some of intelligence test variance. Using the twin design, this study investigates whether genetic variability explains some of the variance in coincidence timing and whether common genetic factors account for the association with intellectual functioning. Fifty-five 16-year old twin pairs (28 MZ, 27 DZ) were tested. Individual differences in number of hits (HITS), mean absolute error (MAE), and intra-individual trial-to-trial consistency (SD) were significantly influenced by genetic factors, accounting for approximately 50 percent of the variance. The correlation between coincidence timing and psychometric IQ was confirmed and ranged from 0.11 to 0.53 with a mean correlation of 0.33. In the limited sample, the correlation between IQ and the coincidence timing measures appeared to be mediated largely by a unique environmental factor, with only a small loading of SD on the genetic factor influencing IQ. However, as the confidence intervals in the other genetic cross loadings are large, we cannot exclude the possibility of a much stronger genetic influence.     

Disentangling stability, variability and adaptability in human performance: focus on the interplay between local variance and serial correlation

We address the complex relationship between the stability, variability, and adaptability of psychological systems by decomposing the global variance of serial performance into two independent parts: the local variance (LV) and the serial correlation structure. For two time series with equal LV, the presence of persistent long-range correlations (or 1/? β noise) increases the global variance. We hypothesized that a coadjustment between these two determinants of variability constitutes a resource for adaptive systems whose appropriate functioning under critical conditions requires the outcome variance to be limited. To test this hypothesis, we looked at the bimanual coordination dynamics at comfortable (stable) and critical (close to phase transition) frequencies. Results showed that a negative correlation appeared gradually as the theoretical stability of coordination modes decreased and reached significance only in the critical condition. We propose that the emergence of a mutual adjustment between LV and serial correlations might be an indicator of effective adaptation to stabilize behavior.     

Comparing subtest profiles of g loadings and correlations with RT measures

It is argued that if a response time (RT) measure correlates with psychometric test (PT) scores because it shares variance in common with general intelligence, g, then the profile of g loadings for a set of PTs would be predictable from the profile of correlations between the RT measure and the PT scores. On the other hand, if an RT correlates with PTs through variance not shared with g, the g loadings of the PTs should be unrelated to the correlations between RT and PTs. The profiles of g loadings and correlations with RT were compared for a set of 6 RT measures and 8 PT scores from Smith and Stanley (1983). To demonstrate statistical significance, standard errors of the statistics were generated by Efron's bootstrap technique (Efron, 1979). It was clearly shown that the profile of the PT's g loadings could be well predicted from the RT-PT correlations for four of the RT measures. It can be concluded that RT tasks do measure general intelligence. Analysis of the errors in prediction suggested that the RTs may correlate more with fluid than crystallized intelligence.     

Maximizing the discriminating power of a multiple-score test

Maximizing the discriminating power of a multiple-score test involves maximizing the homogeneity of each subtest and minimizing the correlations between subtests. A method is presented for constructing such tests from items whose intercorrelations are not too high. Under certain restrictions the saturation, defined as the proportion of inter-item covariance to total variance, is maximized for each subtest. The nucleus of each subtest is three items with high covariancesinter se. All items which will lower the saturation are discarded; the one item is added which will maximize the saturation of the resultant test. This process is repeated until all the items are included or discarded for that subtest. If the correlation between any such subtests approaches the geometric mean of their saturations, their items form a new pool for one or more subtests. Formulas are presented for deciding which items to eliminate in order to reduce further the correlations between subtests.This research was supported in part by the United States Air Force under Contract AF 33(038)-10588 with Human Resources Research Center, Lackland Air Force Base, San Antonio, Texas. Permission is granted for reproduction, translation, publication use and disposal in whole and in part by or for the United States Government.     

Student Teachers' Classroom Management Anxiety: A Study on Behavior Management and Teaching Management

The purpose of this study is to examine student teachers' anxiety related to classroom management in terms of behavior and teaching management. The study is carried out with the participation of 700 student teachers attending Ziya Gökalp Education Faculty at Dicle University in the 2007–2008 academic year. In order to determine their anxiety levels concerning behavior management and teaching management, the scales of “behavior management anxiety” and “teaching management anxiety” are used. Data are analyzed with correlation (Pearson), t test, and analysis of variance techniques. The Scheffé test is used to test significance. According to the student teachers' participation to school experience and to their branches, significant differences are determined between all subscales of behavior management anxiety levels.     

The Correlation of General Cognitive Ability and Psychomotor Tracking Tests

A study was conducted to investigate the nexus of cognitive and psychomotor tests as might be used for personnel selection and assessment. These two domains are frequently seen as independent. A multiple aptitude cognitive test battery and a psychomotor test battery were administered to 354 United States Air Force recruits. The average multiple correlation of the cognitive tests and each psychomotor score as a criterion was 0.34, corrected for range restriction. Confirmatory factor analyses disclosed general cognitive and general psychomotor factors, three lower-order psychomotor factors, and two lower-order cognitive factors. The general cognitive factor accounted for 39% of the variance and the general psychomotor factor accounted for 29% of the variance. Residualized, the lower-order factors accounted for between 10% and 3% of the variance. The average g saturations (loadings) of the cognitive and psychomotor tests were 0.82 and 0.34 respectively. An implication for personnel selection is that the incremental validity of psychomotor tracking tests beyond the validity of cognitive tests will be small due to the commonality of measurement. A further implication of findings is the need to study the validity of the general and specific psychomotor factors.     

A test of the equality of standard errors of measurement

A procedure is proposed for testing the significance of group differences in the standard error of measurement of a psychological test. Wilks' criterion is used to assure that the tests used in ascertaining reliability and hence variance of errors of measurement may be assumed parallel for each group. Votaw's criterion may be used to check whether the test scores of all the groups have the same mean, variance, and covariance. It is possible, however, for the variance and reliability of the test to differ widely from group to group, so that Votaw's criterion is not satisfied even though the variance of errors of measurement stays relatively constant. For this case a modification of Neyman and Pearson's criterion is developed to test agreement among standard errors of measurement despite group differences in mean, variance, and reliability of the test.The author wishes to acknowledge the helpful criticisms of Dr. Harold Gulliksen, who suggested the problem.     

Mathematics beliefs and achievement of adolescent students in Japan: results from the TIMSS 1999 assessment

A recent study (1) of undergraduate students in a precalculus course indicated that they expressed slightly positive attitudes toward mathematics. It is important, however, to examine relationships between students' initial attitudes and achievement outcomes. The present purpose was to assess the relationship between self-beliefs and mathematics achievement for a large national sample of students from the TIMSS 1999 international sample (eighth graders) from Japan. Several significant relationships between mathematics beliefs and test scores were noted. In addition, the overall multiple regression equation that assessed the joint significance of the complete set of self-belief variables was significant (F7.65 = 159.48, p < .001) and explained 20.6% of the variance in mathematics achievement test scores.     

Effect of non-normality on test statistics for one-way independent groups designs

The data obtained from one‐way independent groups designs is typically non‐normal in form and rarely equally variable across treatment populations (i.e. population variances are heterogeneous). Consequently, the classical test statistic that is used to assess statistical significance (i.e. the analysis of variance F test) typically provides invalid results (e.g. too many Type I errors, reduced power). For this reason, there has been considerable interest in finding a test statistic that is appropriate under conditions of non‐normality and variance heterogeneity. Previously recommended procedures for analysing such data include the James test, the Welch test applied either to the usual least squares estimators of central tendency and variability, or the Welch test with robust estimators (i.e. trimmed means and Winsorized variances). A new statistic proposed by Krishnamoorthy, Lu, and Mathew, intended to deal with heterogeneous variances, though not non‐normality, uses a parametric bootstrap procedure. In their investigation of the parametric bootstrap test, the authors examined its operating characteristics under limited conditions and did not compare it to the Welch test based on robust estimators. Thus, we investigated how the parametric bootstrap procedure and a modified parametric bootstrap procedure based on trimmed means perform relative to previously recommended procedures when data are non‐normal and heterogeneous. The results indicated that the tests based on trimmed means offer the best Type I error control and power when variances are unequal and at least some of the distribution shapes are non‐normal.     

A statistical test for means of samples from skew populations

This paper presents a test for determining significance of differences between means of samples which are drawn from positively skewed populations, more specifically, those having a Pearson Type III distribution function. The quantity 2npx _g /x _p (wherep equals the mean squared divided by the variance andn is the number of cases in the sample), which distributes itself as Chi Square for 2np degrees of freedom, may be referred to the tables of Chi Square for testing hypotheses about the value of the true mean. For two independent samples, the larger mean divided by the smaller mean, which distributes itself asF for 2n ₁ p ₁ and 2n ₂ p ₂ degrees of freedom, may be referred to theF distribution tables for testing significance of difference between means. The test assumes that the range of possible scores is from zero to infinity. When a lower theoretical score limit (c) exists which is not zero, the quantity (Mean —c) should be used instead of the mean in all calculations.     

Perceived Parental Acceptance-Rejection and the Development of Children's Locus of Control

In a study of relationships between locus of control and perceived parental acceptance-rejection as they vary by age, 271 boys and girls from 9 through 11 years of age were administered the Nowicki-Strickland Locus of Control Scale, short form, and the Parental Acceptance-Rejection Questionnaire. A multiple regression/correlation analysis revealed that internality increased significantly with children's age and with children's perceptions of increased parental acceptance. Moreover, age and parenting jointly accounted for 22% of the variance in children's locus of control scores.     

The effect of perceived stress on the relationship between perfectionism and burnout in coaches

Abstract

The effects of perceived stress (PS) on the relationship between adaptive and maladaptive forms of perfectionism and burnout were examined. Smith's (1986) stress appraisal model and Kelley, Eklund, and Ritter-Taylor's (1999) model of coach burnout were used to test two models of burnout in a sample of college coaches (N=177). The results indicated that there is an indirect effect of self-evaluative perfectionism (i.e., maladaptive form of perfectionism) on burnout through PS as well as a significant direct link to burnout, accounting for 56% of its variance. In contrast, conscientious perfectionism (CP) (i.e., adaptive perfectionism) did not directly impact burnout, nor was there an indirect effect through PS. Based on Lazarus's (1999) ideas about stress appraisal, the results suggested that maladaptive forms of perfectionism resulted in more threatening perceptions of stress, thus, potentially leading to the experience of burnout. However, adaptive forms of perfectionism did not seem to result in increased appraisals of stress or result in burnout. The results did indicate a significant correlation between the two forms of perfectionism, which may explain why CP did not significantly impact PS or burnout.     

Further evaluating the conditional decision rule for comparing two independent means

Many books on statistical methods advocate a ‘conditional decision rule’ when comparing two independent group means. This rule states that the decision as to whether to use a ‘pooled variance’ test that assumes equality of variance or a ‘separate variance’ Welch t test that does not should be based on the outcome of a variance equality test. In this paper, we empirically examine the Type I error rate of the conditional decision rule using four variance equality tests and compare this error rate to the unconditional use of either of the t tests (i.e. irrespective of the outcome of a variance homogeneity test) as well as several resampling‐based alternatives when sampling from 49 distributions varying in skewness and kurtosis. Several unconditional tests including the separate variance test performed as well as or better than the conditional decision rule across situations. These results extend and generalize the findings of previous researchers who have argued that the conditional decision rule should be abandoned.     

Predicted differences and differences between predictions

WhenK tests are given toN individuals, and for each individual there are two criterion measures, then (1) the multiple regression weight to be applied to the standard score for each test to predict the criterion-difference score equals the difference of the weights for predicting each criterion separately; (2) the difference between the predicted scores equals the predicted difference (each test being assigned the appropriate multiple regression weight); (3) the square of the multiple correlation between predicted and actual criterion-difference scores equals the sum of squares of the multiple correlations of the battery with each criterion less the product of these correlations and the correlation between predicted scores all divided by twice the quantity one minus the criterion intercorrelation; and (4) the variance of errors of estimating the criterion-difference score equals the sum of the variances of errors of estimating each criterion score minus twice the criterion intercorrelation, plus twice the correlation between predicted scores multiplied by the product of the square root of one minus the variance of errors of estimating one criterion and the corresponding square root for the second criterion.The author wishes to express his appreciation for the suggestions and guidance given by Dr. Harold Gulliksen in the preparation of this article. He also wishes to acknowledge the helpful comments of Dr. Paul Horst and Dr. Ledyard Tucker on certain phases of the development.     

The relationship between factorial composition of test items and measures of test reliability

For continuous distributions associated with dichotomous item scores, the proportion of common-factor variance in the test,H ², may be expressed as a function of intercorrelations among items.H ² is somewhat larger than the coefficienta except when the items have only one common factor and its loadings are restricted in value. The dichotomous item scores themselves are shown not to have a factor structure, precluding direct interpretation of the Kuder-Richardson coefficient,r _K-R, in terms of factorial properties. The value ofr _K-R is equal to that of a coefficient of equivalence,H ² , when the mean item variance associated with common factors equals the mean interitem covariance. An empirical study with synthetic test data from populations of varying factorial structure showed that the four parameters mentioned may be adequately estimated from dichotomous data.This study was supported in part by an Air Force project (Contract Number AF18(600-170), monitored by the Crew Research Laboratory, Air Force Personnel and Training Research Center, Randolph Air Force Base, Randolph Field, Texas. Permission is granted for reproduction, translation, publication, use and disposal in whole and in part by or for the United States Government. Further support was given by the Northwestern University Graduate School. The computational assistance of Mr. Norman Miller is acknowledged. Professor Meyer Dwass provided mathematical advice both directly and indirectly relevant to the paper.