On normal theory based inference for multilevel models with distributional violations

Data in social and behavioral sciences are often hierarchically organized though seldom normal, yet normal theory based inference procedures are routinely used for analyzing multilevel models. Based on this observation, simple adjustments to normal theory based results are proposed to minimize the consequences of violating normality assumptions. For characterizing the distribution of parameter estimates, sandwich-type covariance matrices are derived. Standard errors based on these covariance matrices remain consistent under distributional violations. Implications of various covariance estimators are also discussed. For evaluating the quality of a multilevel model, a rescaled statistic is given for both the hierarchical linear model and the hierarchical structural equation model. The rescaled statistic, improving the likelihood ratio statistic by estimating one extra parameter, approaches the same mean as its reference distribution. A simulation study with a 2-level factor model implies that the rescaled statistic is preferable.This research was supported by grants DA01070 and DA00017 from the National Institute on Drug Abuse and a University of North Texas faculty research grant. We would like to thank the Associate Editor and two reviewers for suggestions that helped to improve the paper.     

Concise formulas for the standard errors of component loading estimates

Concise formulas for the asymptotic standard errors of component loading estimates were derived. The formulas cover the cases of principal component analysis for unstandardized and standardized variables with orthogonal and oblique rotations. The formulas can be used under any distributions for observed variables as long as the asymptotic covariance matrix for sample covariances/correlations is available. The estimated standard errors in numerical examples were shown to be equivalent to those by the methods using information matrices.The author is indebted to anonymous reviewers for the corrections and suggestions on this study, which have led to improvements of earlier versions of this article.     

A unified approach to exploratory factor analysis with missing data,nonnormal data,and in the presence of outliers

Factor analysis is regularly used for analyzing survey data. Missing data, data with outliers and consequently nonnormal data are very common for data obtained through questionnaires. Based on covariance matrix estimates for such nonstandard samples, a unified approach for factor analysis is developed. By generalizing the approach of maximum likelihood under constraints, statistical properties of the estimates for factor loadings and error variances are obtained. A rescaled Bartlett-corrected statistic is proposed for evaluating the number of factors. Equivariance and invariance of parameter estimates and their standard errors for canonical, varimax, and normalized varimax rotations are discussed. Numerical results illustrate the sensitivity of classical methods and advantages of the proposed procedures.This project was supported by a University of North Texas Faculty Research Grant, Grant #R49/CCR610528 for Disease Control and Prevention from the National Center for Injury Prevention and Control, and Grant DA01070 from the National Institute on Drug Abuse. The results do not necessarily represent the official view of the funding agencies. The authors are grateful to three reviewers for suggestions that improved the presentation of this paper.     

Interval length and time-use by children with AD/HD: a comparison of four models

Predictions made by 4 competing models of time use by children with AD/HD were tested using a computerized version of the Matching Familiar Figures Test in 2 studies. In Study 1 teacher-identified AD/HD children (N = 10) and non-AD/HD controls (N = 10) completed the task under 3 different trial duration conditions (5, 10, and 15 s). In Study 2 the same task was completed by a group of children with a diagnosis of Hyperkinetic Disorder (N = 12) and controls (N = 12). In both studies AD/HD children performed poorly on trials of both 5- and 15-s duration but as well as controls on the 10-s trials. This quadratic function provided support for the State Regulation Deficit model of time use in AD/HD. The value of tailoring the temporal features of learning contexts to the conceptual style of AD/HD children is discussed.     

The "Natural" and the "Formal"

The paper presents an argument against a metaphysical conception of logic according to which logic spells out a specific kind of mathematical structure that is somehow inherently related to our factual reasoning. In contrast, it is argued that it is always an empirical question as to whether a given mathematical structure really does captures a principle of reasoning. (More generally, it is argued that it is not meaningful to replace an empirical investigation of a thing by an investigation of its a priori analyzable structure without paying due attention to the question of whether it really is the structure of the thing in question.) It is proposed to elucidate the situation by distinguishing two essentially different realms with which our reason must deal: the realm of the natural, constituted by the things of our empirical world, and the realm of the formal, constituted by the structures that we use as prisms to view, to make sense of, and to reconstruct the world. It is suggested that this vantage point may throw light on many foundational problems of logic.     

Likelihood inference on the underlying structure of IRT models

The assumptions underlying item response theory (IRT) models may be expressed as a set of equality and inequality constraints on the parameters of a latent class model. It is well known that the same assumptions imply that the parameters of the manifest distribution have to satisfy a more complicated set of inequality constraints which, however, are necessary but not sufficient. In this paper, we describe how the theory for likelihood-based inference under equality and inequality constraints may be used to test the underlying assumptions of IRT models. It turns out that the analysis based directly on the latent structure is simpler and more flexible. In particular, we indicate how several interesting extensions of the Rasch model may be obtained by partial relaxation of the basic constraints. An application to a data set provided by Educational Testing Service is used to illustrate the approach.We thank Dr. Gorman and Dr. Rogers of the Educational Testing Service for providing the data analyzed in Section 4. We also thank three reviewers for comments and suggestions.This revised article was published online in August 2005 with the PDF paginated correctly.     

Translating Toulmin Diagrams: Theory Neutrality in Argument Representation

The Toulmin diagram layout is very familiar and widely used, particularly in the teaching of critical thinking skills. The conventional box-and-arrow diagram is equally familiar and widespread. Translation between the two throws up a number of interesting challenges. Some of these challenges (such as the relationship between Toulmin warrants and their counterparts in traditional diagrams) represent slightly different ways of looking at old and deep theoretical questions. Others (such as how to allow Toulmin diagrams to be recursive) are diagrammatic versions of questions that have already been addressed in artificial intelligence models of argument. But there are further questions (such as the relationships between refutations, rebuttals and undercutters, and the roles of multiple warrants) that are posed as a specific result of examining the diagram inter-translation problem. These three classes of problems are discussed. To the first class are addressed solutions based on engineering pragmatism; to the second class, are addressed solutions drawn from the appropriate literature; and to the third class, fuller exploration is offered justifying the approaches taken in developing solutions that offer both pragmatic utility and theoretical interest. Finally, these solutions are explored briefly in the context of the Araucaria system, showing the ways in which analysts can tackle arguments either using one diagrammatic style or another, or even a combination of the two.     

“Seeing Conversion Whole”: Testing a Model of Religious Conversion

Rambo (1993) theorized that religious conversion consists of a process involving seven dimensions, labeled context, crisis, quest, encounter, interaction, commitment, and consequences. To test the Rambo model, a new measure, the 97-item Adult Religious Conversion Experiences Questionnaire (ARCEQ), was developed, revised on the basis of feedback from a focus group, and administered by mail to 110 adult volunteers who self-identified as having experienced conversion. Reliability analysis of the subscales of the ARCEQ resulted in five (crisis, quest, encounter, commitment, and consequences) meeting the criterion ( > .80). A sixth (interaction) achieved a reliability of .76 and was retained, but the seventh (context) was dropped. Principal components factor analysis with Varimax rotation produced factors labeled Redemptive Love, Zealous Dedication, Dysphoric Need, Openness to Uncertainty, Extrinsic Crisis, and Experiential Learning, which could readily be interpreted in relation to Rambo's model. Subsidiary analyses showed a number of significant correlations between demographic characteristics of the sample and differences in religious conversion experience.     

Assessing the distinguishability of models and the informativeness of data

A difficulty in the development and testing of psychological models is that they are typically evaluated solely on their ability to fit experimental data, with little consideration given to their ability to fit other possible data patterns. By examining how well model A fits data generated by model B, and vice versa (a technique that we call landscaping), much safer inferences can be made about the meaning of a model's fit to data. We demonstrate the landscaping technique using four models of retention and 77 historical data sets, and show how the method can be used to: (1) evaluate the distinguishability of models, (2) evaluate the informativeness of data in distinguishing between models, and (3) suggest new ways to distinguish between models. The generality of the method is demonstrated in two other research areas (information integration and categorization), and its relationship to the important notion of model complexity is discussed.