Multivariate multiple regression analyses: a permutation method for linear models

A multivariate extension of a univariate procedure for the analysis of experimental designs is presented. A Euclidean-distance permutation procedure is used to evaluate multivariate residuals obtained from a regression algorithm, also based on Euclidean distances. Applications include various completely randomized and randomized block experimental designs such as one-way, Latin square, factorial, nested, and split-plot designs, with and without covariates. Unlike parametric procedures, the only required assumption is the randomization of subjects to treatments.     

Structural Equation Modeling Approaches for Analyzing Partially Nested Data

Study designs involving clustering in some study arms, but not all study arms, are common in clinical treatment-outcome and educational settings. For instance, in a treatment arm, persons may be nested in therapy groups, whereas in a control arm there are no groups. Methodological approaches for handling such partially nested designs have recently been developed in a multilevel modeling framework (MLM-PN) and have proved very useful. We introduce two alternative structural equation modeling (SEM) approaches for analyzing partially nested data: a multivariate single-level SEM (SSEM-PN) and a multiple-arm multilevel SEM (MSEM-PN). We show how SSEM-PN and MSEM-PN can produce results equivalent to existing MLM-PNs and can be extended to flexibly accommodate several modeling features that are difficult or impossible to handle in MLM-PNs. For instance, using an SSEM-PN or MSEM-PN, it is possible to specify complex structural models involving cluster-level outcomes, obtain absolute model fit, decompose person-level predictor effects in the treatment arm using latent cluster means, and include traditional factors as predictors/outcomes. Importantly, implementation of such features for partially nested designs differs from that for fully nested designs. An empirical example involving a partially nested depression intervention combines several of these features in an analysis of interest for treatment-outcome studies.     

Some developments in multivariate generalizability

This article is concerned with estimation of components of maximum generalizability in multifacet experimental designs involving multiple dependent measures. Within a Type II multivariate analysis of variance framework, components of maximum generalizability are defined as those composites of the dependent measures that maximize universe score variance for persons relative to observed score variance. The coefficient of maximum generalizability, expressed as a function of variance component matrices, is shown to equal the squared canonical correlation between true and observed scores. Emphasis is placed on estimation of variance component matrices, on the distinction between generalizability- and decision-studies, and on extension to multifacet designs involving crossed and nested facets. An example of a two-facet partially nested design is provided.Appreciation is expressed to the Office of Research in Medical Education, University of Texas Medical Branch, for permitting use of their data.     

A simple test for heterogeneity of variance in complex factorial designs

A simple procedure for testing heterogeneity of variance is developed which generalizes readily to complex, multi-factor experimental designs. Monte Carlo Studies indicate that the Z-variance test statistic presented here yields results equivalent to other familiar tests for heterogeneity of variance in simple one-way designs where comparisons are feasible. The primary advantage of the Z-variance test is in the analysis of factorial effects on sample variances in more complex designs. An example involving a three-way factorial design is presented.     

Estimation of maximal reliability for multiple‐component instruments in multilevel designs

A procedure for point and interval estimation of maximal reliability of multiple‐component measuring instruments in multi‐level settings is outlined. The approach is applicable to hierarchical designs in which individuals are nested within higher‐order units and exhibit possibly related performance on components of a given homogeneous scale. The method is developed within the framework of multi‐level factor analysis. The proposed procedure is illustrated with an empirical example.     

Ill-structured measurement designs in organizational research: implications for estimating interrater reliability

Organizational research and practice involving ratings are rife with what the authors term ill-structured measurement designs (ISMDs)--designs in which raters and ratees are neither fully crossed nor nested. This article explores the implications of ISMDs for estimating interrater reliability. The authors first provide a mock example that illustrates potential problems that ISMDs create for common reliability estimators (e.g., Pearson correlations, intraclass correlations). Next, the authors propose an alternative reliability estimator--G(q,k)--that resolves problems with traditional estimators and is equally appropriate for crossed, nested, and ill-structured designs. By using Monte Carlo simulation, the authors evaluate the accuracy of traditional reliability estimators compared with that of G(q,k) for ratings arising from ISMDs. Regardless of condition, G(q,k) yielded estimates as precise or more precise than those of traditional estimators. The advantage of G(q,k) over the traditional estimators became more pronounced with increases in the (a) overlap between the sets of raters that rated each ratee and (b) ratio of rater main effect variance to true score variance. Discussion focuses on implications of this work for organizational research and practice.     

FIXED EFFECTS ANALYSIS OF VARIANCE BY REGRESSION ANALYSIS

This paper is concerned with the solution of typical analysis of variance problems using general purpose multiple regression computer programs. Specific models, restrictions on the parameters for hypothesis testing, and computational aspects are discussed. It is argued that this approach has many pedagogical advantages over traditional procedures.     

Proving Causal Propositions : The Foundations Of Program And Experiment Design

Although causal propositions cannot be proven to the point of incorrigibility, they can be disproven (aside from instrument validity problems) or corroborated. Just how one proceeds to such disproof or corroboration depends upon what his interest is in the causes of his dependent variables' values. Testing and qualifying or restricting a specific causal proposition, developing a comprehensive or variance exhaustive linear causal proposition (or multiple regression equation), and mapping or describing the efficacy of a specific set of Treatments imply somewhat,different programs of re- search and experiment designs. Programs and designs for these three interests or strategies are differentiated in terms of a Factor Lattice of all the ex ante relevant variables. The terms of this analysis refer to the regional locations in, density of coverage of, and allocation of replicates to the selected Lattice intersects and the factorial completeness of the design which they constitute, as well as to the type of control exercised over the Factors: production, selection, or stochastic.     

Data generation and analysis using spreadsheets

Modern spreadsheets are powerful and useful tools that can often replace special-purpose programs for generating data and for student analysis of simple statistical problems. The inherent flexibility of spreadsheets makes them especially convenient and extensible. Several templates for generating and analyzing data fort tests and analysis of variance are discussed. Users can specify then, \(\bar X\) , andSD of two or more groups and then execute macros that generate appropriate data. Overwriting the generated data with “real” values turns the spreadsheet into a data analysis program. Spreadsheets hold promise as valuable instructional supplements for simple designs, but they are less suitable for more complex designs, where special-purpose programs may be more appropriate.     

非正态分布下概化理论方差分量变异量估计

方差分量估计是概化理论的必用技术,但受限于抽样,需要对其变异量进行探讨。采用Monte Carlo数据模拟技术,探讨非正态数据分布对四种方法估计概化理论方差分量变异量的影响。结果表明：（1）不同非正态数据分布下,各种估计方法的“性能”表现出差异性;（2）数据分布对方差分量变异量估计有影响,适合于非正态分布数据的方差分量变异量估计方法不一定适合于正态分布数据。     

Nominal analysis of “variance”

Nominal responses are the natural way for people to report actions or opinions. Because nominal responses do not generate numerical data, they have been underutilized in behavioral research. On those occasions in which nominal responses are elicited, the responses are customarily aggregated over people or trials so that large-sample statistics can be employed. A new analysis is proposed that directly associates differences among responses with particular sources in factorial designs. A pair of nominal responses either matches or does not; when responses do not match, they vary. That analogue to variance is incorporated in the nominal analysis of “variance” (Nanova ) procedure, wherein the proportions of matches associated with sources play the same role as do sums of squares in an anova . The Nanova table is structured like an ANOVA table. The significance levels of the N ratios formed by comparing proportions are determined by resampling. Fictitious behavioral examples featuring independent groups and repeated measures designs are presented. A Windows program for the analysis is available.     

Multilevel factorial experiments for developing behavioral interventions: power, sample size, and resource considerations

Factorial experimental designs have many potential advantages for behavioral scientists. For example, such designs may be useful in building more potent interventions by helping investigators to screen several candidate intervention components simultaneously and to decide which are likely to offer greater benefit before evaluating the intervention as a whole. However, sample size and power considerations may challenge investigators attempting to apply such designs, especially when the population of interest is multilevel (e.g., when students are nested within schools, or when employees are nested within organizations). In this article, we examine the feasibility of factorial experimental designs with multiple factors in a multilevel, clustered setting (i.e., of multilevel, multifactor experiments). We conduct Monte Carlo simulations to demonstrate how design elements-such as the number of clusters, the number of lower-level units, and the intraclass correlation-affect power. Our results suggest that multilevel, multifactor experiments are feasible for factor-screening purposes because of the economical properties of complete and fractional factorial experimental designs. We also discuss resources for sample size planning and power estimation for multilevel factorial experiments. These results are discussed from a resource management perspective, in which the goal is to choose a design that maximizes the scientific benefit using the resources available for an investigation.     

Modern robust statistical methods: an easy way to maximize the accuracy and power of your research

Classic parametric statistical significance tests, such as analysis of variance and least squares regression, are widely used by researchers in many disciplines, including psychology. For classic parametric tests to produce accurate results, the assumptions underlying them (e.g., normality and homoscedasticity) must be satisfied. These assumptions are rarely met when analyzing real data. The use of classic parametric methods with violated assumptions can result in the inaccurate computation of p values, effect sizes, and confidence intervals. This may lead to substantive errors in the interpretation of data. Many modern robust statistical methods alleviate the problems inherent in using parametric methods with violated assumptions, yet modern methods are rarely used by researchers. The authors examine why this is the case, arguing that most researchers are unaware of the serious limitations of classic methods and are unfamiliar with modern alternatives. A range of modern robust and rank-based significance tests suitable for analyzing a wide range of designs is introduced. Practical advice on conducting modern analyses using software such as SPSS, SAS, and R is provided. The authors conclude by discussing robust effect size indices.     

Power analysis for multivariate and repeated measures designs: A flexible approach using the SPSS MANOVA procedure

Although power analysis is an important component in the planning and implementation of research designs, it is often ignored. Computer programs for performing power analysis are available, but most have limitations, particularly for complex multivariate designs. An SPSS procedure is presented that can be used for calculating power for univariate, multivariate, and repeated measures models with and without time-varying and time-constant covariates. Three examples provide a framework for calculating power via this method: an ANCOVA, a MANOVA, and a repeated measures ANOVA with two or more groups. The benefits and limitations of this procedure are discussed.     

COMMONALITY ANALYSIS: A METHOD FOR DECOMPOSING EXPLAINED VARIANCE IN MULTIPLE REGRESSION ANALYSES

Commonality analysis is a procedure for decomposing R² in multiple regression analyses into the percent of variance in the dependent variable associated with each independent variable uniquely, and the proportion of explained variance associated with the common effects of predictors. Commonality analysis thus sheds additional light on the magnitude of an obtained multivariate relationship by identifying the relative importance of all independent variables, findings which can be of theoretical and practical significance. In this paper we offer a brief explication of commonality analysis; a step-by-step discussion of how communication researchers may perform commonality analyses using output from computer-assisted statistical analysis programs like SPSS; and we provide an extended example illustrating a commonality analysis.     

An example of cooperating compact data analysis programs

Some user-oriented compact data analysis programs are described. One program is useful for transforming and reformatting data, and the others perform analysis of variance and multiple regression. Along with other programs not described here, these form an adequate statistical package without sacrificing ease of use or computational power.     

Reliability of multiple‐component measuring instruments: Improved evaluation in repeated measure designs

A covariance structure analysis method for improved point and interval estimation of composite reliability in repeated measure designs is outlined that accounts for specificity variance. The approach also permits the testing of time‐invariance in reliability of multiple‐component instruments in terms of the ratio of ‘pure’ measurement error variance to observed scale score variance. In addition, the procedure allows interval estimation of the difference in composite reliability coefficients across assessment occasions. The method described is illustrated with data from a cognitive intervention study.     

An estimation of true scores in the case of items scored on a continuous scale

For the tests in which the score on an item is not restricted to 0 and 1, but is any number on a continuous scale, a procedure for estimating an examinee's true score is given. For the case of 0, 1 item scoring this problem was considered by Lord [1959]. Following Lord, the least squares estimation procedure is used and the regression coefficient is obtained, which is compared with the generalized KR(20) and KR(21) formulas. Also, results are discussed using analysis of variance models.Now at Brooklyn College of the City University of New York.     

Regression analyses as a tool for studying reading processes: Comment on Just and Carpenters eye fixation theory

Just and Carpenter (1980) presented a theory of reading based on eye fixations wherein their “psycholinguistic” variables accounted for 72% of the variance in word gaze durations. This comment raises some statistical and theoretical problems with their use of simultaneous regression analysis of gaze duration measures and with the resulting theory of reading. A major problem was the confounding of perceptual with psycholinguistic factors. New eye fixation data are presented to support these criticisms. Analysis of fixations within words revealed that most gaze duration variance was contributed by number of fixations rather than by fixation duration.