A Comparative Investigation of Confidence Intervals for IndependentVariables in Linear Regression

In linear regression, the most appropriate standardized effect size for individual independent variables having an arbitrary metric remains open to debate, despite researchers typically reporting a standardized regression coefficient. Alternative standardized measures include the semipartial correlation, the improvement in the squared multiple correlation, and the squared partial correlation. No arguments based on either theoretical or statistical grounds for preferring one of these standardized measures have been mounted in the literature. Using a Monte Carlo simulation, the performance of interval estimators for these effect-size measures was compared in a 5-way factorial design. Formal statistical design methods assessed both the accuracy and robustness of the four interval estimators. The coverage probability of a large-sample confidence interval for the semipartial correlation coefficient derived from Aloe and Becker was highly accurate and robust in 98% of instances. It was better in small samples than the Yuan-Chan large-sample confidence interval for a standardized regression coefficient. It was also consistently better than both a bootstrap confidence interval for the improvement in the squared multiple correlation and a noncentral interval for the squared partial correlation.     

Confidence intervals for standardized linear contrasts of means

Most psychology journals now require authors to report a sample value of effect size along with hypothesis testing results. The sample effect size value can be misleading because it contains sampling error. Authors often incorrectly interpret the sample effect size as if it were the population effect size. A simple solution to this problem is to report a confidence interval for the population value of the effect size. Standardized linear contrasts of means are useful measures of effect size in a wide variety of research applications. New confidence intervals for standardized linear contrasts of means are developed and may be applied to between-subjects designs, within-subjects designs, or mixed designs. The proposed confidence interval methods are easy to compute, do not require equal population variances, and perform better than the currently available methods when the population variances are not equal.     

An effect size measure and Bayesian analysis of single-case designs

This article describes a linear modeling approach for the analysis of single-case designs (SCDs). Effect size measures in SCDs have been defined and studied for the situation where there is a level change without a time trend. However, when there are level and trend changes, effect size measures are either defined in terms of changes in R² or defined separately for changes in slopes and intercept coefficients. We propose an alternate effect size measure that takes into account changes in slopes and intercepts in the presence of serial dependence and provides an integrated procedure for the analysis of SCDs through estimation and inference based directly on the effect size measure. A Bayesian procedure is described to analyze the data and draw inferences in SCDs. A multilevel model that is appropriate when several subjects are available is integrated into the Bayesian procedure to provide a standardized effect size measure comparable to effect size measures in a between-subjects design. The applicability of the Bayesian approach for the analysis of SCDs is demonstrated through an example.     

Circular interpretation of regression coefficients

The interpretation of the effect of predictors in projected normal regression models is not straight-forward. The main aim of this paper is to make this interpretation easier such that these models can be employed more readily by social scientific researchers. We introduce three new measures: the slope at the inflection point (b_c), average slope (AS) and slope at mean (SAM) that help us assess the marginal effect of a predictor in a Bayesian projected normal regression model. The SAM or AS are preferably used in situations where the data for a specific predictor do not lie close to the inflection point of a circular regression curve. In this case b_c is an unstable and extrapolated effect. In addition, we outline how the projected normal regression model allows us to distinguish between an effect on the mean and spread of a circular outcome variable. We call these types of effects location and accuracy effects, respectively. The performance of the three new measures and of the methods to distinguish between location and accuracy effects is investigated in a simulation study. We conclude that the new measures and methods to distinguish between accuracy and location effects work well in situations with a clear location effect. In situations where the location effect is not clearly distinguishable from an accuracy effect not all measures work equally well and we recommend the use of the SAM.     

A comparison of methods to combine speed and accuracy measures of performance: A rejoinder on the binning procedure

In cognitive research, speed and accuracy are two important aspects of performance. When analyzed separately, these performance variables sometimes lead to contradictory conclusions about the effect of a manipulation. To avoid such conflicts, several measures that integrate speed and accuracy have been proposed, but the added value of using such measures remains unclear. The present paper compares the relative utility of seven integrated performance measures, namely four variations on a binning procedure that weights response times of correct and incorrect trials differently, and three measures that combine averaged speed and accuracy scores. The properties of these integrated measures were explored in three simulation studies. The first study compared three binning measures and showed that one measure failed to grasp the performance difference between two conditions. The second study showed that the sampling distributions of the measures were symmetric, except for a strong skewness on the rate correct score. The third study varied the trade-off and the effect sizes of speed and accuracy in four different combinations of size and direction of speed and accuracy effects. These studies highlighted some further shortcomings of the binning measures. The combination measures performed well, but linear integration of speed and accuracy and rate correct score were most efficient in detecting effects and accounting for a larger proportion of the variance. The paper concludes that these combination measures are useful provided that the speed and accuracy data are also inspected.     

Models for pedestrian gap acceptance behaviour analysis at unprotected mid-block crosswalks under mixed traffic conditions

Pedestrian safety is an important aspect while crossing the road and it can be explained by pedestrian gap acceptance behaviour. The statistical models such as multiple linear regression (MLR) is often used to model linear relationships between dependent variable (viz., pedestrian gap acceptance behaviour) and independent variables, due to their ability to quantitatively predict the effect of various factors on the dependent variable. However such linear models cannot consider the effect of several variables on the output variable, due to primary assumptions of normality, linear, homoscedasticity and multicollinearity. In this regard, the non-linear models based on the artificial neural network (ANN), which are free from assumptions of linear models, can be easily employed for obtaining the effect of several input variables on the pedestrian accepted gap size. However, researchers have rarely applied ANN modelling technique for predicting the pedestrian gap acceptance behaviour, as the pedestrian gap acceptance behaviour depends on several pedestrian, traffic and vehicular characteristics. The ANN based models would be quite useful in establishing relationship between these factors on the pedestrian gap acceptance behaviour at midblock crosswalks under mixed traffic conditions. In this direction, the present study adopts both MLR as well as ANN with different pedestrian, traffic and vehicular characteristics to assess the significant contributing factors for pedestrians’ gap acceptance behaviour at unprotected mid-block crosswalks under mixed traffic conditions. For this purpose, a video graphic survey was conducted at a six lane divided road at unprotected mid-block crossing in Mumbai, India. The data such as pedestrian (gender and age), vehicular, traffic and pedestrian behavioural characteristics were extracted to model pedestrian accepted gaps. The model results show that pedestrian rolling behaviour has a significant effect on pedestrian accepted gap size. The model results concluded that ANN has a better prediction with possibility to consider the effect of more number of variables on the pedestrian gap acceptance behaviour as compared to the MLR model under mixed traffic conditions. However, the quantification of significant contributing variables on pedestrian accepted gap size is easy by MLR model as compared to the ANN technique. So, both models have their own significant role in pedestrian gap acceptance analysis. The developed models may be useful to enhance the existing mid-block crosswalk facilities or planning new facilities by more accurate prediction of the pedestrian gap acceptance behaviour considering the influence of various factors under mixed traffic conditions.     

Single-case effect size calculation: Comparing regression and non-parametric approaches across previously published reading intervention data sets

Growing from demands for accountability and research-based practice in the field of education, there is recent focus on developing standards for the implementation and analysis of single-case designs. Effect size methods for single-case designs provide a useful way to discuss treatment magnitude in the context of individual intervention. Although a standard effect size methodology does not yet exist within single-case research, panel experts recently recommended pairing regression and non-parametric approaches when analyzing effect size data. This study compared two single-case effect size methods: the regression-based, Allison-MT method and the newer, non-parametric, Tau-U method. Using previously published research that measured the Words read Correct per Minute (WCPM) variable, these two methods were examined by comparing differences in overall effect size scores and rankings of intervention effect. Results indicated that the regression method produced significantly larger effect sizes than the non-parametric method, but the rankings of the effect size scores had a strong, positive relation. Implications of these findings for research and practice are discussed.     

FACTORS AFFECTING INTERPERSONAL CONSTRUCT DIFFERENTIATION WHEN MEASURED USING THE REPERTORY GRID

This article focuses on the notion of Interpersonal Construct Differentiation (ICD) and its measurement, using measures derived from the repertory grid technique. To study the nature of these measures and factors such as grid size and type of scale that could affect them, a set of randomly generated grid data sets are compared to a sample of grids administered to subjects, among which we can distinguish a clinical (n = 422) and a nonclinical (n = 304) sample. Results show that ICD related measures derived from 140 randomly generated grids are clearly distinct from those of subjects. Subjects' grids show lower levels of ICD, polarization, and conflict. Grid size and type of scale exert a powerful mathematical effect on the values of these measures, as illustrated by the sample of random grids, but the grid size effect seems to vanish when applied to the subject sample. However, this mathematical effect should not be ignored in further research.     

Psychology of the scientist: LXIII. Perceiving scattergrams: is visual line fitting related to estimation of the correlation coefficient?

Visual line fitting and direct estimation of the correlation coefficient were carried out by 50 subjects using computer-generated scattergrams as stimuli. In visual line fitting, slopes of visual lines were generally greater than the corresponding regression slopes, in agreement with the hypothesis that visual lines are placed so as to bisect the cloud of displayed points at the cloud's major axis rather than to approximate a regression line. Subjects tended to underestimate the correlation coefficient, scaling their judgments of linear structure somewhat more as if they were judging the coefficient of determination. With the actual degree of linear structure partialed out, there were no strong relationships between measures of visual line fitting and measures of estimation. While both of these tasks offer quickly-obtained correlates of linear structure in scattergrams, users should be aware of their biases. We suggest that visual lines do not approximate regression lines very well and estimates of correlation do not approximate the correlation coefficient very well, because the perceptual processes involved perform operations other than regression and correlation. In the present data, these operations appeared to be independent of each other.     

Statistical guidelines for studies of the theory of reasoned action and the theory of planned behaviour

Abstract

We identify potential problems in the statistical analysis of social cognition model data, with special emphasis on the theories of reasoned action (TRA) and planned behaviour (TPB). Some statistical guidelines are presented for empirical studies of the TRA and the TPB based upon multiple linear regression and structural equation modelling (SEM). If the model is tested using multiple regression, the assumptions of this technique must be considered and variables transformed if necessary. Adjusted R² (not R²) should be used as a measure of explained variance and semipartial correlations are useful in assessing each component's unique contribution to explained variance. R² is not an indicator of model adequacy and residuals should be examined. Expectancy-value variables that are the product of expectancy and value measures represent the interaction term in a multiple regression and should not be used. SEM approaches make explicit the assumptions of unidimensionality of constructs in the TRA/TPB, assumptions that might usefully be challenged by competing models with multidimensional constructs. Finally, statistical power and sample size should be considered for both approaches. Inattention to any of these aspects of analysis threatens the validity of TRA/TPB research.     

Generalized eta and omega squared statistics: measures of effect size for some common research designs

The editorial policies of several prominent educational and psychological journals require that researchers report some measure of effect size along with tests for statistical significance. In analysis of variance contexts, this requirement might be met by using eta squared or omega squared statistics. Current procedures for computing these measures of effect often do not consider the effect that design features of the study have on the size of these statistics. Because research-design features can have a large effect on the estimated proportion of explained variance, the use of partial eta or omega squared can be misleading. The present article provides formulas for computing generalized eta and omega squared statistics, which provide estimates of effect size that are comparable across a variety of research designs.     

AGSuite: Software to conduct feature analysis of artificial grammar learning performance

To simplify the problem of studying how people learn natural language, researchers use the artificial grammar learning (AGL) task. In this task, participants study letter strings constructed according to the rules of an artificial grammar and subsequently attempt to discriminate grammatical from ungrammatical test strings. Although the data from these experiments are usually analyzed by comparing the mean discrimination performance between experimental conditions, this practice discards information about the individual items and participants that could otherwise help uncover the particular features of strings associated with grammaticality judgments. However, feature analysis is tedious to compute, often complicated, and ill-defined in the literature. Moreover, the data violate the assumption of independence underlying standard linear regression models, leading to Type I error inflation. To solve these problems, we present AGSuite, a free Shiny application for researchers studying AGL. The suite’s intuitive Web-based user interface allows researchers to generate strings from a database of published grammars, compute feature measures (e.g., Levenshtein distance) for each letter string, and conduct a feature analysis on the strings using linear mixed effects (LME) analyses. The LME analysis solves the inflation of Type I errors that afflicts more common methods of repeated measures regression analysis. Finally, the software can generate a number of graphical representations of the data to support an accurate interpretation of results. We hope the ease and availability of these tools will encourage researchers to take full advantage of item-level variance in their datasets in the study of AGL. We moreover discuss the broader applicability of the tools for researchers looking to conduct feature analysis in any field.     

Estimating linear effects in ANOVA designs: The easy way

Research in cognitive science has documented numerous phenomena that are approximated by linear relationships. In the domain of numerical cognition, the use of linear regression for estimating linear effects (e.g., distance and SNARC effects) became common following Fias, Brysbaert, Geypens, and d'Ydewalle's (1996) study on the SNARC effect. While their work has become the model for analyzing linear effects in the field, it requires statistical analysis of individual participants and does not provide measures of the proportions of variability accounted for (cf. Lorch & Myers, 1990). In the present methodological note, using both the distance and SNARC effects as examples, we demonstrate how linear effects can be estimated in a simple way within the framework of repeated measures analysis of variance. This method allows for estimating effect sizes in terms of both slope and proportions of variability accounted for. Finally, we show that our method can easily be extended to estimate linear interaction effects, not just linear effects calculated as main effects.     

Maximal reliability and power in covariance structure models

In covariance structure modelling, the non‐centrality parameter of the asymptotic chi‐squared distribution is typically used as an indicator of asymptotic power for hypothesis tests. When a latent linear regression is of interest, the contribution to power by the maximal reliability coefficient, which is associated with used latent variable indicators, is examined and this relationship is further explicated in the case of congeneric measures. It is also shown that item parcelling may reduce power of tests of latent regression parameters. Recommendations on weights for parcelling to avoid power loss are provided, which are found to be those of optimal linear composites with maximal reliability.     

Cautionary Remarks on the Use of Clusterwise Regression

Clusterwise linear regression is a multivariate statistical procedure that attempts to cluster objects with the objective of minimizing the sum of the error sums of squares for the within-cluster regression models. In this article, we show that the minimization of this criterion makes no effort to distinguish the error explained by the within-cluster regression models from the error explained by the clustering process. In some cases, most of the variation in the response variable is explained by clustering the objects, with little additional benefit provided by the within-cluster regression models. Accordingly, there is tremendous potential for overfitting with clusterwise regression, which is demonstrated with numerical examples and simulation experiments. To guard against the misuse of clusterwise regression, we recommend a benchmarking procedure that compares the results for the observed empirical data with those obtained across a set of random permutations of the response measures. We also demonstrate the potential for overfitting via an empirical application related to the prediction of reflective judgment using high school and college performance measures.     

Sacred Canopies or Religious Markets? The Effect of County-Level Religious Diversity on Later Changes in Religious Involvement

Secularization theories, such as Berger's Sacred Canopy argument, hold that religious diversity leads to a decline in religious participation. Religious market models (e.g., Finke and Stark) argue the opposite. Voas, Olson, and Crockett found that nearly all of the vast research exploring this important question prior to 2002 was flawed due to a previously unrecognized noncausal statistical relationship between measures of religious diversity and measures of religious participation. Since 2002, this methodological issue has largely stymied research on this important topic. We first describe how, following Voas et al.’s recommendations, longitudinal models can overcome these problems. We then apply these methods to data measuring the religious composition of all U.S. counties found in the Religious Congregations and Membership Studies from 1980, 1990, 2000, and 2010. Using multilevel longitudinal regression models, we find that greater county-level religious diversity is followed by later declines in county-level religious participation rates. The negative effect size of religious diversity is large and robust to changes in the control variables and different methods of measuring religious diversity.     

Spiritual pathology and religious coping as predictors of forgiveness

Spiritual pathology, religious coping, and dispositional forgiveness were investigated in two studies with graduate students at a Christian university-based seminary. Spiritual pathology was operationalised using measures of spiritual instability and spiritual grandiosity. Study 1 (N?=?194) examined patterns of correlation between positive and negative religious coping items, spiritual pathology, and dispositional forgiveness. Spiritual instability correlated with numerous positive and negative religious coping items while spiritual grandiosity did not. Dispositional forgiveness correlated with more positive than negative religious coping items. Study 2 (N?=?214) tested regression models with these variables. Spiritual grandiosity showed a significant quadratic (concave down) effect in predicting dispositional forgiveness while the linear effect was not significant. A hierarchical regression model showed positive religious coping, spiritual instability, and the quadratic effect for spiritual grandiosity each predicted unique variance in dispositional forgiveness after controlling for spiritual impression management. Negative religious coping was not related to dispositional forgiveness when included with these variables.     

Marital status as a candidate moderator variable of male-female differences in sexual jealousy: the need for representative population samples

Evolutionary psychological theories predict pronounced and universal male-female differences in sexual jealousy. Recent cross-cultural research, using the forced-choice jealousy items pioneered by Buss, et al., 1992, repeatedly found a large sex differential on these self-report measures: men significantly more often than women choose their mate's imagined sexual infidelity to be more distressing or upsetting to them than an imagined emotional infidelity. However, this body of evidence is solely based on undergraduate samples and does not take into account demographic factors. This study examined male-female differences in sexual jealousy in a community sample (N = 335, Eastern Austria). Within a logistic regression model, with other variables controlled for, marital status was a stronger predictor for sexual jealousy than respondents' sex. Contrary to previous research, the sex differential's effect size was only modest. These findings stress the pitfalls of prematurely generalizing evidence from undergraduate samples to the general population and the need for representative population samples in this research area.     

Do Active Labour Market Policies Promote the Subjective Well-Being of the Unemployed? Evidence from the UK National Well-Being Programme

In the past 5 years, the UK government has expanded its efforts to understand, measure and incorporate indicators of subjective well-being (SWB) into the policy-making process. Utilizing the new data collected as part of the government’s well-being agenda, this paper investigates whether active labour market programmes (ALMPs) are associated with increased SWB amongst the unemployed. Unemployment has long been shown to be detrimental to mental health and happiness. In recent years, ALMPs have been increasingly proposed as potential mechanisms to improve the SWB of the unemployed. Using multiple linear regression models, the findings suggest that ALMPs do improve the SWB of the unemployed. However, there are three caveats. First, the effect of ALMPs appears to be far stronger for evaluative measures of SWB over affective measures. Second, the effect of ALMPs is larger for men than for women. Third, the impact of an ALMP is dependent upon the type of intervention: work-oriented ALMPs are more effective than employment-assistance ALMPs. In light of these findings, the theoretical and policy consequences are discussed.     

Naive empiricism and dogmatism in confidence research: a critical examination of the hard-easy effect

Two robust phenomena in research on confidence in one's general knowledge are the overconfidence phenomenon and the hard-easy effect. In this article, the authors propose that the hard-easy effect has been interpreted with insufficient attention to the scale-end effects, the linear dependency, and the regression effects in data and that the continued adherence to the idea of a "cognitive overconfidence bias" is mediated by selective attention to particular data sets. A quantitative review of studies with 2-alternative general knowledge items demonstrates that, contrary to widespread belief, there is (a) very little support for a cognitive-processing bias in these data; (b) a difference between representative and selected item samples that is not reducible to the difference in difficulty; and (c) near elimination of the hard-easy effect when there is control for scale-end effects and linear dependency.