An asymptotic partial correlation test for the Goodman‐Kruskal λ

Goodman and Kruskal introduced a measure of predictive association when predicting the category of a variable A from a category of a variable B. This measure, denoted λ, is the asymmetric proportional reduction in error measure in predicting an individual's A category that can be eliminated by using knowledge of the B classification. It takes values on the unit interval, with a zero value meaning no predictive gain, while a value of unity indicates a perfect predictive association between A and B. A test of H₀: λ = 0 versus H₁: λ > 0 is analogous to a test for the significance of the correlation coefficient. A test of the partial λ coefficient, which is analogous to a test of the partial correlation coefficient, answers the question of whether knowledge of an additional third (or higher) classification or categorical variable results in a significant increase in predicting the variable A. Suich and Turek developed an exact test for the partial λ coefficient, but only for the situation where the predicted categorical variable A is dichotomous. The present paper completes the previous work by developing an asymptotic test where the predicted category A is any polytomous variable.     

Measures of feeling-of-knowing accuracy: A new look at an old problem

Two ordinal measures of feeling-of-knowing performance appropriate for n × n data arrays are reviewed. Goodman and Kruskal's gamma provides a measure of association between recognition performance and feeling-of-knowing judgements. The Hamann coefficient provides a measure of agreement accuracy. The relative strengths and weaknesses of each measure are compared at length. A proof is provided, which reveals a lack of one-to-one relation between the two and suggests that they may be independent under most circumstances. It is concluded that both measures should be reported together as complementary indices as each captures a different facet of feeling-of-knowing performance. Alternative measures for gamma and the Hamann coefficient are considered and a number of recommendations are made for future research.     

Maximum-corrected and chance-corrected measures of effect size for the Mantel-Haenszel test

Two measures of effect size are described for the Mantel-Haenszel test. Both measures belong to the r-family of effect size measures. One measure is based on a maximum-corrected model, and the second measure is based on a chance-corrected model.     

Similarity and contrast in memory for relations

The paper reports four experiments concerned with memory for sets of categorically related or unrelated nouns, as a function of the extent to which initial processing focused on categorical or distinctive features of the nouns. The general finding is that distinctive features are valuable for later intracategory discrimination in recall and recognition, but categorical features are not, as judged by cued recall, recognition, measures of association between items, and categorical intrusion errors. A theoretical framework, based on the diagnostic value of features for various classifications, is offered as an account of the results.     

An effect size measure and Bayesian analysis of single-case designs

This article describes a linear modeling approach for the analysis of single-case designs (SCDs). Effect size measures in SCDs have been defined and studied for the situation where there is a level change without a time trend. However, when there are level and trend changes, effect size measures are either defined in terms of changes in R² or defined separately for changes in slopes and intercept coefficients. We propose an alternate effect size measure that takes into account changes in slopes and intercepts in the presence of serial dependence and provides an integrated procedure for the analysis of SCDs through estimation and inference based directly on the effect size measure. A Bayesian procedure is described to analyze the data and draw inferences in SCDs. A multilevel model that is appropriate when several subjects are available is integrated into the Bayesian procedure to provide a standardized effect size measure comparable to effect size measures in a between-subjects design. The applicability of the Bayesian approach for the analysis of SCDs is demonstrated through an example.     

Effect size and power in assessing moderating effects of categorical variables using multiple regression: a 30-year review

The authors conducted a 30-year review (1969-1998) of the size of moderating effects of categorical variables as assessed using multiple regression. The median observed effect size (f(2)) is only .002, but 72% of the moderator tests reviewed had power of .80 or greater to detect a targeted effect conventionally defined as small. Results suggest the need to minimize the influence of artifacts that produce a downward bias in the observed effect size and put into question the use of conventional definitions of moderating effect sizes. As long as an effect has a meaningful impact, the authors advise researchers to conduct a power analysis and plan future research designs on the basis of smaller and more realistic targeted effect sizes.     

Measures of effect size

Two different approaches have been used to derive measures of effect size. One approach is based on the comparison of treatment means. The standardized mean difference is an appropriate measure of effect size when one is merely comparing two treatments, but there is no satisfactory analogue for comparing more than two treatments. The second approach is based on the proportion of variance in the dependent variable that is explained by the independent variable. Estimates have been proposed for both fixed-factor and random-factor designs, but their sampling properties are not well understood. Nevertheless, measures of effect size can allow quantitative comparisons to be made across different studies, and they can be a useful adjunct to more traditional outcome measures such as test statistics and significance levels.     

Measuring Feeling of Knowing: Comment on Schraw (1995)

Schraw compares two coefficients for the 2×2 contingency tables resulting from many feeling of knowing (FOK) studies: Hamann's coefficient and Goodman and Kruskal's γ. He favours Hamann's coefficient and gives examples where Hamann's coefficient produces what might be considered the more intuitive result. Further scrutiny reveals that these examples are not as convincing as Schraw makes them out to be. Because Hamann's coefficient depends on the row and column marginals, it does not map onto FOK ability as well as Goodman and Kruskal's γ, which is a direct measure of the diagnostic worth of FOK ratings.     

How to separate coordinate and categorical spatial relation components in integrated spatial representations: A new methodology for analysing sketch maps

Spatial relations between landmarks can be represented by means of categories and coordinates. In the present research, this paradigm was applied to sketch maps based on information acquired in goal-directed behaviour of exploration of a university campus area. The first aim was to investigate whether categorical and coordinate information can be considered conceptually independent in sketch maps. The second aim was to assess which kind of distance measure served better to represent coordinate information in the present case study, and finally to assess the factorial structure of coordinate and categorical data. Analytic methodology as well as statistical analysis were found to confirm that separating coordinate and categorical components was formally as well as empirically appropriate. A series of confirmatory factor analyses showed the best fit for the model with two correlated components, as well as an acceptable reliability of measures emerged. The two components were moderately correlated. Moreover, the adoption of Manhattan distance seemed to be the most effective method to represent coordinate spatial relations in spatial sketch maps of areas acquired through navigation.     

Quantifying construct validity: two simple measures

Construct validity is one of the most central concepts in psychology. Researchers generally establish the construct validity of a measure by correlating it with a number of other measures and arguing from the pattern of correlations that the measure is associated with these variables in theoretically predictable ways. This article presents 2 simple metrics for quantifying construct validity that provide effect size estimates indicating the extent to which the observed pattem of correlations in a convergent-discriminant validity matrix matches the theoretically predicted pattern of correlations. Both measures, based on contrast analysis, provide simple estimates of validity that can be compared across studies, constructs, and measures meta-analytically, and can be implemented without the use of complex statistical procedures that may limit their accessibility.     

Differentiability of Kruskal's stress at a local minimum

It is shown that Kruskal's multidimensional scaling loss function is differentiable at a local minimum. Or, to put it differently, that in multidimensional scaling solutions using Kruskal's stress distinct points cannot coincide.     

从效应量应有的性质看中介效应量的合理性

效应量的作用有两个方面, 一是弥补了统计检验的不足, 二是使得效应有可比性。结合统计显著性和效应量, 才能得出适当的统计结论。效应量应当具有一些基本性质, 包括与测量单位无关、单调性、不受样本容量的影响。国际上流行的中介效应量κ平方就是因为缺乏单调性而引发质疑和研究, 从而被彻底终结了其作为中介效应量的合法性。R平方型中介效应量同样有缺乏单调性的问题。文末讨论了如何报告中介效应量以及有待研究的问题。     

Setting things straight: A comparison of measures of saccade trajectory deviation

In eye movements, saccade trajectory deviation has often been used as a physiological operationalization of visual attention, distraction, or the visual system’s prioritization of different sources of information. However, there are many ways to measure saccade trajectories and to quantify their deviation. This may lead to noncomparable results and poses the problem of choosing a method that will maximize statistical power. Using data from existing studies and from our own experiments, we used principal components analysis to carry out a systematic quantification of the relationships among eight different measures of saccade trajectory deviation and their power to detect the effects of experimental manipulations, as measured by standardized effect size. We concluded that (1) the saccade deviation measure is a good default measure of saccade trajectory deviation, because it is somewhat correlated with all other measures and shows relatively high effect sizes for two well-known experimental effects; (2) more generally, measures made relative to the position of the saccade target are more powerful; and (3) measures of deviation based on the early part of the saccade are made more stable when they are based on data from an eyetracker with a high sampling rate. Our recommendations may be of use to future eye movement researchers seeking to optimize the designs of their studies.     

Correlation and explaining variance: To square or not to square?

Despite previous articles dating back 80 years, the questions of whether and when to square correlations continue to puzzle and confuse researchers. In this editorial, I point out that correlations can serve two independent purposes: they can be measures of effect size in themselves and their function as regression coefficients can be used to estimate proportion of variance in one measure for which another measure accounts. Using examples relevant to intelligence researchers, I show that the answer to the question of whether or not to square a correlation is ‘it depends.’ It depends on the purpose and it depends on the underlying theoretical model of the causal association between the variables.     

Effect size,statistical power,and sample size for assessing interactions between categorical and continuous variables

The reporting and interpretation of effect size estimates are widely advocated in many academic journals of psychology and related disciplines. However, such concern has not been adequately addressed for analyses involving interactions between categorical and continuous variables. For the purpose of improving current practice, this article presents fundamental features and theoretical developments for the variance of standardized slopes as a desirable standardized effect size measure for the degree of disparity between several slope coefficients. To estimate the effect size, a consistent and nearly unbiased estimator is described and a simple refinement is emphasized for extreme situations whenever appropriate. The essential problems of power and sample size calculations for testing the equality of slope coefficients are also considered. According to the analytic justification and empirical assessment, the exact approach has a clear advantage over the approximate methods. Both SAS and R computer codes are provided to facilitate practical accessibility of the proposed techniques in interaction studies.     

A new measure for an important construct: the attachment and object relations inventory

This article describes the development of the Attachment and Object Relations Inventory (AORI). The AORI, a dimensional measure based on factor structure, was developed in response to the psychometric problems typical to attachment style measures that are categorical in nature. Test-retest reliability and internal consistency were high. A factor analysis of the 75 AORI items confirmed the existence of at least 6 factors and explained approximately 50% of the item variance. Convergent and discriminant validity of the AORI was supported through appropriate relationships with measures of depression, anxiety, and personality, and other measures of attachment and object relations.     

A New Measure for an Important Construct: The Attachment and Object Relations Inventory

This article describes the development of the Attachment and Object Relations Inventory (AORI). The AORI, a dimensional measure based on factor structure, was developed in response to the psychometric problems typical to attachment style measures that are categorical in nature. Test-retest reliability and internal consistency were high. A factor analysis of the 75 AORI items confirmed the existence of at least 6 factors and explained approximately 50% of the item variance. Convergent and discriminant validity of the AORI was supported through appropriate relationships with measures of depression, anxiety, and personality, and other measures of attachment and object relations.     

Categorical invariance and structural complexity in human concept learning

An alternative account of human concept learning based on an invariance measure of the categorical stimulus is proposed. The categorical invariance model (CIM) characterizes the degree of structural complexity of a Boolean category as a function of its inherent degree of invariance and its cardinality or size. To do this we introduce a mathematical framework based on the notion of a Boolean differential operator on Boolean categories that generates the degrees of invariance (i.e., logical manifold) of the category in respect to its dimensions. Using this framework, we propose that the structural complexity of a Boolean category is indirectly proportional to its degree of categorical invariance and directly proportional to its cardinality or size. Consequently, complexity and invariance notions are formally unified to account for concept learning difficulty. Beyond developing the above unifying mathematical framework, the CIM is significant in that: (1) it precisely predicts the key learning difficulty ordering of the SHJ [Shepard, R. N., Hovland, C. L., & Jenkins, H. M. (1961). Learning and memorization of classifications. Psychological Monographs: General and Applied, 75(13), 1-42] Boolean category types consisting of three binary dimensions and four positive examples; (2) it is, in general, a good quantitative predictor of the degree of learning difficulty of a large class of categories (in particular, the 41 category types studied by Feldman [Feldman, J. (2000). Minimization of Boolean complexity in human concept learning. Nature, 407, 630-633]); (3) it is, in general, a good quantitative predictor of parity effects for this large class of categories; (4) it does all of the above without free parameters; and (5) it is cognitively plausible (e.g., cognitively tractable).     

Holistic processing in the composite task depends on face size

Holistic processing is a hallmark of face processing. There is evidence that holistic processing is strongest for faces at identification distance 2–10 metres from the observer. However, this evidence is based on tasks that have been little used in the literature and that are indirect measures of holistic processing. We use the composite task—a well validated and frequently used paradigm—to measure the effect of viewing distance on holistic processing. In line with previous work, we find a congruency x alignment effect that is strongest for faces that are close (2 m equivalent distance) than for faces that are further away (24 m equivalent distance). In contrast, the alignment effect for same trials, used by several authors to measure holistic processing, produced results that are difficult to interpret. We conclude that our results converge with previous findings providing more direct evidence for an effect of size on holistic processing.     

Reliability and expected loss: A unifying principle

We provide a unified, theoretical basis on which measures of data reliability may be derived or evaluated, for both quantitative and qualitative data. This approach evaluates reliability as the proportional reduction in loss (PRL) that is attained in a sample by an optimal estimator. The resulting measure is between 0 and 1, linearly related to expected loss, and provides a direct way of contrasting the measured reliability in the sample with the least reliable and most reliable data-generating cases. The PRL measure is a generalization of many of the commonly-used reliability measures.We show how the quantitative measures from generalizability theory can be derived as PRL measures (including Cronbach's alpha and measures proposed by Winer). For categorical data, we develop a new measure for the general case in which each of N judges assigns a subject to one of K categories and show that it is equivalent to a measure proposed by Perreault and Leigh for the case where N is 2.Bruce Cooil is an Associate Professor of Statistics, and Roland T. Rust is a Professor and area head for Marketing. The authors thank three anonymous reviewers and an Associate Editor for their helpful comments and suggestions. This work was supported in part by the Dean's Fund for Faculty Research of the Owen Graduate School of Management, Vanderbilt University.