An ordinal coefficient of relational agreement for multiple judges

In a recent article, Fagot proposed a generalized family of coefficients of relational agreement for multiple judges, focusing on the concept of empirically meaningful relationships. In this paper an ordinal coefficient of relational agreement, based on ranking data, is presented as a special case of the generalized family. It is shown that the proposed ordinal coefficient encompasses other ordinal coefficients, such as the Kendall coefficient of concordance, the average Spearman rank-order coefficient, and intraclass correlation based on ranks. It is also shown that the Kendall coefficient of concordance, corrected for chance agreement, is equivalent to the ordinal coefficient proposed in this paper.     

Agreement between Two Independent Groups of Raters

We propose a coefficient of agreement to assess the degree of concordance between two independent groups of raters classifying items on a nominal scale. This coefficient, defined on a population-based model, extends the classical Cohen’s kappa coefficient for quantifying agreement between two raters. Weighted and intraclass versions of the coefficient are also given and their sampling variance is determined by the Jackknife method. The method is illustrated on medical education data which motivated the research.     

On Association Coefficients for 2×2 Tables and Properties That Do Not Depend on the Marginal Distributions

We discuss properties that association coefficients may have in general, e.g., zero value under statistical independence, and we examine coefficients for 2×2 tables with respect to these properties. Furthermore, we study a family of coefficients that are linear transformations of the observed proportion of agreement given the marginal probabilities. This family includes the phi coefficient and Cohen’s kappa. The main result is that the linear transformations that set the value under independence at zero and the maximum value at unity, transform all coefficients in this family into the same underlying coefficient. This coefficient happens to be Loevinger’s H.     

An Alternative Interpretation of the Linearly Weighted Kappa Coefficients for Ordinal Data

When two (or more) observers are independently categorizing a set of observations, Cohen’s kappa has become the most notable measure of interobserver agreement. When the categories are ordinal, a weighted form of kappa becomes desirable. The two most popular weighting schemes are the quadratic weights and linear weights. Quadratic weights have been justified by the fact that the corresponding weighted kappa is asymptotically equivalent to an intraclass correlation coefficient. This paper deals with linear weights and shows that the corresponding weighted kappa is equivalent to the unweighted kappa when cumulative probabilities are substituted for probabilities. A numerical example is provided.     

Recent Trends in the Nordic Doctrine

A review is provided of methods that estimate the magnitude of effects within experimental designs. Intraclass correlation type procedures are appropriate with any ANOVA model. Friedman's r _m and Cohen's power analysis apply to many non-ANOVA type situations, but only to the Fixed Effects ANOVA design. The Gaito utility procedure has two advantages over other intraclass correlation type measures: the coefficients sum to unity (or to 100%), and coefficients for error components are obtained.     

Sample size determinations for the two rater kappa statistic

This paper gives a method for determining a sample size that will achieve a prespecified bound on confidence interval width for the interrater agreement measure,. The same results can be used when a prespecified power is desired for testing hypotheses about the value of kappa. An example from the literature is used to illustrate the methods proposed here.     

Monotonic measures of agreement for ranked data

The τ_b and y statistics are interpreted as rank-monotonic coefficients of partial agreement. Using a method of transposition employed by Pearson's r_i intraclass correlation coefficient, the τ_bi and y_i intraclass coefficients of total monotonic agreement are created. Transpositional measures of agreement like τ_bi and τ_i measure the combined effects of cell and marginal disagreement which make them particularly suitable for reliability studies. The coefficients are also made applicable to K > 2 sets of ranks.     

An implicit enumeration method for an exact test of weighted kappa

The kappa coefficient is one of the most widely used measures for evaluating the agreement between two raters asked to assign N objects to one of K nominal categories. Weighted versions of kappa enable partial credit to be awarded for near agreement, most notably in the case of ordinal categories. An exact significance test for weighted kappa can be conducted by enumerating all rater agreement tables with the same fixed marginal frequencies as the observed table, and accumulating the probabilities for all tables that produce a weighted kappa index that is greater than or equal to the observed measure. Unfortunately, complete enumeration of all tables is computationally unwieldy for modest values of N and K. We present an implicit enumeration algorithm for conducting an exact test of weighted kappa, which can be applied to tables of non‐trivial size. The algorithm is particularly efficient for ‘good’ to ‘excellent’ values of weighted kappa that typically have very small p‐values. Therefore, our method is beneficial for situations where resampling tests are of limited value because the number of trials needed to estimate the p‐value tends to be large.     

Some Paradoxical Results for the Quadratically Weighted Kappa

The quadratically weighted kappa is the most commonly used weighted kappa statistic for summarizing interrater agreement on an ordinal scale. The paper presents several properties of the quadratically weighted kappa that are paradoxical. For agreement tables with an odd number of categories n it is shown that if one of the raters uses the same base rates for categories 1 and n, categories 2 and n−1, and so on, then the value of quadratically weighted kappa does not depend on the value of the center cell of the agreement table. Since the center cell reflects the exact agreement of the two raters on the middle category, this result questions the applicability of the quadratically weighted kappa to agreement studies. If one wants to report a single index of agreement for an ordinal scale, it is recommended that the linearly weighted kappa instead of the quadratically weighted kappa is used.     

What structural mythology owes to Henri Hubert

In pointing to Durkheimian precedents for structuralism, Claude Lévi-Strauss typically indicates Marcel Mauss. Yet, although Mauss wrote much on myth, Lévi-Strauss never cites Mauss as setting precedents for structural mythology. This seems so for at least two reasons. First, Henri Hubert, not Mauss, turns out to be the real myth specialist of Emile Durkheim's original équipe, thus making the équipe's theory primarily Hubert's. Second, Hubert's theory of myth is only problematically structural. More consistent with theories at odds with structuralism, especially Maurice Leenhardt's religious phenomenology, Durkheimian mythology must be displaced if structural mythology is to distinguish itself.     

Comparing dependent robust correlations

Let r₁ and r₂ be two dependent estimates of Pearson's correlation. There is a substantial literature on testing H₀ : ρ₁ = ρ₂, the hypothesis that the population correlation coefficients are equal. However, it is well known that Pearson's correlation is not robust. Even a single outlier can have a substantial impact on Pearson's correlation, resulting in a misleading understanding about the strength of the association among the bulk of the points. A way of mitigating this concern is to use a correlation coefficient that guards against outliers, many of which have been proposed. But apparently there are no results on how to compare dependent robust correlation coefficients when there is heteroscedasicity. Extant results suggest that a basic percentile bootstrap will perform reasonably well. This paper reports simulation results indicating the extent to which this is true when using Spearman's rho, a Winsorized correlation or a skipped correlation.     

Frontal deficits differentiate progressive supranuclear palsy from Parkinson's disease

The clinical differentiation of progressive supranuclear palsy from Parkinson's disease can be challenging, due to overlapping clinical features and a lack of diagnostic markers. Abnormalities in cognitive function form part of the clinical spectrums of these diseases and distinctive cognitive profiles may be helpful in differentiating these diseases in the diagnostic period. A comprehensive neuropsychological test battery was administered to 12 patients with clinically diagnosed progressive supranuclear palsy and 12 patients with Parkinson's disease matched for age and disease duration. Effect size (Cohen's d) was calculated for cognitive tests that were significantly different between groups. Patients with progressive supranuclear palsy performed significantly worse than those with Parkinson's disease on measures of processing speed, verbal fluency, planning, verbal abstract reasoning, verbal memory, and made more perseverative responses on a set shifting task. Measures of executive function, manual dexterity and processing speed were most diagnostically useful (Cohen's d > 2.0) in differentiating between progressive supranuclear palsy and Parkinson's disease. These findings suggest that more severe and prominent ‘frontal’ cognitive deficits in patients with progressive parkinsonism would be helpful in predicting progressive supranuclear palsy rather than Parkinson's disease and these findings may contribute to the development of diagnostic criteria.     

Hume's Internalist Epistemology in EHU 12

Much has been written about Kemp Smith's (1941) famous problem regarding the tension between Hume's naturalism and his scepticism. However, most commentators have focused their attention on the Treatise; those who address the Enquiry often take it to express essentially the same message as the Treatise. When Hume's scepticism in the Enquiry has been investigated in its own right, commentators have tended to focus on Hume's inductive scepticism in Sections 4 and 5. All in all, it seems that Section 12 has been unduly neglected. This paper seeks to address Kemp Smith's problem from the standpoint of Hume's treatment of scepticism in EHU 12, and finds an interesting internalist account that makes sense both of Hume's discussion in EHU 12, and his aims in the Enquiry as a whole. Moreover, it is one that is of substantive philosophical interest, having intriguing parallels to contemporary epistemological accounts.     

WHY RELIGION MATTERS AND THE PURPOSES OF HIGHER EDUCATION: A DIALOGUE WITH HUSTON SMITH

This article examines Huston Smith's critique of and remedy for modernity from the perspective of a college professor who adopted “Why Religion Matters” (2001) as required reading for undergraduates. Smith's heartfelt plea to consider, if not embrace, the common wisdom of traditional religious worldviews deserves a hearing. But Smith's approach is also in need of qualification, supplementation, and critique. This article, ironically, finds the needed qualification, supplementation, and critique in Huston Smith's much earlier publication, The Purposes of Higher Education (1955). This article provides the dialogue.     

Test–retest reliability of the Brunel Lifestyle Physical Activity Questionnaire

ObjectivesTo establish the test–retest reliability of planned physical activity (PPA) and unplanned physical activity (UPA) components of the Brunel Lifestyle Physical Activity Questionnaire (BLPAQ). To provide evidence of the BLPAQ's stability using the proportion of agreement (PoA) method over a 5-week period.DesignTest–retest over a 5-week period using three diverse samples of adults.MethodsThe 277 participants were subdivided into three adult samples: gymnasium users (n = 80), undergraduate students (n = 111), and university staff members (n = 86). They were asked to complete the test–retest measure in their places of exercise, study, or work respectively.ResultsCorrelation coefficients between test–retest administrations were calculated for each participant group and intraclass correlations were calculated for each item. Pearson's product-moment correlations ranged from r = 0.95 to r = 0.96 for the PPA subscale and r = 0.93 to r = 0.98 for the UPA subscale. Intraclass correlations ranged from R = 0.52 to R = 0.99 for PPA and R = 0.87 to R = 0.99 for UPA. Fisher's z tests indicated that the test–retest correlation coefficients for the BLPAQ subscales were, on the whole, significantly stronger than those of older, comparable subscales from lifestyle physical activity questionnaires. The PoA analysis for each item revealed that the test–retest administrations were in high agreement (>95%).ConclusionsOverall, the PPA and UPA factors of the BLPAQ demonstrated high reliability and stability. The present study also illustrates the utility of PoA analysis in establishing the stability of physical activity measures.     

Detecting Clusters/Communities in Social Networks

Cohen's κ, a similarity measure for categorical data, has since been applied to problems in the data mining field such as cluster analysis and network link prediction. In this paper, a new application is examined: community detection in networks. A new algorithm is proposed that uses Cohen's κ as a similarity measure for each pair of nodes; subsequently, the κ values are then clustered to detect the communities. This paper defines and tests this method on a variety of simulated and real networks. The results are compared with those from eight other community detection algorithms. Results show this new algorithm is consistently among the top performers in classifying data points both on simulated and real networks. Additionally, this is one of the broadest comparative simulations for comparing community detection algorithms to date.     

Heterogeneity Coefficients for Mahalanobis' D as a Multivariate Effect Size

The Mahalanobis distance D is the multivariate generalization of Cohen's d and can be used as a standardized effect size for multivariate differences between groups. An important issue in the interpretation of D is heterogeneity, that is, the extent to which contributions to the overall effect size are concentrated in a small subset of variables rather than evenly distributed across the whole set. Here I present two heterogeneity coefficients for D based on the Gini coefficient, a well-known index of inequality among values of a distribution. I discuss the properties and limitations of the two coefficients and illustrate their use by reanalyzing some published findings from studies of gender differences.     

On Similarity Coefficients for 2×2 Tables and Correction for Chance

This paper studies correction for chance in coefficients that are linear functions of the observed proportion of agreement. The paper unifies and extends various results on correction for chance in the literature. A specific class of coefficients is used to illustrate the results derived in this paper. Coefficients in this class, e.g. the simple matching coefficient and the Dice/Sørenson coefficient, become equivalent after correction for chance, irrespective of what expectation is used. The coefficients become either Cohen’s kappa, Scott’s pi, Mak’s rho, Goodman and Kruskal’s lambda, or Hamann’s eta, depending on what expectation is considered appropriate. Both a multicategorical generalization and a multivariate generalization are discussed.     

A new rank correlation coefficient with application to the consensus ranking problem

The consensus ranking problem has received much attention in the statistical literature. Given m rankings of n objects the objective is to determine a consensus ranking. The input rankings may contain ties, be incomplete, and may be weighted. Two solution concepts are discussed, the first maximizing the average weighted rank correlation of the solution ranking with the input rankings and the second minimizing the average weighted Kemeny–Snell distance. A new rank correlation coefficient called τ_x is presented which is shown to be the unique rank correlation coefficient which is equivalent to the Kemeny‐Snell distance metric. The new rank correlation coefficient is closely related to Kendall's tau but differs from it in the way ties are handled. It will be demonstrated that Kendall's τ_b is flawed as a measure of agreement between weak orderings and should no longer be used as a rank correlation coefficient. The use of τ_x in the consensus ranking problem provides a more mathematically tractable solution than the Kemeny–Snell distance metric because all the ranking information can be summarized in a single matrix. The methods described in this paper allow analysts to accommodate the fully general consensus ranking problem with weights, ties, and partial inputs. Copyright © 2002 John Wiley & Sons, Ltd.     

Adam Smith,Ethicist

This essay argues that Adam Smith's political economy is premised upon a moral anthropology, and that greater attention to Smith from religious ethicists may both improve Smith scholarship and deepen dialogue on economic themes within the field of religious ethics. It does so first by surveying common readings of Smith and noting that engagement of his work within religious ethics and theology tends to rely on misconceptions prevalent in these readings. It then outlines the moral psychology that links Smith's Theory of Moral Sentiments and Wealth of Nations and explains the importance of this moral psychology for Smith's ambivalent analysis of commercial society. Reflecting on the case of Smith's work, it concludes by arguing that attention from religious ethicists may also improve contemporary political economic debates, given that they are often premised upon latent assumptions about moral anthropology.