Functional anatomy of the null hypothesis and of tests of it

The author compared simulations of the "true" null hypothesis (zeta) test, in which sigma was known and fixed, with the t test, in which s, an estimate of sigma, was calculated from the sample because the t test was used to emulate the "true" test. The true null hypothesis test bears exclusively on calculating the probability that a sample distance (mean) is larger than a specified value. The results showed that the value of t was sensitive to sampling fluctuations in both distance and standard error. Large values of t reflect small standard errors when n is small. The value of t achieves sensitivity primarily to distance only when the sample sizes are large. One cannot make a definitive statement about the probability or "significance" of a distance solely on the basis of the value of t.     

Are Effect Sizes and Confidence Levels Problems For or Solutions To the Null Hypothesis Test?

Some have proposed that the null hypothesis significance test, as usually conducted using the t test of the difference between means, is an impediment to progress in psychology. To improve its prospects, using Neyman-Pearson confidence intervals and Cohen's standardized effect sizes, d, is recommended. The purpose of these approaches is to enable us to understand what can appropriately be said about the distances between the means and their reliability. Others have written extensively that these recommended strategies are highly interrelated and use identical information. This essay was written to remind us that the t test, based on the sample—not the true—standard deviation, does not apply solely to distance between means. The t test pertains to a much more ambiguous specification: the difference between samples, including sampling variations of the standard deviation.     

Some Thoughts Regarding the Sensitivity of t to Error in Estimated Within-Group Variance

A. J. Riopelle (2003) has eloquently demonstrated that the null hypothesis assessed by the t test involves not only mean differences but also error in the estimation of the within-group standard deviation, s. He is correct in his conclusion that the precision of the interpretation of a significant t and the null hypothesis tested is complex, particularly when sample sizes are small. In this article, the author expands on Riopelle's thoughts by comparing t with some equivalent or closely related tests that make the reliance of t on the accurate estimation of error perhaps more salient and by providing a simulation that may address more directly the magnitude of the interpretational problem.     

The variable-criteria sequential stopping rule: Generality to unequal sample sizes,unequal variances,or to large ANOVAs

The variable-criteria sequential stopping rule (SSR) is a method for conducting planned experiments in stages after the addition of new subjects until the experiment is stopped because the p value is less than or equal to a lower criterion and the null hypothesis has been rejected, the p value is above an upper criterion, or a maximum sample size has been reached. Alpha is controlled at the expected level. The table of stopping criteria has been validated for a t test or ANOVA with four groups. New simulations in this article demonstrate that the SSR can be used with unequal sample sizes or heterogeneous variances in a t test. As with the usual t test, the use of a separate-variance term instead of a pooled-variance term prevents an inflation of alpha with heterogeneous variances. Simulations validate the original table of criteria for up to 20 groups without a drift of alpha. When used with a multigroup ANOVA, a planned contrast can be substituted for the global F as the focus for the stopping rule. The SSR is recommended when significance tests are appropriate and when the null hypothesis can be tested in stages. Because of its efficiency, the SSR should be used instead of the usual approach to the t test or ANOVA when subjects are expensive, rare, or limited by ethical considerations such as pain or distress.     

Betting on conditionals

A study is reported testing two hypotheses about a close parallel relation between indicative conditionals, if A then B, and conditional bets, I bet you that if A then B. The first is that both the indicative conditional and the conditional bet are related to the conditional probability, P(B|A). The second is that de Finetti's three-valued truth table has psychological reality for both types of conditional—true, false, or void for indicative conditionals and win, lose, or void for conditional bets. The participants were presented with an array of chips in two different colours and two different shapes, and an indicative conditional or a conditional bet about a random chip. They had to make judgements in two conditions: either about the chances of making the indicative conditional true or false or about the chances of winning or losing the conditional bet. The observed distributions of responses in the two conditions were generally related to the conditional probability, supporting the first hypothesis. In addition, a majority of participants in further conditions chose the third option, “void”, when the antecedent of the conditional was false, supporting the second hypothesis.     

Loneliness and social monitoring: A conceptual replication of Knowles et al.

Contrary to the claim that loneliness routinely impairs the decoding of social cues such as emotion displays, Knowles, Lucas, Baumeister, and Gardner (2015) proposed that lonely adults “choke under pressure,” experiencing impairments only when social monitoring is framed as diagnostic of general social skill. In four experiments, Knowles et al. showed that lonely individuals performed worse than nonlonely individuals at decoding social cues when the decoding task was framed as a test of social aptitude, but not when it was framed as a test of academic aptitude. The studies were small (N's ranging from 78 to 203), and all employed a convenience sample of mostly female undergraduate students, impairing both statistical power and external validity. In addition, the lack of a true control group precluded the studies from establishing whether loneliness inhibits social monitoring ability if no frame is offered. This study conceptually replicates the central hypothesis of Knowles et al. using a sample of adults that is substantially larger and more diverse demographically and geographically, and using a true control group in addition to the comparison group. Results revealed a significant main effect of loneliness on social monitoring ability but did not replicate the choking under pressure phenomenon.     

The MBR metric

P. H. Schönemann's (1982, Bulletin of the Psychonomic Society, 19, 317–319; 1983, Journal of Mathematical Psychology, 27, 311–324) MBR metric is designed to account for distortion effects in distance estimates caused by upper bounds in the response scales. To test the MBR hypothesis, given data have to be transformed first into the half-open interval [0, 1). This is achieved by defining the greatest distance estimate as the upper bound, and then dividing all values by this bound plus some “small constant” e, which ensures that the interval is open. How e should be picked is left open. It is shown here that very small differences in the value chosen for e have massive effects on the fit of the MBR model. Moreover, if e → 0, then the fit will eventually get bad for all realistic data.     

A REASSESSMENT OF STATISTICAL POWER ANALYSIS IN HUMAN COMMUNICATION RESEARCH

This study conducted a statistical power analysis of 64 articles appearing in the first four volumes of Human Communication Research, 1974–1978. Each article was examined, using Cohen's revised handbook, assuming nondirectional null hypotheses and an alpha level of .05. Statistical power, the probability of rejecting a false null hypothesis, was calculated for small, medium, and large experimental effect sizes and averaged by article and volume. Results indicated that the average probability of beta errors appears to have decreased over time, providing a greater chance of rejecting false null hypotheses, but this also raised several power-related issues relevant to communication research in general.     

A note on consistency of non-parametric rank tests and related rank transformations

The extent to which rank transformations result in the same statistical decisions as their non‐parametric counterparts is investigated. Simulations are presented using the Wilcoxon–Mann–Whitney test, the Wilcoxon signed‐rank test and the Kruskal–Wallis test, together with the rank transformations and t and F tests corresponding to each of those non‐parametric methods. In addition to Type I errors and power over all simulations, the study examines the consistency of the outcomes of the two methods on each individual sample. The results show how acceptance or rejection of the null hypothesis and differences in p‐values of the test statistics depend in a regular and predictable way on sample size, significance level, and differences between means, for normal and various non‐normal distributions.     

False memories in Lewy‐body disease

Recently, de Boysson, Belleville, Phillips et al. (2011) found that patients with Lewy‐body disease (LBD) showed significantly lower rates of false memories than healthy controls, using the Deese–Roediger–McDermott (DRM) experimental procedure. Given that this result could be explained by the practically null rate of true recognition in the LBD group (0.09), we decided to replicate the study by de Boysson et al. (2011), but including a new condition that would maximize the true recognition rate (and analyze its effect on the rate of false memories). Specifically, in a DRM experiment, we manipulated (within subjects) two study and recognition conditions: in the “immediate” condition, both the LBD patients and the control group of healthy older people received a different recognition test after each study list (containing twelve words associated with a non‐presented critical word), while in the “delayed” condition (similar to the one in de Boysson et al., 2011), the participants received the entire series of study lists and then took only one recognition test. The results showed that, in both samples, the “immediate” condition produced higher corrected rates of both true and false recognition than the “delayed” condition, although they were both lower in the LBD patients, which shows that these patients are capable of encoding and recognizing the general similitude underlying information (gist memory) in the right conditions.     

Comparing the meanings of “if” and “all”

In this study, we compared the everyday meanings of conditionals (“if p then q”) and universally quantified statements (“all p are q”) when applied to sets of elements. The interpretation of conditionals was predicted to be directly related to the conditional probability, such that P(“if p then q”) = P(q|p). Quantified statements were assumed to have two interpretations. According to an instance-focused interpretation, quantified statements are equivalent to conditionals, such that P(“all p are q”) = P(q|p). According to a set-focused interpretation, “all p are q” is true if and only if every instance in set p is an instance of q, so that the statement would be accepted when P(q|p) = 1 and rejected when this probability was below 1. We predicted an instance-focused interpretation of “all” when the relation between p and q expressed a general law for an infinite set of elements. A set-focused interpretation of “all” was predicted when the relation between p and q expressed a coincidence among the elements of a finite set. Participants were given short context stories providing information about the frequency of co-occurrence of cases of p, q, not-p, and not-q in a population. They were then asked to estimate the probability that a statement (conditional or quantified) would be true for a random sample taken from that population. The probability estimates for conditionals were in accordance with an instance-focused interpretation, whereas the estimates for quantified statements showed features of a set-focused interpretation. The type of the relation between p and q had no effect on this outcome.     

Even statisticians are not immune to misinterpretations of Null Hypothesis Significance Tests

We investigated the way experienced users interpret Null Hypothesis Significance Testing (NHST) outcomes. An empirical study was designed to compare the reactions of two populations of NHST users, psychological researchers and professional applied statisticians, when faced with contradictory situations. The subjects were presented with the results of an experiment designed to test the efficacy of a drug by comparing two groups (treatment/placebo). Four situations were constructed by combining the outcome of the t test (significant vs. nonsignificant) and the observed difference between the two means D (large vs. small). Two of these situations appeared as conflicting (t significant/D small and t nonsignificant/D large). Three fundamental aspects of statistical inference were investigated by means of open questions: drawing inductive conclusions about the magnitude of the true difference from the data in hand, making predictions for future data, and making decisions about stopping the experiment. The subjects were 25 statisticians from pharmaceutical companies in France, subjects well versed in statistics, and 20 psychological researchers from various laboratories in France, all with experience in processing and analyzing experimental data. On the whole, statisticians and psychologists reacted in a similar way and were very impressed by significant results. It must be outlined that professional applied statisticians were not immune to misinterpretations, especially in the case of nonsignificance. However, the interpretations that accustomed users attach to the outcome of NHST can vary from one individual to another, and it is hard to conceive that there could be a consensus in the face of seemingly conflicting situations. In fact, beyond the superficial report of “erroneous” interpretations, it can be seen in the misuses of NHST intuitive judgmental “adjustments” that try to overcome its inherent shortcomings. These findings encourage the many recent attempts to improve the habitual ways of analyzing and reporting experimental data.     

The ontology of words: a structural approach

Words form a fundamental basis for our understanding of linguistic practice. However, the precise ontology of words has eluded many philosophers and linguists. A persistent difficulty for most accounts of words is the type-token distinction [Bromberger, S. 1989. “Types and Tokens in Linguistics.” In Reflections on Chomsky, edited by A. George, 58–90. Basil Blackwell; Kaplan, D. 1990. “Words.” Aristotelian Society Supplementary Volume LXIV: 93–119]. In this paper, I present a novel account of words which differs from the atomistic and platonistic conceptions of previous accounts which I argue fall prey to this problem. Specifically, I proffer a structuralist account of linguistic items, along the lines of structuralism in the philosophy of mathematics [Shapiro, S. 1997. Philosophy of Mathematics: Structure and Ontology. Oxford University Press], in which words are defined in part as positions in larger linguistic structures. I then follow Szabò [1999. “Expressions and Their Representations.” The Philosophical Quarterly 49 (195): 145–163] and Parsons [1990. “The Structuralist View of Mathematical Objects.” Synthese 84: 303–346] in further defining words as quasi-concrete objects according to a representation relation. This view aims for general correspondence with contemporary generative linguistic approaches to the study of language.     

The Standard Metre in Paris

In Philosophical Investigations, Wittgenstein argues that we can neither say of the standard One Metre in Paris that it is a single metred length, nor that it is not. Kripke's reply to the puzzle is well known: the sentence expressing the assertion that the standard One Metre is one metre in length (at time t₀) is a true, a priori and contingent sentence. In this paper, I would like to show the nature of the intuition that runs behind Kripke's reply to the puzzle, and why, in the final analysis, it is not satisfactory, with respect to the point made by Wittgenstein. In addition, I will show that the case of the One Metre in Paris exemplifies the radical break Wittgenstein makes with traditional concepts of meaning. I then draw a general lesson that shows that the structure of concepts and functions (measures) in Wittgenstein is given by the idea of an arbitrary choice of “an object of comparison.” Concepts and functions (measures) are materialised and internalised in the form of objects that are arbitrarily sampled from a sample space of same logical‐type objects.     

What Are the “True” Statistics of the Environment?

A widespread assumption in the contemporary discussion of probabilistic models of cognition, often attributed to the Bayesian program, is that inference is optimal when the observer's priors match the true priors in the world—the actual “statistics of the environment.” But in fact the idea of a “true” prior plays no role in traditional Bayesian philosophy, which regards probability as a quantification of belief, not an objective characteristic of the world. In this paper I discuss the significance of the traditional Bayesian epistemic view of probability and its mismatch with the more objectivist assumptions about probability that are widely held in contemporary cognitive science. I then introduce a novel mathematical framework, the observer lattice, that aims to clarify this issue while avoiding philosophically tendentious assumptions. The mathematical argument shows that even if we assume that “ground truth” probabilities actually do exist, there is no objective way to tell what they are. Different observers, conditioning on different information, will inevitably have different probability estimates, and there is no general procedure to determine which one is right. The argument sheds light on the use of probabilistic models in cognitive science, and in particular on what exactly it means for the mind to be “tuned” to its environment.     

“Dignity was the First to Leave”:* Godinez V. Moran,Colin Ferguson,and the Trial of Mentally Disabled Criminal Defendants

This article considers the Colin Ferguson trial in the context of the United States Supreme Court's decision in Godinez v. Moran, establishing a unitary standard for the determinations of competence to stand trial, competence to plead guilty, and competence to waive counsel. The Ferguson trial was widely seen as a “charade.” I argue that the Ferguson spectacle was the inevitable denouement of the Godinez decision. I then look at the Ferguson trial (through contemporaneous press and television coverage) under the filters of “sanism” and “pretextuality.” I conclude that the “dignity” value—a prerequisite for a constitutionally-acceptable fair trial—was, as a result of Godinez, lacking in the Ferguson case.     

The asymmetry and temporal dynamics of incidental letter–location bindings in working memory

Verbal–spatial bindings are integral to routine cognitive operations (e.g., reading), yet the processes supporting them in working memory are little understood. Campo and colleagues [Campo, P., Poch, C., Parmentier, F. B. R., Moratti, S., Elsley, J. V., Castellanos, N., … Maestú, F. (2010). Oscillatory activity in prefrontal and posterior regions during implicit letter-location binding. Neuroimage, 49, 2807–2815] recently reported data suggesting obligatory letter–location binding when participants were directed to remember the letters in a display (of letters in locations), but no evidence for binding when instructed to remember the filled locations. The present study contrasted two explanations for this binding asymmetry. First, it may result from an obligatory dependence on “where” during the representation of “what” information, while “where” information may be held independently of its contents (the strong asymmetry hypothesis). Second, it may constitute a snapshot of a dynamic feature inhibition process that had partially completed by test: the asymmetrical inhibition hypothesis. Using Campo and colleagues’ task with a variable retention interval between display and test, we presented four consonants in distinct locations and contrasted performance between “remember letters” and “remember locations” instructions. Our data supported the strong asymmetry hypothesis through demonstrating binding in the verbal task, but not in the spatial task. Critically, when present, verbal–spatial bindings were remarkably stable, enduring for at least 15 seconds.     

DIMENSIONS OF COMPLIANCE-GAINING SITUATIONS

A seven-factor model of situation perception, including “personal benefits,”“intimacy,”“rights,”“resistance,”“dominance,”“situation apprehension,” and “relational consequences” was proposed. Confirmatory factor analysis was employed to test the fit of the model to the data. One sample (N=450) was employed to test a six-factor sub model (excluding relational consequences) and a separate sample (N= 270), employing a different situation, was utilized to test the seven-factor model. The proposed model proved to fit the data well (x²/df ratios of 3.14 and 2.97, respectively) and was superior to alternative models. Future research recommendations were provided.     

Concepts of Creativity: Similarities and Differences among Mainland,Hong Kong and Taiwanese Chinese

Aspects of creativity concepts across different Chinese populations were examined. A Likert style questionnaire consisting of 60 adjectives was administered to 451 undergraduates from Beijing, Guangzhou, Taipei and Hong Kong. The results show that: (a) the core characteristics of creativity identical in all the samples are: “originality”, “innovativeness”, “thinking” and “observational skills”, “flexibility”, “willingness to try”, “self confidence”, and “imagination”; (b) the Taipei sample, unlike the other three samples, does not associate “wisdom”, “assertiveness”, and “individualism” with creativity; (c) in all Chinese populations the three factors labeled innovative, dynamic, and intellectual were distinguishable in the concept of creativity; (d) “artistic” and “humorous” were missing in the Chinese perception of creativity; (e) creativity characteristics received relatively low ratings on the desirability scale.     

THE PROBLEM OF “GOD” IN PSYCHOLOGY OF RELIGION: LONERGAN'S “COMMON SENSE” (RELIGION) VERSUS “THEORY” (THEOLOGY)

The emphasis on God in American psychology of religion generates the problem of explaining divine‐versus‐natural causality in “spiritual experiences.” Especially “theistic psychology” champions divine involvement. However, its argument exposes a methodological error: to pit popular religious opinions against technical scientific conclusions. Countering such homogenizing “postmodern agnosticism,” Bernard Lonergan explained these two as different modes of thinking: “common sense” and “theory”—which resolves the problem: When theoretical science is matched with theoretical theology, “the God‐hypothesis” explains the existence of things whereas science explains their natures; and, barring miracles, God is irrelevant to natural science. A review of the field shows that the problem is pervasive; attention to “miracles”—popularly so‐named versus technically—focuses the claims of divine‐versus‐natural causality; and specifications of the meaning of spiritual, spirituality, science, worldview, and meaning itself (suffering that same ambiguity: personal import versus cognitive content) offer further clarity. The problem is not naturalism versus theism, but commonsensical versus theoretical thinking. This solution demands “hard” social science.