Coherence and Reliability: The Case of Overlapping Testimonies

A measure of coherence is said to be reliability conducive if and only if a higher degree of coherence (as measured) among testimonies implies a higher probability that the witnesses are reliable. Recently, it has been proved that several coherence measures proposed in the literature are reliability conducive in scenarios of equivalent testimonies (Olsson and Schubert 2007; Schubert, to appear). My aim is to investigate which coherence measures turn out to be reliability conducive in the more general scenario where the testimonies do not have to be equivalent. It is shown that four measures are reliability conducive in the present scenario, all of which are ordinally equivalent to the Shogenji measure. I take that to be an argument for the Shogenji measure being a fruitful explication of coherence.     

Is coherence conducive to reliability?

A measure of coherence is said to be reliability conducive if and only if a higher degree of coherence (as measured) of a set of testimonies implies a higher probability that the witnesses are reliable. Recently, it has been proved that the Shogenji measure of coherence is reliability conducive in restricted scenarios (e.g., Olsson and Schubert, Synthese, 157:297?C308, 2007). In this article, I investigate whether the Shogenji measure, or any other coherence measure, is reliability conducive in general. An impossibility theorem is proved to the effect that this is not the case. I conclude that coherence is not reliability conducive.     

Reliability conducive measures of coherence

A measure of coherence is said to be truth conducive if and only if a higher degree of coherence (as measured) results in a higher likelihood of truth. Recent impossibility results strongly indicate that there are no (non-trivial) probabilistic coherence measures that are truth conducive. Indeed, this holds even if truth conduciveness is understood in a weak ceteris paribus sense (Bovens & Hartmann, 2003, Bayesian epistemology. New York, Oxford: Oxford University Press; Olsson, 2005, Against coherence: Truth probability and justification. Oxford: Oxford University Press). This raises the problem of how coherence could nonetheless be an epistemically important property. Our proposal is that coherence may be linked in a certain way to reliability. We define a measure of coherence to be reliability conducive if and only if a higher degree of coherence (as measured) results in a higher probability that the information sources are reliable. Restricting ourselves to the most basic case, we investigate which coherence measures in the literature are reliability conducive. It turns out that, while a number of measures fail to be reliability conducive, except possibly in a trivial and uninteresting sense, Shogenji’s measure and several measures generated by Douven and Meijs’s recipe are notable exceptions to this rule.     

Coherence,striking agreement,and reliability

Striving for a probabilistic explication of coherence, scholars proposed a distinction between agreement and striking agreement. In this paper I argue that only the former should be considered a genuine concept of coherence. In a second step the relation between coherence and reliability is assessed. I show that it is possible to concur with common intuitions regarding the impact of coherence on reliability in various types of witness scenarios by means of an agreement measure of coherence. Highlighting the need to separate the impact of coherence and specificity on reliability it is finally shown that a recently proposed vindication of the Shogenji measure qua measure of coherence vanishes.     

EXPLAINING THE LIMITS OF OLSSON'S IMPOSSIBILITY RESULT

In his groundbreaking book, Against Coherence (2005), Erik Olsson presents an ingenious impossibility theorem that appears to show that there is no informative relationship between probabilistic measures of coherence and higher likelihood of truth. Although Olsson's result provides an important insight into probabilistic models of epistemological coherence, the scope of his negative result is more limited than generally appreciated. The key issue is the role conditional independence conditions play within the witness testimony model Olsson uses to establish his result. Olsson maintains that his witness model yields charitable ceteris paribus conditions for any theory of probabilistic coherence. Not so. In fact, Olsson's model, like Bayesian witness models in general, selects a peculiar class of models that are in no way representative of the range of options available to coherence theorists. Recent positive results suggest that there is a way to develop a formal theory of coherence after all. Further, although Bayesian witness models are not conducive to the truth, they are conducive to reliability.     

Confirmation and justification. A commentary on Shogenji’s measure

So far no known measure of confirmation of a hypothesis by evidence has satisfied a minimal requirement concerning thresholds of acceptance. In contrast, Shogenji’s new measure of justification (Shogenji, Synthese, this number 2009) does the trick. As we show, it is ordinally equivalent to the most general measure which satisfies this requirement. We further demonstrate that this general measure resolves the problem of the irrelevant conjunction. Finally, we spell out some implications of the general measure for the Conjunction Effect; in particular we give an example in which the effect occurs in a larger domain, according to Shogenji justification, than Carnap’s measure of confirmation would have led one to expect.     

Probabilistic Measures of Coherence and the Problem of Belief Individuation

Coherentism in epistemology has long suffered from lack of formal and quantitative explication of the notion of coherence. One might hope that probabilistic accounts of coherence such as those proposed by Lewis, Shogenji, Olsson, Fitelson, and Bovens and Hartmann will finally help solve this problem. This paper shows, however, that those accounts have a serious common problem: the problem of belief individuation. The coherence degree that each of the accounts assigns to an information set (or the verdict it gives as to whether the set is coherent tout court) depends on how beliefs (or propositions) that represent the set are individuated. Indeed, logically equivalent belief sets that represent the same information set can be given drastically different degrees of coherence. This feature clashes with our natural and reasonable expectation that the coherence degree of a belief set does not change unless the believer adds essentially new information to the set or drops old information from it; or, to put it simply, that the believer cannot raise or lower the degree of coherence by purely logical reasoning. None of the accounts in question can adequately deal with coherence once logical inferences get into the picture. Toward the end of the paper, another notion of coherence that takes into account not only the contents but also the origins (or sources) of the relevant beliefs is considered. It is argued that this notion of coherence is of dubious significance, and that it does not help solve the problem of belief individuation.     

Ways in which coherence is confirmation conducive

Recent works in epistemology show that the claim that coherence is truth conducive – in the sense that, given suitable ceteris paribus conditions, more coherent sets of statements are always more probable – is dubious and possibly false. From this, it does not follows that coherence is a useless notion in epistemology and philosophy of science. Dietrich and Moretti (Philosophy of science 72(3): 403–424, 2005) have proposed a formal of account of how coherence is confirmation conducive—that is, of how the coherence of a set of statements facilitates the confirmation of such statements. This account is grounded in two confirmation transmission properties that are satisfied by some of the measures of coherence recently proposed in the literature. These properties explicate everyday and scientific uses of coherence. In his paper, I review the main findings of Dietrich and Moretti (2005) and define two evidence-gathering properties that are satisfied by the same measures of coherence and constitute further ways in which coherence is confirmation conducive. At least one of these properties vindicates important applications of the notion of coherence in everyday life and in science.     

When “reliability” fails,must a measure be discarded?—The case of Kinesthetic Aftereffect

Critics of Kinesthetic Aftereffect (KAE) recommend abandoning it as a personality measure largely because of poor test-retest reliability. Although no test can be valid if lacking true reliability, to discard a measure because of poor retest reliability is an oversimplification of validation procedures. This pitfall is exemplified here by a reexamination of KAE. KAE scores involve measures before (pretest) and after (test) aftereffect induction. Internal analysis of a KAE study showed: Differential bias is present; its locus is the second session pretest; its form makes second-session pretest scores functionally more similar to first- and second-session test scores and functionally more dissimilar to first-session pretest scores. Given this second session bias, the retest correlation tells us nothing about the true reliability of a one-session KAE score. However, if a measure possesses external validity, it must to some degree show true reliability. Based upon a literature review of one-session KAE validity studies, we conclude that one-session KAE scores are valid and hence show true reliability. KAE remains a promising personality measure.     

The Impossibility of Coherence

There is an emerging consensus in the literature on probabilistic coherence that such coherence cannot be truth conducive unless the information sources providing the cohering information are individually credible and collectively independent. Furthermore, coherence can at best be truth conducive in a ceteris paribus sense. Bovens and Hartmann have argued that there cannot be any measure of coherence that is truth conducive even in this very weak sense. In this paper, I give an alternative impossibility proof. I provide a relatively detailed comparison of the two results, which turn out to be logically unrelated, and argue that my result answers a question raised by Bovens and Hartmann’s study. Finally, I discuss the epistemological ramifications of these findings and try to make plausible that a shift to an explanatory framework such as Thagard’s is unlikely to turn the impossibility into a possibility.     

WITNESS AGREEMENT AND THE TRUTH‐CONDUCIVENESS OF COHERENTIST JUSTIFICATION

Some recent work in formal epistemology shows that “witness agreement” by itself implies neither an increase in the probability of truth nor a high probability of truth—the witnesses need to have some “individual credibility.” It can seem that, from this formal epistemological result, it follows that coherentist justification (i.e., doxastic coherence) is not truth‐conducive. I argue that this does not follow. Central to my argument is the thesis that, though coherentists deny that there can be noninferential justification, coherentists do not deny that there can be individual credibility.     

The suggestibility of child witnesses: A review

The idea that child witnesses are unreliable because of their high suggestibility is a common one. However, it is questionable if suggestibility can be attributed to individuals without considering the situational determinants of the phenomenon. This paper reviews studies of suggestibility in adults and children with particular reference to its determinants. The possibility that interview techniques might be developed which could enhance the reliability of child witnesses is considered.     

Why There Cannot be a Single Probabilistic Measure of Coherence

Bayesian Coherence Theory of Justification or, for short, Bayesian Coherentism, is characterized by two theses, viz. (i) that our degree of confidence in the content of a set of propositions is positively affected by the coherence of the set, and (ii) that coherence can be characterized in probabilistic terms. There has been a longstanding question of how to construct a measure of coherence. We will show that Bayesian Coherentism cannot rest on a single measure of coherence, but requires a vector whose components exhaustively characterize the coherence properties of the set. Our degree of confidence in the content of the information set is a function of the reliability of the sources and the components of the coherence vector. The components of this coherence vector are weakly but not strongly separable, which blocks the construction of a single coherence measure.     

Problems with Priors in Probabilistic Measures of Coherence

Two of the probabilistic measures of coherence discussed in this paper take probabilistic dependence into account and so depend on prior probabilities in a fundamental way. An example is given which suggests that this prior-dependence can lead to potential problems. Another coherence measure is shown to be independent of prior probabilities in a clearly defined sense and consequently is able to avoid such problems. The issue of prior-dependence is linked to the fact that the first two measures can be understood as measures of coherence as striking agreement, while the third measure represents coherence as agreement. Thus, prior (in)dependence can be used to distinguish different conceptions of coherence.     

Coherence measures and inference to the best explanation

This paper considers an application of work on probabilistic measures of coherence to inference to the best explanation (IBE). Rather than considering information reported from different sources, as is usually the case when discussing coherence measures, the approach adopted here is to use a coherence measure to rank competing explanations in terms of their coherence with a piece of evidence. By adopting such an approach IBE can be made more precise and so a major objection to this mode of reasoning can be addressed. Advantages of the coherence-based approach are pointed out by comparing it with several other ways to characterize ‘best explanation’ and showing that it takes into account their insights while overcoming some of their problems. The consequences of adopting this approach for IBE are discussed in the context of recent discussions about the relationship between IBE and Bayesianism.     

The defeasible nature of coherentist justification

The impossibility results of Bovens and Hartmann (2003, Bayesian epistemology. Oxford: Clarendon Press) and Olsson (2005, Against coherence: Truth, probability and justification. Oxford: Oxford University Press.) show that the link between coherence and probability is not as strong as some have supposed. This paper is an attempt to bring out a way in which coherence reasoning nevertheless can be justified, based on the idea that, even if it does not provide an infallible guide to probability, it can give us an indication thereof. It is further shown that this actually is the case, for several of the coherence measures discussed in the literature so far. We also discuss how this affects the possibility to use coherence as a means of epistemic justification.     

A note on using alpha and stratified alpha to estimate the reliability of a test composed of item parcels

Several authors have suggested that prior to conducting a confirmatory factor analysis it may be useful to group items into a smaller number of item ‘parcels’ or ‘testlets’. The present paper mathematically shows that coefficient alpha based on these parcel scores will only exceed alpha based on the entire set of items if W, the ratio of the average covariance of items between parcels to the average covariance of items within parcels, is greater than unity. If W is less than unity, however, and errors of measurement are uncorrelated, then stratified alpha will be a better lower bound to the reliability of a measure than the other two coefficients. Stratified alpha are also equal to the true reliability of a test when items within parcels are essentially tau‐equivalent if one assumes that errors of measurement are not correlated.     

Plausibilistic coherence

Why should coherence be an epistemic desideratum? One response is that coherence is truth-conducive: mutually coherent propositions are more likely to be true, ceteris paribus, than mutually incoherent ones. But some sets of propositions are more coherent, while others are less so. How could coherence be measured? Probabilistic measures of coherence exist; some are identical to probabilistic measures of confirmation, while others are extensions of such measures. Probabilistic measures of coherence are fine when applicable, but many situations are so information-poor that the requisite probabilities cannot be determined. To measure coherence in these cognitively impoverished situations, this article proposes that the discussion be broadened to include plausibilistic measures of coherence. It shows how plausibilistic measures of coherence can be defined using plausibilistic measures of confirmation. It then illustrates how plausibilisic coherence can be measured in situations where probabilistic coherence cannot be determined. The coherence values obtained through the use of plausibilistic measures are often, though not always, comparable. The article also shows that coherence can be instantiated on different levels, one of which permits connections to inductive strength and deductive validity.