Subjects' inferential performance and the interaction of data and hypotheses in probabilistic inference tasks

In a recent paper, Knez (1991) showed an interaction of data and hypotheses in probabilistic inference tasks. The results illustrated two, earlier not obtained, significant main effects on subjects' hypothesis sampling , viz. the effect of different forms of data presentation and subjects' execution of cognitive control over their hypothesis pool throughout the series of trials. The present paper followed up these results in that the subjects' hypothesis testing , in Knez (1991) was subjected to an analysis. Hence, to see if the effects mentioned above significantly influenced the subjects' hypothesis testing, as they did for subjects' hypothesis sampling. The results showed a consistency with Knez (1991), i.e. the results emphasize the interaction of data and hypothesis in probabilistic inference tasks, as well as the subjects' execution of cognitive control over their hypothesis pool concerning both the subjects' hypothesis sampling and testing.     

Learning and hypothesis testing in probabilistic inference tasks

The relation between learning of probabilistic inference tasks and the availability of correct hypotheses for the tasks in individual subjects was investigated in two experiments. The results showed that having the correct hypotheses as measured before learning the tasks was neither a sufficient nor a necessary condition for learning the task. It was concluded that the hypothesis measurements are not reliable enough to allow predictions on an individual level. The results also showed that subjects have not only the four basic functions: positive linear, negative linear, U-shaped and inversely U-shaped, in their hypothesis hierarchies but also J-shaped functions. That the subjects can leam such functions is thus no discodinnation of the hypotheses sampling conception of learning as assumed in earlier studies.     

Note on hypothesis testing in probabilistic inference tasks

Subjects' ability to test hypotheses about the form of functional relations in probabilistic inference tasks was investigated in two experiments which varied the functional relation in the task, the hypotheses to be tested, and the validity of the cue. The results showed that the subjects used the same amount of information regardless of the validity of the cue, that nonlinear hypotheses were harder to test accurately than linear hypothesis, and that hypotheses were harder to test when the relation in the task was nonlinear than when it was linear.     

Length of the postfeedback interval and learning of probabilistic inference tasks

Learning of negative linear and J-shaped cue-criterion relations in single-cue learning tasks was studied in an experiment which varied the length of the postfeedback interval. Contrary to the predictions from a hypothesis construction model assuming that subjects construct hypotheses from the data provided by the task, there was no effect of the postfeedback interval. The results were interpreted to mean that the subjects' constructions are based on elements stored in memory, rather than on data from the task.     

Statistical inference and sensitivity to sampling in 11-month-old infants

Research on initial conceptual knowledge and research on early statistical learning mechanisms have been, for the most part, two separate enterprises. We report a study with 11-month-old infants investigating whether they are sensitive to sampling conditions and whether they can integrate intentional information in a statistical inference task. Previous studies found that infants were able to make inferences from samples to populations, and vice versa [Xu, F., & Garcia, V. (2008). Intuitive statistics by 8-month-old infants. Proceedings of the National Academy of Sciences of the United States of America, 105, 5012-5015]. We found that when employing this statistical inference mechanism, infants are sensitive to whether a sample was randomly drawn from a population or not, and they take into account intentional information (e.g., explicitly expressed preference, visual access) when computing the relationship between samples and populations. Our results suggest that domain-specific knowledge is integrated with statistical inference mechanisms early in development.     

Effect of cue validity on learning of complex rules in probabilistic inference tasks

The effect of cue validity on the learning of positive and negative J-shaped relations in single-cue probability learning was studied. As would be expected from a hypothesis construction model, learning of the J-shaped relations was better when cue validity was high than when it was low. However, the results also showed that positive J-rules were learned more efficiently than negative J-rules. This shows that subjects do not construct their hypotheses on the basis of data from task only. It was concluded that subjects construct their hypotheses from elements stored in memory, and that data are used only to test their hypotheses.     

Effects of information-presentation mode and task complexity on the learning of probabilistic inference tasks

Learning of two-cue probabilistic inference tasks was studied in a 2 level of information-presentation mode (visual access to an ordered array of cue values vs. no such access) by 2 levels of task complexity (linear vs. nonlinear cue-criterion relations). The results replicated earlier findings in that performance was better in the linear tasks than in the nonlinear ones, and also showed that visual access to an ordered array of cue values facilitated performance, viz. there were no interactions. The results were interpreted as evidence that at least part of the suboptimal performance in ordinary experiments of this kind is due to cognitive strain introduced by the lack of control of the sequence of instances of cue-criterion pairs.     

Effects of information presentation mode and learning paradigm on the learning of probabilistic inference tasks under different levels of memory strain

The effects of information presentation mode (i.e. visual access vs. not access to an ordered array of cue combinations) and learning paradigm (i.e. selection vs. reception of cue-combinations) on learning of two-cue probabilistic inference tasks were studied in two 2 × 2 factorial experiments representing two levels of memory strain. The results showed that information presentation mode affected performance regardless of amount of memory strain, while learning paradigm affected performance only under conditions of high memory strain.     

A probabilistic model of intensive designs

Without internal validity, experimental data are uninterpretable. With intensive designs, most methods presented to quantify a design's internal validity have been subject to criticism. A probabilistic model of intensive designs is presented that demonstrates the high degree of internal validity of these designs without relying on adaptations from traditional inferential statistics. Where the experimenter is able to conform to the restrictions of the model, the equations provide an estimation of internal validity for either reversal or multiple-baseline designs. More importantly, the model provides mathematical bases for some of the common recommendations and design considerations in intensive research (such as the desirability of within-subject replications and of four or more multiple baselines).     

The Savanna-IQ interaction hypothesis: A critical examination of the comprehensive case presented in Kanazawa's The Intelligence Paradox

Kanazawa (2012b) has recently presented the most comprehensive case yet for his Savanna-IQ Interaction Hypothesis. According to the hypothesis, intelligence is a domain-specific adaptation which has been selected for as humans have moved away from the (evolutionarily familiar) Savanna. As such, ability in ‘evolutionarily novel’ tasks and ‘evolutionarily novel’ preferences are positively correlated with high IQ. This article will present a critical examination of the hypothesis, arguing that there is a strong case against anchoring human nature on the Savanna, the hypothesis predicts contradictory findings, there is empirical evidence against it, it is not falsifiable, and it is not necessary to explain that data which Kanazawa presents.     

The Quantitative/Qualitative Watershed for Rules of Uncertain Inference

We chart the ways in which closure properties of consequence relations for uncertain inference take on different forms according to whether the relations are generated in a quantitative or a qualitative manner. Among the main themes are: the identification of watershed conditions between probabilistically and qualitatively sound rules; failsafe and classicality transforms of qualitatively sound rules; non-Horn conditions satisfied by probabilistic consequence; representation and completeness problems; and threshold-sensitive conditions such as ‘preface’ and ‘lottery’ rules. Special Issue Formal Epistemology I. Edited by Branden Fitelson     

Effects of context on the rate of conjunctive responses in the probabilistic truth table task

The probabilistic truth table task involves assessing the probability of "If A then C" conditional sentences. Previous studies have shown that a majority of participants assess this probability as the conditional probability P(C│A) while a substantial minority responds with the probability of the conjunction A and C. In an experiment involving 96 participants, we investigated the impact on the rate of conjunctive responses of the context in which the task is framed. We show that a context intended to lead participants to consider all the possible cases (i.e. the throw of a die known to allow six possibilities) elicited more conjunctive responses than a context assumed not to have this effect (an unfamiliar deck of cards). These results suggest that the step of inferring the probability can distort our assessment of participants' interpretation of conditional sentences. This might compromise the validity of the probabilistic task in studying conditional reasoning.     

On the measurement of preferences in the analytic hierarchy process

In this paper we apply multiattribute value theory as a framework for examining the use of pairwise comparisons in the analytic hierarchy process (AHP). On one hand our analysis indicates that pairwise comparisons should be understood in terms of preference differences between pairs of alternatives. On the other hand it points out undesirable effects caused by the upper bound and the discretization of any given ratio scale. Both these observations apply equally well to the SMART procedure which also uses estimates of weight ratios. Furthermore, we demonstrate that the AHP can be modified so as to produce results similar to those of multiattribute value measurement; we also propose new balanced scales to improve the sensitivity of the AHP ratio scales. Finally we show that the so-called supermatrix technique does not eliminate the rank reversal phenomenon which can be attributed to the normalizations in the AHP. © 1997 John Wiley & Sons, Ltd.     

Likelihood inference on the underlying structure of IRT models

The assumptions underlying item response theory (IRT) models may be expressed as a set of equality and inequality constraints on the parameters of a latent class model. It is well known that the same assumptions imply that the parameters of the manifest distribution have to satisfy a more complicated set of inequality constraints which, however, are necessary but not sufficient. In this paper, we describe how the theory for likelihood-based inference under equality and inequality constraints may be used to test the underlying assumptions of IRT models. It turns out that the analysis based directly on the latent structure is simpler and more flexible. In particular, we indicate how several interesting extensions of the Rasch model may be obtained by partial relaxation of the basic constraints. An application to a data set provided by Educational Testing Service is used to illustrate the approach.We thank Dr. Gorman and Dr. Rogers of the Educational Testing Service for providing the data analyzed in Section 4. We also thank three reviewers for comments and suggestions.This revised article was published online in August 2005 with the PDF paginated correctly.     

Intervention effects and relative variation as dimensions in experts' use of visual inference

Recent research indicates that when analyzing graphically presented single-subject data, subjects trained in visual inference appear to attend to large changes between phases regardless of relative variation and do not differentiate among common intervention effect patterns. In this follow-up study, experts in applied behavior analysis completed a free-sort task designed to assess the effects of these dimensions on their use of visual inference. The results indicate that they tended to differentiate among common intervention effect patterns but did not attend to relative variation in the data.     

A computational-level explanation of the speed of goal inference

The ability to understand the goals that drive another person’s actions is an important social and cognitive skill. This is no trivial task, because any given action may in principle be explained by different possible goals (e.g., one may wave ones arm to hail a cab or to swat a mosquito). To select which goal best explains an observed action is a form of abduction. To explain how people perform such abductive inferences, Baker, Tenenbaum, and Saxe (2007) proposed a computational-level theory that formalizes goal inference as Bayesian inverse planning (BIP). It is known that general Bayesian inference–be it exact or approximate–is computationally intractable (NP-hard). As the time required for computationally intractable computations grows excessively fast when scaled from toy domains to the real world, it seems that such models cannot explain how humans can perform Bayesian inferences quickly in real world situations. In this paper we investigate how the BIP model can nevertheless explain how people are able to make goal inferences quickly. The approach that we propose builds on taking situational constraints explicitly into account in the computational-level model. We present a methodology for identifying situational constraints that render the model tractable. We discuss the implications of our findings and reflect on how the methodology can be applied to alternative models of goal inference and Bayesian models in general.     

Learning about the internal structure of categories through classification and feature inference

Previous research on category learning has found that classification tasks produce representations that are skewed toward diagnostic feature dimensions, whereas feature inference tasks lead to richer representations of within-category structure. Yet, prior studies often measure category knowledge through tasks that involve identifying only the typical features of a category. This neglects an important aspect of a category's internal structure: how typical and atypical features are distributed within a category. The present experiments tested the hypothesis that inference learning results in richer knowledge of internal category structure than classification learning. We introduced several new measures to probe learners' representations of within-category structure. Experiment 1 found that participants in the inference condition learned and used a wider range of feature dimensions than classification learners. Classification learners, however, were more sensitive to the presence of atypical features within categories. Experiment 2 provided converging evidence that classification learners were more likely to incorporate atypical features into their representations. Inference learners were less likely to encode atypical category features, even in a “partial inference” condition that focused learners' attention on the feature dimensions relevant to classification. Overall, these results are contrary to the hypothesis that inference learning produces superior knowledge of within-category structure. Although inference learning promoted representations that included a broad range of category-typical features, classification learning promoted greater sensitivity to the distribution of typical and atypical features within categories.     

On the sampling interpretation of confidence intervals and hypothesis tests in the context of conditional maximum likelihood estimation

In the context ofconditional maximum likelihood (CML) estimation, confidence intervals can be interpreted in three different ways, depending on the sampling distribution under which these confidence intervals contain the true parameter value with a certain probability. These sampling distributions are (a) the distribution of the data given theincidental parameters, (b) the marginal distribution of the data (i.e., with the incidental parameters integrated out), and (c) the conditional distribution of the data given the sufficient statistics for the incidental parameters. Results on the asymptotic distribution of CML estimates under sampling scheme (c) can be used to construct asymptotic confidence intervals using only the CML estimates. This is not possible for the results on the asymptotic distribution under sampling schemes (a) and (b). However, it is shown that theconditional asymptotic confidence intervals are also valid under the other two sampling schemes. I am indebted to Theo Eggen, Norman Verhelst and one of Psychometrika's reviewers for their helpful comments.     

Axiomatizations with Context Rules of Inference in Modal Logic

A certain type of inference rules in (multi-) modal logics, generalizing Gabbay's Irreflexivity rule, is introduced and some general completeness results about modal logics axiomatized with such rules are proved.     

Doubly Robust-Type Estimation for Covariate Adjustment in Latent Variable Modeling

Due to the difficulty in achieving a random assignment, a quasi-experimental or observational study design is frequently used in the behavioral and social sciences. If a nonrandom assignment depends on the covariates, multiple group structural equation modeling, that includes the regression function of the dependent variables on the covariates that determine the assignment, can provide reasonable estimates under the condition of correct specification of the regression function. However, it is usually difficult to specify the correct regression function because the dimensions of the dependent variables and covariates are typically large. Therefore, the propensity score adjustment methods have been proposed, since they do not require the specification of the regression function and have been applied to several applied studies. However, these methods produce biased estimates if the assignment mechanism is incorrectly specified. In order to make a more robust inference, it would be more useful to develop an estimation method that integrates the regression approach with the propensity score methodology. In this study we propose a doubly robust-type estimation method for marginal multiple group structural equation modeling. This method provides a consistent estimator if either the regression function or the assignment mechanism is correctly specified. A simulation study indicates that the proposed estimation method is more robust than the existing methods. This research was partially supported by the Ministry of Education, Science, Sports and Culture, Grant-in-Aid for Young Scientists (B), 187-30406.