用“逻辑陷阱”实现测谎功能的方法和数学证明

在心理测验过程中,有些受试者往往会说谎,作假答案,导致测验结果失真。本文使用逻辑推理的理论,探讨了在测验量表中设置“逻辑陷阱”来捕捉说谎者的方法。详细论述“逻辑陷阱”的测谎原理、设计方法。并给出具体推理过程的数学描述和证明。     

Evidence and inference in educational assessment

Educational assessment concerns inference about students' knowledge, skills, and accomplishments. Because data are never so comprehensive and unequivocal as to ensure certitude, test theory evolved in part to address questions of weight, coverage, and import of data. The resulting concepts and techniques can be viewed as applications of more general principles for inference in the presence of uncertainty. Issues of evidence and inference in educational assessment are discussed from this perspective.Probability isn't really about numbers; it's about the structure of reasoning.Presidential address to the Psychometric Society, presented June 25, 1994, in Champaign, Illinois.Supported by (1) Contract No. N00014-91-J-4101, R&T 4421573-01, from the Cognitive Science Program, Cognitive and Neural Sciences Division, Office of Naval Research, (2) the National Center for Research on Evaluation, Standards, Student Testing (CRESST), Educational Research and Development Program, cooperative agreement number R117G10027 and CFDA catalog number 84.117G, as administered by the Office of Educational Research and Improvement, U.S. Department of Education, and (3) the Statistical and Psychometric Research Division of Educational Testing Service. I am grateful for comments and suggestions from Henry Braun, Drew Gitomer, Richard Patz, Jonathan Troper, and Howard Wainer.     

Estimation of the hypothesis hierarchy in probabilistic inference tasks*

Is there a hierarchical order among the hypotheses about functional rules in probabilistic inference tasks, i.e. what is the construction and the procedure of the “hypothesis sampling mechanism” employed by the subjects in this kind of task? According to the hypothesis sampling model initially proposed by Brehmer (1974) there should be a hierarchical order among the hypotheses in the subject's hypothesis pools. The procedures of hypothesis sampling and testing ought to follow this strict data independent order (see e.g. Sniezek, 1986; Brehmer, 1987). Knez (1991a, b) showed, however, that this assumption may be incorrect. As a follow up to these results the question regarding the construction of the subject's hypothesis pools was reapproached in the present study. The results indicated a consistency with the hierarchical assumption (Brehmer, 1974) only regarding the relation between the linear and nonlinear rules but not within these types of rules.     

On Keats' generalization of the rasch model

Parameter estimation for Keats generalization of the Rasch model that takes account of guessing behavior is investigated. It is shown that no minimal sufficient statistics for the ability parameters independent of the difficulty parameters exist. Thus Andersen's conditional inference technique for consistent estimation is not applicable to Keats' model. The notion of separability of the parameters is generalized.     

Covariation and quantifier polarity: what determines causal attribution in vignettes?

Tests of causal attribution often use verbal vignettes, with covariation information provided through statements quantified with natural language expressions. The effect of covariation information has typically been taken to show that set size information affects attribution. However, recent research shows that quantifiers provide information about discourse focus as well as covariation information. In the attribution literature, quantifiers are used to depict covariation, but they confound quantity and focus. In four experiments, we show that focus explains all (Experiment 1) or some (Experiment 2-4) of the impact of covariation information on the attributions made, confirming the importance of the confound. Attribution experiments using vignettes that present covariation information with natural language quantifiers may overestimate the impact of set size information, and ignore the impact of quantifier-induced focus.     

Do We “do”?

A normative framework for modeling causal and counterfactual reasoning has been proposed by Spirtes, Glymour, and Scheines (1993; cf. Pearl, 2000). The framework takes as fundamental that reasoning from observation and intervention differ. Intervention includes actual manipulation as well as counterfactual manipulation of a model via thought. To represent intervention, Pearl employed the do operator that simplifies the structure of a causal model by disconnecting an intervened-on variable from its normal causes. Construing the do operator as a psychological function affords predictions about how people reason when asked counterfactual questions about causal relations that we refer to as undoing, a family of effects that derive from the claim that intervened-on variables become independent of their normal causes. Six studies support the prediction for causal (A causes B) arguments but not consistently for parallel conditional (if A then B) ones. Two of the studies show that effects are treated as diagnostic when their values are observed but nondiagnostic when they are intervened on. These results cannot be explained by theories that do not distinguish interventions from other sorts of events.     

The perception of causality in chimpanzees (Pan spp.)

Chimpanzees (Pan spp.) were tested on a habituation/dishabituation paradigm that was originally developed to test for comprehension of causality in very young human infants. Three versions of the test were used: a food item being moved by a hand, a human pushing another human off a chair to obtain a food item, and a film clip of natural chimpanzee behaviour (capturing and eating a monkey). Chimpanzees exhibited similar results to those obtained with human infants, with significantly elevated levels of looking on the dishabituation trials. Since the level of response was significantly greater on natural/unnatural sequences than on unnatural/natural sequences, we conclude that the chimpanzees were not responding just to novelty but rather to events that infringed their sense of natural causation.     

On the sampling interpretation of confidence intervals and hypothesis tests in the context of conditional maximum likelihood estimation

In the context ofconditional maximum likelihood (CML) estimation, confidence intervals can be interpreted in three different ways, depending on the sampling distribution under which these confidence intervals contain the true parameter value with a certain probability. These sampling distributions are (a) the distribution of the data given theincidental parameters, (b) the marginal distribution of the data (i.e., with the incidental parameters integrated out), and (c) the conditional distribution of the data given the sufficient statistics for the incidental parameters. Results on the asymptotic distribution of CML estimates under sampling scheme (c) can be used to construct asymptotic confidence intervals using only the CML estimates. This is not possible for the results on the asymptotic distribution under sampling schemes (a) and (b). However, it is shown that theconditional asymptotic confidence intervals are also valid under the other two sampling schemes. I am indebted to Theo Eggen, Norman Verhelst and one of Psychometrika's reviewers for their helpful comments.     

Visual analysis of single-case time series: Effects of variability, serial dependence, and magnitude of intervention effects

Visual analysis is the dominant method of analysis for single-case time series. The literature assumes that visual analysts will be conservative judges. We show that previous research into visual analysis has not adequately examined false alarm and miss rates or the effect of serial dependence. In order to measure false alarm and miss rates while varying serial dependence, amount of random variability, and effect size, 37 students undertaking a postgraduate course in single-case design and analysis were required to assess the presence of an intervention effect in each of 27 AB charts constructed using a first-order autoregressive model. Three levels of effect size and three levels of variability, representative of values found in published charts, were combined with autocorrelation coefficients of 0, 0.3 and 0.6 in a factorial design. False alarm rates were surprisingly high (16% to 84%). Positive autocorrelation and increased random variation both significantly increased the false alarm rates and interacted in a nonlinear fashion. Miss rates were relatively low (0% to 22%) and were not significantly affected by the design parameters. Thus, visual analysts were not conservative, and serial dependence did influence judgment.