Sampling fluctuations resulting from the sampling of test items

Sampling fluctuations resulting from the sampling of test items rather than of examinees are discussed. It is shown that the Kuder-Richardson reliability coefficients actually are measures of this type of sampling fluctuation. Formulas for certain standard errors are derived; in particular, a simple formula is given for the standard error of measurement of an individual examinee's score. A common misapplication of the Wilks-Votaw criterion for parallel tests is pointed out. It is shown that the Kuder-Richardson formula-21 reliability coefficient should be used instead of the formula-20 coefficient in certain common practical situations.Most of the work reported here was carried out under contract with the Office of Naval Research. The writer is indebted to Professor S. S. Wilks, who has checked over certain critical portions of a draft of this paper.     

Classical and bayesian inference in certain latency processes

Certain aspects of point estimation are treated for two kinds of exponential latency processes. The first unitary process is represented by a simple exponential density in which the rate parameter may be viewed as an unknown constant or as a random variable. If a second, slower exponential process is grafted onto the first, there results a postulated two-component latency between stimulus and response. Moments estimators are derived for the two parameters of this latter density, and the relevance of the second parameter to decision time is emphasized.This research was supported in part by National Institutes of Health Grant MH-04439-05. The author would like to express his appreciation to James Baker of Oregon Research Institute for certain helpful comments concerning aspects of this work.     

Differences in factor content of rights and wrongs scores

The right-response scores and wrong-response scores of speeded aptitude tests were factor analyzed to determine whether they differ in factorial content. The information thus obtained was used to derive scoring formulas that yield purer measures of a factor than do scoring formulas derived in other ways.This paper is a revision of a dissertation submitted in partial fulfillment of the requirements for the Doctor of Philosophy degree at the University of Southern California, 1948, and a paper read to the American Psychological Association in September, 1949. The writer is greatly indebted to Dr. J. P. Guilford for providing the intercorrelation and distribution data from the files of the School of Aviation Medicine and for general guidance throughout the study. He also wishes to express his appreciation to T/Sgt. Frank C. Ivens and Sgt. James R. MacDonald for computational assistance in the extraction and rotation of the centroid factors, which were performed while the writer was a civilian employee of the Air Training Command, Human Resources Research Center. The opinions expressed are those of the writer and do not necessarily reflect the official views of the USAF.     

Ignorance or integration: the cognitive processes underlying choice behavior

The fast‐and‐frugal heuristic framework assumes noncompensatory tools for human preferences (i.e., priority heuristic) and inferences (i.e., take the best heuristic). According to this framework, these heuristics predict choice behavior as well as model the cognitive processes underlying such behavior. The current paper presents two studies that juxtapose predictions derived from these two simple heuristics with alternative predictions derived from compensatory principles. Dependent measures that included reaction time, choice pattern, confidence level, and accuracy were better predicted by compensatory indices than by noncompensatory indices. These findings suggest that people do not rely on limited arguments only, but tend to integrate all acquired information into their choice processes. This tendency was replicated even when the experimental task facilitated the use of noncompensatory principles. We argue that the fast and frugal heuristics can predict the final outcome only under certain conditions, but even in these particular situations they are not applicable to the processes underlying choice behavior. An integrative model for choice behavior is proposed that better represents the data. Copyright © 2009 John Wiley & Sons, Ltd.     

A Predictive Approach to Nonparametric Inference for Adaptive Sequential Sampling of Psychophysical Experiments

We present a predictive account on adaptive sequential sampling of stimulus-response relations in psychophysical experiments. Our discussion applies to experimental situations with ordinal stimuli when there is only weak structural knowledge available such that parametric modeling is no option. By introducing a certain form of partial exchangeability, we successively develop a hierarchical Bayesian model based on a mixture of Pólya urn processes. Suitable utility measures permit us to optimize the overall experimental sampling process. We provide several measures that are either based on simple count statistics or more elaborate information theoretic quantities. The actual computation of information theoretic utilities often turns out to be infeasible. This is not the case with our sampling method, which relies on an efficient algorithm to compute exact solutions of our posterior predictions and utility measures. Finally, we demonstrate the advantages of our framework on a hypothetical sampling problem.     

Random fluctuations of response rate

A simple model for fluctuating interresponse times is developed and studied. It involves a mechanism that generates regularly spaced excitations, each of which can trigger off a response after a random delay. The excitations are not observable, but their periodicity is reflected in a regular patterning of responses. The probability distribution of the time between responses is derived and its properties are analyzed. Several limiting cases are also examined.This paper was completed while the writer was a visiting summer scientist at the Lincoln Laboratory, Lexington, Mass.     

Value-based decision-making battery: A Bayesian adaptive approach to assess impulsive and risky behavior

Using simple mathematical models of choice behavior, we present a Bayesian adaptive algorithm to assess measures of impulsive and risky decision making. Practically, these measures are characterized by discounting rates and are used to classify individuals or population groups, to distinguish unhealthy behavior, and to predict developmental courses. However, a constant demand for improved tools to assess these constructs remains unanswered. The algorithm is based on trial-by-trial observations. At each step, a choice is made between immediate (certain) and delayed (risky) options. Then the current parameter estimates are updated by the likelihood of observing the choice, and the next offers are provided from the indifference point, so that they will acquire the most informative data based on the current parameter estimates. The procedure continues for a certain number of trials in order to reach a stable estimation. The algorithm is discussed in detail for the delay discounting case, and results from decision making under risk for gains, losses, and mixed prospects are also provided. Simulated experiments using prescribed parameter values were performed to justify the algorithm in terms of the reproducibility of its parameters for individual assessments, and to test the reliability of the estimation procedure in a group-level analysis. The algorithm was implemented as an experimental battery to measure temporal and probability discounting rates together with loss aversion, and was tested on a healthy participant sample.     

Spaced responding and choice: a preliminary analysis

Pigeons were exposed to reinforcement both for short (2 < IRT < 3 sec) and long (10 < IRT < 11 sec) interresponse times. They developed bimodal interresponse-time distributions, which were decomposable into two independent component distributions under the control of the short and long contingencies respectively. The birds' allocation of responses between these two distributions was determined by a simple power-law relationship between reinforcement ratios, and response ratios derived from the component distributions. Comparison between this situation and concurrent choice situations raises the possibility that the power-law relation between ratios may be a more general law of choice than the matching of relative frequencies (probabilities).     

The latencies of correct and incorrect responses in discrimination and detection tasks: Their interpretation in terms of a model based on simple counting

A model for two-choice discrimination based on a process of simple counting is described, and two experiments are performed to test the predictions of the model concerning the graph of latency as a function of response proportion. Two main forms of this graph are identified and predicted to arise in different circumstances. The experimental results support the model, and its possible extension to other psychophysical situations, especially signal detection, are then discussed. It is compared with a model derived directly from the detection situation, and the usefulness of testing these models is pointed out.     

Choice as a dependent measure in autoshaping: sensitivity to frequency and duration of food presentation.

Previous investigations have shown that rate, latency, and percentage of trials with at least one response are somewhat insensitive measures of the strength of autoshaped responding. In the present studies, these measures were contrasted with the allocation of responding during simultaneous choice tests, a measure of response strength frequently used in operant paradigms. In two experiments, nine pigeons were exposed to a forward pairing autoshaping procedure. Training sessions consisted of the successive presentation of three stimuli, each followed by food on either 100%, 50%, or 0% of the trials. Choice testing involved the simultaneous presentation of the three stimuli. In Experiment I, all pigeons consistently directed their initial choice responses and the majority of subsequent responses to the stimulus always followed by food, despite the fact that during training sessions the response rates of most birds were highest in the presence of the stimulus followed by food on 50% of the trials. In Experiment II, rate, latency, and percentage of trials with at least one response did not change appreciably as a function of duration of feeder presentations. However, choice responding was lawfully affected by duration of feeder presentations. These data suggest that choice is perhaps a more sensitive measure of the strength of autoshaped responding than other, more commonly employed, indices.     

A model of problem solving with incomplete constraint knowledge

The problem-solving behavior of subjects presented with a series of Tower of Hanoi problems is examined. A production system model which incorporates elements of domain-specific knowledge into a general problem-solving framework is presented. Other models developed for the task are based on understanding of complete solution strategies and are not satisfactory models of nonexpert human performance. The current model discriminates between problem-solving behavior based on constraint knowledge and behavior based on nonspecific general search strategies. A variety of move choice and latency measures are used to compare the performance of the model to human subject performance.     

Verification of Donders' subtraction method

In an experiment on 6 young adults advance (precued) information of the correct choice response was utilized completely: (a) For precue-to-stimulus intervals (PSIs) clearly shorter than the difference between mean choice and simple reaction time (RT), median response latency (L) measured from precue onset was invariant; (b) For clearly longer PSIs, median RT was very near the value for simple RT. This precue-utilization effect would be expected if response actualization had been delayed until the response had been selected and if the requirement for discrimination and selection had no adverse effect on readiness to respond. Donders' (1868/1969) hypothesis in his initial application of the subtraction method, that choice and simple reactions are identical except for the serial insertion of discrimination and selection operations in the former, is thereby strongly supported. If this formulation is accepted, models that hold that response processing can overlap other processing stages may be considered valid only for response selection, not response actualization.     

An individual differences pure extinction process

A pure extinction process memory retrieval latency model is completely characterized by the number of items to be recalled n and the recall rate λ. In this paper λ is allowed to vary across the population of subjects. Results are given for arbitrary individual differences distributions on λ. When λ is distributed gamma the results are very simple and method of moment estimators are easily obtained. In addition the shape parameter of the gamma individual differences distribution turns out to be an index of the degree of heterogeneity of the recall parameter. The pure extinction process model is also appropriate for certain collection and detection tasks.     

Personality and the topography of the conditioned eyelid response

One-hundred and four women were tested on an eyelid conditioning paradigm in a 2 x 2 x 2 factorial design where two levels of US intensity (1 vs 3 p.s.i.) were balanced against two rest pause interpolations (after 25 and after 50 trials), and the presence or absence of a warning stimulus prior to CS-US presentation. Subjects were later classified as high, low or intermediate extraverts on the basis of a personality questionnaire. A very detailed analysis of conditioned responses was carried out, using both simple and composite measures including work-ratio, utility-ratio, CR frequency, peak latency, peak amplitude, response area and effective response area, degree of avoidance amplitude and latency, etc. Major findings related to similar effects of high intensity US vs low intensity US, and introversion vs extraversion; introverts react as if they were responding to more intense stimuli than extraverts. This finding cuts across other parameter variables, and supports Eysenck's formulation of personality-conditioning relationships in terms of higher cortical arousal in introverts as compared with extraverts.     

Choice of situations and congruence models of interactionism

Recent studies of person-situation interactions have focused nearly exclusively on the statistical rather than reciprocal form of the concept. In an attempt to reverse this trend, two models of reciprocal interactionism are offered: choice of situations and congruence response models. The choice model proposes that individuals select situations and avoid others on the basis of certain underlying needs and dispositions. According to the congruence model, individuals should experience greater positive affect and less negative affect in situations which are congruent with their personality characteristics. Individuals indicated the frequency with which thay had recently participated in various recreation situations and also reported the affects they felt when in those situations. Some support was found for both models of interactionism, thus the choice of model appeared stronger. Implications of the choice model for the personality consistency issue and personality assessment are discussed, and suggestions for future research are offered.     

Stochastic learning theories for a response continuum with non-determinate reinforcement

Continuous analogues of the finite linear and stimulus sampling theories are developed for non-determinate reinforcement schedules. Closed-form expressions are derived for the asymptotic response distribution and certain sequential statistics. Computations for a target experiment are given to illustrate the character of the theoretical results.This research was supported by the Rockefeller Foundation and the Group Psychology Branch of the Office of Naval Research.     

Two item response theory models for analysing normative forced‐choice personality items

This paper proposes two unidimensional item response theory (IRT) models for analysing normative forced‐choice personality items. Both models are derived from a common theoretical framework and arise as a result of different assumptions regarding the mechanism of choice. The simplest mechanism gives rise to the one‐parameter normal‐ogive model. The second mechanism gives rise to a new IRT model, which is closely related to the Coombs–Zinnes probabilistic unfolding model. The second model is compared theoretically to the normal‐ogive model in terms of item characteristic curves and amount of item information. Next, procedures for estimating the respondent and the item parameters in the second model are described. Finally, both models are empirically compared by using two well‐known personality measures.     

A Signal Detection Model for Multiple-Choice Exams

A model for multiple-choice exams is developed from a signal-detection perspective. A correct alternative in a multiple-choice exam can be viewed as being a signal embedded in noise (incorrect alternatives). Examinees are assumed to have perceptions of the plausibility of each alternative, and the decision process is to choose the most plausible alternative. It is also assumed that each examinee either knows or does not know each item. These assumptions together lead to a signal detection choice model for multiple-choice exams. The model can be viewed, statistically, as a mixture extension, with random mixing, of the traditional choice model, or similarly, as a grade-of-membership extension. A version of the model with extreme value distributions is developed, in which case the model simplifies to a mixture multinomial logit model with random mixing. The approach is shown to offer measures of item discrimination and difficulty, along with information about the relative plausibility of each of the alternatives. The model, parameters, and measures derived from the parameters are compared to those obtained with several commonly used item response theory models. An application of the model to an educational data set is presented.     

Delay and number of food reinforcers: Effects on choice and latencies

Pigeons were given a choice between two identical-duration situations (terminal links of chain schedules). One terminal link of the choice pair provided two food deliveries, and the other provided five. The exact times of these food deliveries differed between the terminal links and were varied over conditions. A single response during the initial link gave immediate access to the corresponding terminal link. Forced trials, during which only one of the initial-link keys was lighted, were interspersed with choice trials during which both initial-link keys were lighted. Choice tended to favor whichever terminal link was correlated with the higher sum of the immediacies (i.e., the sum of the reciprocals of the delays to each of the reinforcers following the choice, with all delays measured from the choice). Latencies on forced trials and on choice trials also were related (negatively) to the sum of the immediacies. This correlation among response measures (choice and latencies) suggests that both measures are manifestations of the effect of conditioned reinforcement on response tendencies.