Alphabetic confusion: A test of models for individuals

An earlier article reported extensive analyses of confusion data compiled from group averages (Townsend, 1971). The present study provided for essentially the same analyses with different long-term data obtained with two individuals, the primary intent being to examine the ability of recognition and scaling models to explain data at the individual level (which the recognition models purport to describe) and to compare the confusion characteristics of the English uppercase alphabet between the two Ss and between the individual Ss and the group-averaged data. The choice and overlap models were superior to the all-or-none model in predicting the empirical confusion matrices and tended to explain the data structure in a similar manner. Multidimensional scaling analysis again supported a Euclidean metric and suggested four or five underlying stimulus dimensions. However, as before, there were no overriding intuitively appealing psychological dimensions corresponding to these, and possible reasons are discussed. The choice and overlap models appeared to fit as well or better at the individual level than at the group level and the all-or-none model to fit worse. In the present study, probability correct was fit even better by the all-or-none model than in the group study and replicated the result of being better here than the overlap and choice models. Individuals and the group were consistent in their sensory confusions as represented by similarity parameters in the choice and overlap models but differed in their response biases. A simple measure of physical similarity explained 50% of the variance of the similarity structure in the confusion data.     

Value-based decision-making battery: A Bayesian adaptive approach to assess impulsive and risky behavior

Using simple mathematical models of choice behavior, we present a Bayesian adaptive algorithm to assess measures of impulsive and risky decision making. Practically, these measures are characterized by discounting rates and are used to classify individuals or population groups, to distinguish unhealthy behavior, and to predict developmental courses. However, a constant demand for improved tools to assess these constructs remains unanswered. The algorithm is based on trial-by-trial observations. At each step, a choice is made between immediate (certain) and delayed (risky) options. Then the current parameter estimates are updated by the likelihood of observing the choice, and the next offers are provided from the indifference point, so that they will acquire the most informative data based on the current parameter estimates. The procedure continues for a certain number of trials in order to reach a stable estimation. The algorithm is discussed in detail for the delay discounting case, and results from decision making under risk for gains, losses, and mixed prospects are also provided. Simulated experiments using prescribed parameter values were performed to justify the algorithm in terms of the reproducibility of its parameters for individual assessments, and to test the reliability of the estimation procedure in a group-level analysis. The algorithm was implemented as an experimental battery to measure temporal and probability discounting rates together with loss aversion, and was tested on a healthy participant sample.     

Estimating latent distributions in recurrent choice data

This paper introduces a flexible class of stochastic mixture models for the analysis and interpretation of individual differences in recurrent choice and other types of count data. These choice models are derived by specifying elements of the choice process at the individual level. Probability distributions are introduced to describe variations in the choice process among individuals and to obtain a representation of the aggregate choice behavior. Due to the explicit consideration of random effect sources, the choice models are parsimonious and readily interpretable. An easy to implement EM algorithm is presented for parameter estimation. Two applications illustrate the proposed approach.     

Levels of aggregation and the generalized matching law.

Concurrent operant behavior is often described using molar response tendencies measured over many sessions. In contrast, other models relate the allocation of behavior to momentary or molecular processes that operate over shorter periods. In this article, a formal model of the statistical properties of molecular behavioral matching is developed. Molar matching is the result when many individual bouts of responding are aggregated. However, behavioral allocation at the molecular level will differ from the molar result. The model indicates that aggregation effects should be controlled in analyses of matching and that the comparison of molecular theories of concurrent operant behavior with molecular models of matching performance affords ideographic analyses of choice behavior.     

Individual and group modelling of aesthetic judgment strategies

Individual differences in aesthetic judgments were investigated by comparing quantitative group and individual performance models of the judgment processes. Aesthetic judgments of beauty over novel, formal, graphic patterns were collected from 34 non‐artist college students using a two‐step ranking‐rating procedure. Their judgment processes were individually modelled using Judgment Analysis. The participants showed noted individual differences. Certain features of the stimulus material, which were considered to contribute to the picture's beauty by one participant, were used in an opposing fashion by another. A group model was derived based on the average ratings of the patterns' beauty. It was concluded that the group model was not an adequate representation of the present data, whereas the data revealed systematic judgment processes at the individual subject level.     

Individual differences in impulsive choice and timing in rats

Individual differences in impulsive choice behavior have been linked to a variety of behavioral problems including substance abuse, smoking, gambling, and poor financial decision-making. Given the potential importance of individual differences in impulsive choice as a predictor of behavioral problems, the present study sought to measure the extent of individual differences in a normal sample of hooded Lister rats. Three experiments utilized variations of a delay discounting task to measure the degree of variation in impulsive choice behavior across individual rats. The individual differences accounted for 22-55% of the variance in choice behavior across the three experiments. In Experiments 2 and 3, the individual differences were still apparent when behavior was measured across multiple choice points. Large individual differences in the rate of responding, and modest individual differences in timing of responding were also observed during occasional peak trials. The individual differences in timing and rate, however, did not correlate consistently with individual differences in choice behavior. This suggests that a variety of factors may affect choice behavior, response rate, and response timing.     

Structural equation modeling of paired-comparison and ranking data

L. L. Thurstone's (1927) model provides a powerful framework for modeling individual differences in choice behavior. An overview of Thurstonian models for comparative data is provided, including the classical Case V and Case III models as well as more general choice models with unrestricted and factor-analytic covariance structures. A flow chart summarizes the model selection process. The authors show how to embed these models within a more familiar structural equation modeling (SEM) framework. The different special cases of Thurstone's model can be estimated with a popular SEM statistical package, including factor analysis models for paired comparisons and rankings. Only minor modifications are needed to accommodate both types of data. As a result, complex models for comparative judgments can be both estimated and tested efficiently.     

Process traces of binary choices: Evidence for selective and adaptive decision heuristics

While previous research has shown that human decision processes adjust to the characteristics of choice situations and task demands, the present study examined whether information processing would even adapt to the characteristics of an individual choice pair. By a process tracing method it was found that the amount of inspected information systematically depended upon the particular choice pair. In order to specify the selective and adaptive information processing, criterion-dependent choice models were introduced. These models postulate that information processing continues until the accumulated amount by which one alternative is better than the other reaches or exceeds a certain criterion. These models are strongly supported by the empirical data of the present study. Deviations between model predictions and observed data are explained in terms of fluctuating feature values of the choice alternatives.     

Cognitive models of risky choice: parameter stability and predictive accuracy of prospect theory

In the behavioral sciences, a popular approach to describe and predict behavior is cognitive modeling with adjustable parameters (i.e., which can be fitted to data). Modeling with adjustable parameters allows, among other things, measuring differences between people. At the same time, parameter estimation also bears the risk of overfitting. Are individual differences as measured by model parameters stable enough to improve the ability to predict behavior as compared to modeling without adjustable parameters? We examined this issue in cumulative prospect theory (CPT), arguably the most widely used framework to model decisions under risk. Specifically, we examined (a) the temporal stability of CPT’s parameters; and (b) how well different implementations of CPT, varying in the number of adjustable parameters, predict individual choice relative to models with no adjustable parameters (such as CPT with fixed parameters, expected value theory, and various heuristics). We presented participants with risky choice problems and fitted CPT to each individual’s choices in two separate sessions (which were 1 week apart). All parameters were correlated across time, in particular when using a simple implementation of CPT. CPT allowing for individual variability in parameter values predicted individual choice better than CPT with fixed parameters, expected value theory, and the heuristics. CPT’s parameters thus seem to pick up stable individual differences that need to be considered when predicting risky choice.     

A marginal maximum likelihood method for the vector threshold model to analyze dichotomous choice data

Abstract: At least two types of models, the vector model and the unfolding model can be used for the analysis of dichotomous choice data taken from, for example, the pick any/ n method. The previous vector threshold models have a difficulty with estimation of the nuisance parameters such as the individual vectors and thresholds. This paper proposes a new probabilistic vector threshold model, where, unlike the former vector models, the angle that defines an individual vector is a random variable, and where the marginal maximum likelihood estimation method using the expectation-maximization algorithm is adopted to avoid incidental parameters. The paper also attempts to discuss which of the two models is more appropriate to account for dichotomous choice data. Two sets of dichotomous choice data are analyzed by the model.     

Modeling individual differences in cognition

Many evaluations of cognitive models rely on data that have been averaged or aggregated across all experimental subjects, and so fail to consider the possibility of important individual differences between subjects. Other evaluations are done at the single-subject level, and so fail to benefit from the reduction of noise that data averaging or aggregation potentially provides. To overcome these weaknesses, we have developed a general approach to modeling individual differences using families of cognitive models in which different groups of subjects are identified as having different psychological behavior. Separate models with separate parameterizations are applied to each group of subjects, and Bayesian model selection is used to determine the appropriate number of groups. We evaluate this individual differences approach in a simulation study and show that it is superior in terms of the key modeling goals of prediction and understanding. We also provide two practical demonstrations of the approach, one using the ALCOVE model of category learning with data from four previously analyzed category learning experiments, the other using multidimensional scaling representational models with previously analyzed similarity data for colors. In both demonstrations, meaningful individual differences are found and the psychological models are able to account for this variation through interpretable differences in parameterization. The results highlight the potential of extending cognitive models to consider individual differences.     

Awareness and working memory in strategy adaptivity

To further the understanding of the mechanisms of strategy choice, in three experiments, we investigate the role of explicit awareness and working memory in strategy adaptivity. Experiment 1 provided correlational evidence that individual differences in strategy adaptivity to changing base rates are related to individual differences in awareness of those changes but appear not to be related to individual differences in working memory capacity. Experiment 2 replicated the role of awareness, and the results suggest that awareness at the time of the base-rate change, rather than afterwards, is related to increased strategy adaptivity. Experiment 3 measured working memory capacity using a different procedure and manipulated working memory load with a dual-task procedure; again, no apparent role of working memory capacity in strategy adaptivity was found. This juxtaposition of findings presents a challenge for existing models of strategy choice.     

Working memory capacity modulates task performance but has little influence on task choice

Variation in the ability to maintain internal goals while resolving competition from multiple information streams has been related to individual differences in working memory capacity (WMC). In a multitask environment, task choice and task performance are influenced by internal goals, prior behavior within the environment, and the availability of relevant and irrelevant information in the environment. Using the voluntary task-switching procedure, task performance, as measured by switch costs, was related to WMC, but only at short preparation intervals. Task choice processes were only weakly related to WMC. These findings are consistent with models of cognitive control that separate task choice processes from the processes of activating and maintaining task readiness. WMC is related to regulation of specific task parameters but not to choice processes integral to the coordination of multiple sources of information.     

The variance of d' estimates obtained in yes-no and two-interval forced choice procedures

In studies of detection and discrimination, data are often obtained in the form of a 2 x 2 matrix and then converted to an estimate of d' based on the assumptions that the underlying decision distributions are Gaussian and equal in variance. The statistical properties of the estimate of d', d' are well understood for data obtained using the yes-no procedure, but less effort has been devoted to the more commonly used two-interval forced choice (2IFC) procedure. The variance associated with d' is a function of true d' in both procedures, but for small values of true d' the variance of d' obtained using the 2IFC procedure is predicted to be less than the variance of d' obtained using yes-no; for large values of true d', the variance of d' obtained using the 2IFC procedure is predicted to be greater than the variance of d' from yes-no. These results follow from standard assumptions about the relationship between the two procedures. The present paper reviews the statistical properties of d' obtained using the two standard procedures and compares estimates of the variance of d' as a function of true d' with the variance observed in values of d' obtained with a 2IFC procedure.     

Structural equation modeling of approval voting data

The method of approval voting is a commonly used voting procedure in which each judge selects a subset of the alternatives. By postulating that the random utilities associated with the choice options in approval voting elections follow a multivariate normal distribution under the Thurstonian framework, Regenwetter, Ho, and Tsetlin (2007) attempted to integrate the normative theories and individual variabilities in modeling social behavior. However, their approach is limited to only three alternatives, due to computational intractability as the number of alternatives increases. In this article, we reparameterize extensions of their models under the structural equation modeling framework and propose the use of limited information methods for estimating model parameters. As a result, we are able to extend their previous approach to the analysis of approval voting data with any number of alternatives. Two applications are presented to illustrate the usefulness of such an approach.     

Comparison of basic assumptions embedded in learning models for experience-based decision making

The present study examined basic assumptions embedded in learning models for predicting behavior in decisions based on experience. In such decisions, the probabilities and payoffs are initially unknown and are learned from repeated choice with payoff feedback. We examined combinations of two rules for updating past experience with new payoff feedback and of two choice rule assumptions for mapping experience onto choices. The combination of these assumptions produced four classes of models that were systematically compared. Two methods were employed to evaluate the success of learning models for approximating players’ choices: One was based on estimating parameters from each person’s data to maximize the prediction of choices one step ahead, conditioned by the observed past history of feedback. The second was based on making a priori predictions for the entire sequence of choices using parameters estimated from a separate experiment. The results indicated the advantage of a class of models incorporating decay of previous experience, whereas the ranking of choice rules depended on the evaluation method used.     

Signed Orders, Choice Probabilities, and Linear Polytopes

Self-reflecting signed orders were proposed as an aid in estimating preference between subsets of items on the basis of limited information. The data of a signed order are preference comparisons that consider the desirability of excluding items from a subset as well as including items in a subset. This paper investigates signed orders within the setting of probabilistic choice and random utility theory. It focuses on linear signed order polytopes and their relationship to binary choice probabilities. The theory applies both to individual choice behavior and to group decision making. Copyright 2001 Academic Press.     

Development and validation of a questionnaire about the main variables affecting the individual investor's behavior in the Stock Exchange

Development and validation of a questionnaire about the main variables affecting the individual investor's behavior in the Stock Exchange. There is a considerable lack of information about the methodology usually used in most of the studies about individual investor's behavior. The studies reviewed do not show the method used in the selection of the items or the psychometric properties of the questionnaires. Because of the importance of investment in the Stock Exchange nowadays, it seems relevant to obtain a reliable instrument to understand individual investor's behavior in the Stock Exchange. Therefore, the goal of the present work is to validate a questionnaire about the main variables involved in individual investors' behavior in the Stock Exchange. Based on previous studies, we elaborated a questionnaire using the Delphi methodology with a group of experts. The internal consistency (Cronbach alpha=.934) and validity evidence of the questionnaire show that it may be an effective instrument and can be applied with some assurance.     

Measurement of Ability in Adaptive Learning and Assessment Systems when Learners Use On-Demand Hints

Adaptive learning and assessment systems support learners in acquiring knowledge and skills in a particular domain. The learners’ progress is monitored through them solving items matching their level and aiming at specific learning goals. Scaffolding and providing learners with hints are powerful tools in helping the learning process. One way of introducing hints is to make hint use the choice of the student. When the learner is certain of their response, they answer without hints, but if the learner is not certain or does not know how to approach the item they can request a hint. We develop measurement models for applications where such on-demand hints are available. Such models take into account that hint use may be informative of ability, but at the same time may be influenced by other individual characteristics. Two modeling strategies are considered: (1) The measurement model is based on a scoring rule for ability which includes both response accuracy and hint use. (2) The choice to use hints and response accuracy conditional on this choice are modeled jointly using Item Response Tree models. The properties of different models and their implications are discussed. An application to data from Duolingo, an adaptive language learning system, is presented. Here, the best model is the scoring-rule-based model with full credit for correct responses without hints, partial credit for correct responses with hints, and no credit for all incorrect responses. The second dimension in the model accounts for the individual differences in the tendency to use hints.