An Optimized Design of Choice Experiments: A New Approach for Studying Decision Behavior in Choice Task Experiments

In this paper, we present a new approach for the optimal experimental design problem of generating diagnostic choice tasks, where the respondent's decision strategy can be unambiguously deduced from the observed choice. In this new approach, we applied a genetic algorithm that creates a one‐to‐one correspondence between a set of predefined decision strategies and the alternatives of the choice task; it also manipulates the characteristics of the choice tasks. In addition, this new approach takes into account the measurement errors that can occur when the preferences of the decision makers are being measured. The proposed genetic algorithm is capable of generating diagnostic choice tasks even when the search space of possible choice tasks is very large. As proof‐of‐concept, we used this novel approach to generate respondent‐specific choice tasks with either low or high context‐based complexity that we operationalize by the similarity of alternatives and the conflict between alternatives. We find in an experiment that an increase in the similarity of the alternatives and an increase in the number of conflicts within the choice task lead to an increased use of non‐compensatory strategies and a decreased use of compensatory decision strategies. In contrast, the size of the choice tasks, measured by the number of attributes and alternatives, only weakly influences the strategy selection. Copyright © 2014 John Wiley & Sons, Ltd.     

Factors influencing the use of internal summary evaluations versus external information in choice

Decision makers cope with more demanding tasks by shifting their cognitive strategies, balancing effort expenditure against the desire to produce an accurate response. In choice tasks, one method for reducing effort while simultaneously maintaining accuracy is to shift from a reliance on information obtained from the external environment to that retrieved from memory, particularly summary evaluations (i.e. relatively stable evaluative reactions to the overall attractiveness of individual alternatives) acquired from previous decision-making experience. We examined four factors that should all encourage decision makers to rely to some extent on internal summary evaluations rather than external information search: (1) increased external search costs, (2) greater time pressure, (3) increased incentives to make an accurate choice, and (4) increased levels of prior experience with the choice context. Sixty experimental subjects chose their preferred alternative from a set of women's magazines. Decision processes were inferred from verbal protocols, computer-generated search records, and decision times. Only increased external search costs led to a greater reliance on summary evaluations. Although increased time pressure, increased incentives, and increased experience with the context did lead to changes in search and evaluation processes, contrary to initial expectations decision makers did not shift to a greater reliance on summary evaluations. These results suggest that while decision makers do sometimes use summary evaluations as a substitute for external search, this is only one of several mechanisms available for effort reduction. Understanding when and where decision makers rely upon summary evaluations probably requires a broader conceptualization of knowledge structures, particularly a consideration of the knowledge that decision makers possess about different choice strategies and their effectiveness.     

Component processes of phased decision strategies

It is often assumed that decision makers apply mixtures of strategies including heuristic processes in complex choice tasks. In support of this assumption the component processes involved in phased strategies were investigated. Expectations for an experimental investigation were generated by simulations within the effort-quality framework. Effort-quality trade-off functions for tasks of different complexity favor compensatory dimension-based strategies for binary choice. For multialternative choice, a combination of initial dimension-based elimination procedures followed by alternative-based final processing would be expected. These expectations were tested in two experiments. While choosing between two, four, or eight alternatives, subjects could successively retrieve parts of the available information. Depending on their menu selection, either all features of one alternative were displayed together (alternative-based retrieval), or all features on one dimension were displayed together (dimension-based retrieval). Vincent averaging was used to analyze the retrieval sequences with respect to their stationarity. While the stationary sequences for binary choice are compatible with single processing strategies, non-stationarity of the sequences obtained for multialternative choice indicates the use of phased strategies. In particular, non-stationarity due to decreasing dimension-based retrievals over time suggests that subjects first applied dimensional elimination and later derived their choices by more thorough, alternative-based processing of the remaining alternatives.     

The Effect of Decision Strategy on Deciding to Defer Choice

Recent research has examined consumer decision making when the option of not choosing any of the alternatives is also provided. The findings from this research suggest that the decision to defer choice is sensitive to the uncertainty of choosing the most preferred option from the set of alternatives provided. Building on this research, the author tests whether the decision to defer choice is also influenced by task variables that influence decision uncertainty. In the first experiment, this proposition is tested for choice problems in which information on three relatively equally attractive alternatives is presented either sequentially or simultaneously. As predicted, the preference for the defer-choice option was greater when the three alternatives were presented simultaneously. A second study forced subjects into using one of four decision strategies in order to choose between two non-dominated alternatives. The preference for the no-choice option was found to be higher when the rule required explicit attribute tradeoffs and lower when it simplified choice. These results suggest that choice uncertainty is influenced by the decision strategy used to determine the preference among alternatives. We conclude with a discussion of the implications of the results for marketers' communication strategies.     

Individual differences in the use of simplification strategies in a complex decision-making task

Decision time results were used to assess the strategies that 90 college undergraduates used in a complex decision-making task. Trend analyses revealed that the functions relating choice time to the number of choice alternatives in a set and the number of attributes comprising those alternatives contained linear (increasing) components. In addition, for a portion of the subjects, there was a quadratic effect of the number of attributes available to the decision maker on choice time, suggesting that these subjects adopted simplification strategies at high levels of task complexity. Reliable individual differences in these trend components were observed, consistent with individual differences in motivation and/or processing capacities. These individual differences were included in an information-processing model of decision behavior that captured the choice time data observed in this study. Subjects' ratings of apartments were used as a basis to assess the extent to which the use of simplification strategies resulted in preference reversals. Contrary to expectation, subjects whose choice times contained quadratic components demonstrated fewer preference reversals at high levels of information load.     

Probabilistic Subset Conjunction

The authors introduce subset conjunction as a classification rule by which an acceptable alternative must satisfy some minimum number of criteria. The rule subsumes conjunctive and disjunctive decision strategies as special cases. Subset conjunction can be represented in a binary-response model, for example, in a logistic regression, using only main effects or only interaction effects. This results in a confounding of the main and interaction effects when there is little or no response error. With greater response error, a logistic regression, even if it gives a good fit to data, can produce parameter estimates that do not reflect the underlying decision process. The authors propose a model in which the binary classification of alternatives into acceptable/unacceptable categories is based on a probabilistic implementation of a subset-conjunctive process. The satisfaction of decision criteria biases the odds toward one outcome or the other. The authors then describe a two-stage choice model in which a (possibly large) set of alternatives is first reduced using a subset-conjunctive rule, after which an alternative is selected from this reduced set of items. They describe methods for estimating the unobserved consideration probabilities from classification and choice data, and illustrate the use of the models for cancer diagnosis and consumer choice. They report the results of simulations investigating estimation accuracy, incidence of local optima, and model fit. The authors thank the Editor, the Associate Editor, and three anonymous reviewers for their constructive suggestions, and also thank Asim Ansari and Raghuram Iyengar for their helpful comments. They also thank Sawtooth Software, McKinsey and Company, and Intelliquest for providing the PC choice data, and the University of Wisconsin for making the breast-cancer data available at the machine learning archives.     

Attention and attribute overlap in preferential choice

Attributes that are common, or overlapping, across alternatives in two-alternative forced preferential choice tasks are often non-diagnostic. In many settings, attending to and evaluating these attributes does not help the decision maker determine which of the available alternatives is the most desirable. For this reason, many existing behavioural theories propose that decision makers ignore common attributes while deliberating. Across six experiments, we find that decision makers do direct their attention selectively and ignore attributes that are not present in or associated with either of the available alternatives. However, they are as likely to attend to common attributes as they are to attend to attributes that are unique to a single alternative. These results suggest the need for novel theories of attention in preferential choice.     

The three-class ideal observer for univariate normal data: Decision variable and ROC surface properties

Although a fully general extension of ROC analysis to classification tasks with more than two classes has yet to be developed, the potential benefits to be gained from a practical performance evaluation methodology for classification tasks with three classes have motivated a number of research groups to propose methods based on constrained or simplified observer or data models. Here we consider an ideal observer in a task with underlying data drawn from three univariate normal distributions. We investigate the behavior of the resulting ideal observer’s decision variables and ROC surface. In particular, we show that the pair of ideal observer decision variables is constrained to a parametric curve in two-dimensional likelihood ratio space, and that the decision boundary line segments used by the ideal observer can intersect this curve in at most six places. From this, we further show that the resulting ROC surface has at most four degrees of freedom at any point, and not the five that would be required, in general, for a surface in a six-dimensional space to be non-degenerate. In light of the difficulties we have previously pointed out in generalizing the well-known area under the ROC curve performance metric to tasks with three or more classes, the problem of developing a suitable and fully general performance metric for classification tasks with three or more classes remains unsolved.     

The Influence of Decision Aids on Choice Strategies: An Experimental Analysis of the Role of Cognitive Effort

This paper examines the role of computer-based decision aids in reducing cognitive effort and therefore influencing strategy selection. It extends and complements the works reported in the behavioral decision theory literature on the role of effort and accuracy in choice tasks. The central proposition of this paper is that specific features can be incorporated within a set of decision aids that will alter the effort required to implement a particular choice strategy relative to other strategies, and that this will influence strategy selection by the decision maker. In a laboratory experiment, using a repeated measures design, 48 subjects performed a preferential choice task using different decision aids. Subjects were given different levels of support to reduce the cognitive effort associated with different preferential choice strategies. In particular, the decision aids provided varying levels of support for the processing associated with either elimination by aspects or additive difference strategies. The study examined how information processing strategies were influenced by the decision aids. The results show that decision aids which reduce the effort associated with the elimination by aspects strategy induce behaviors associated with elimination by aspects. More importantly, there was an interaction effect caused by the distinct behavior of the group which was provided support for additive difference but not for elimination by aspects. This group processed the information about available choices in a way that is consistent with an additive difference strategy. These findings indicate that a decision aid can induce additive processing by altering the relative effort requirements of different choice strategies.     

Analyzing information search patterns to test the use of a two-phased decision strategy

This study compares three procedures, Payne's index, Van Raaij's index and the single Markov model, to test the hypothesis that subjects apply two decision strategies successively in making a choice. The information search patterns were gathered for 72 subjects by means of an information display board. The decision tasks varied in complexity (3, 5, 8 alternatives and 4, 6, 8 attributes). The number of tests which resulted in a significant difference (p < 0.05) were respectively, six out of eighteen t-tests for Payne's index, fourteen out of eighteen t-tests regarding Van Raaij's index, and eight out of nine Chi-square tests for the single Markov model. The theoretical considerations as well as the results suggest that Payne's index is the least sensitive, and that Van Raaij's index, compared to the Markov model, is less sensitive to detect a difference in strategies used.     

选项数量和属性重要性对决策中信息加工的影响

本研究采用决策过程研究方法中的信息板技术,以现实生活中的职业选择作为决策任务,考察了选项数量和选项属性的重要性对决策过程中信息加工的影响方式。结果表明:选项数量对决策过程的信息搜索深度、搜索模式和策略的补偿性产生了影响;选项属性的重要性并不影响决策者对属性的查看时间,但影响了决策者对属性的点击次数。     

Choice among two and three alternatives

Although choice between two alternatives has been widely researched, fewer studies have examined choice across multiple (more than two) alternatives. Past models of choice behavior predict that the number of alternatives should not affect relative response allocation, but more recent research has found violations of this principle. Five pigeons were presented with three concurrently scheduled alternatives. Relative reinforcement rates across these alternatives were assigned 9:3:1. In some conditions three keys were available; in others, only two keys were available. The number of available alternatives did not affect relative response rates for pairs of alternatives; there were no significant differences in behavior between the two and three key conditions. For two birds in the three‐alternative conditions and three birds in the two‐alternative conditions, preference was more extreme for the pair of alternatives with the lower overall pairwise reinforcer rate (3:1) than the pair with higher overall reinforcer rate (9:3). However, when responding during the changeover was removed three birds showed the opposite pattern in the three‐alternative conditions; preference was more extreme for the pair of alternatives with the higher overall reinforcer rate. These findings differ from past research and do not support established theories of choice behavior.     

The role of attitudes in memory-based decision making

Frequently, considerable knowledge of the attributes of decision alternatives is available in memory so as to permit a thoughtful and deliberate choice. However, in many instances, individuals neglect to use such knowledge and instead rely on "attitude-based" strategy to make a memory-based decision. The findings from two experiments suggest that as to the motivation to make a correct decision or the opportunity to use the available attribute knowledge decreases, the likelihood that attitudes will guide a memory-based decision increases. The findings illustrate the functional role attitudes play in guiding decisions and behavior. By providing a ready means of evaluating choice alternatives, attitudes enable an individual to make a decision relatively quickly and effortlessly.     

Children’s choice strategies: the effects of age and task demands

Two experiments examined the effect of age and cognitive demands on children’s choice strategies. Children aged 8–9 and 12–13 years were asked to choose among either two or four products that differed in several attributes of varying importance to them. Choice tasks were designed to differentiate between the lexicographic and the equal-weighting strategies that the children used. The results showed that older children use the two strategies more appropriately than younger children do. The choice of strategies reflects the demands of the task. All children were more likely to use the strategies correctly when choosing between two alternatives and tended to rely on the relatively less cognitive-demanding lexicographic strategy to choose among four alternatives. However, they were less likely to employ the lexicographic strategy in simpler tasks and when its use was difficult to justify, indicating an adaptive use.     

A Brief History of Research on Phantom Alternatives: Evidence for Seven Empirical Generalizations About Phantoms

Phantom alternatives (unavailable options) can appear in many common decision tasks and can influence a wide variety of everyday behaviors. Although the term phantom is of recent vintage, researchers have investigated psychological reactions to unavailable alternatives for almost 60 years. In this article, we review 13 domains of research on unavailable alternatives to identify seven empirical generalizations about phantoms, These generalizations call into questions some basic assumptions of formal choice theory and suggest new research directions for studying phantom options.     

Anticipations of Effort and Accuracy in Multiattribute Choice

Research on multiattribute choice strategies suggests that decision makers adapt to variations in task features by switching decision strategies. One perspective is that strategy selection is determined by a trade-off between the goals of minimizing effort and maximizing accuracy. However, since strategy selection is based on anticipated effort and accuracy, empirical evidence is needed on the nature of these anticipations. An experiment examines how well decision makers anticipate the influence of two particular task features, information display organization and number of alternatives, on effort and accuracy. Subjects chose the best alternative from sets of loan applications described by six relevant attributes. Prior to each choice, they prospectively judged the effort required and the accuracy that would result. Results suggest that anticipated consequences deviate substantially from experienced consequences for both effort and accuracy. Neither task experience nor explicit feedback brought anticipations and experiences closer together. These findings underscore the importance of anticipation judgments in multiattribute choice.     

Replicated alternatives and the role of confusion,chasing, and regret in decisions from experience

The current paper explores choice among alternatives that can be classified into distinct classes. All the members of a particular class were ‘replicated alternatives’: they promised the same payoff distribution. Information to decision makers was limited to feedback concerning the realized (obtained and foregone) payoffs. Experiment 1 demonstrates that increasing the number of replicated alternatives creates confusion (which facilitates random choice) and changes the implications of the tendency to chase recent returns (i.e., select the alternative with the best recent outcomes). This effect, termed ‘confused chasing,’ facilitates risk seeking even when this behavior impairs expected earnings. Experiment 2 reveals that increasing the number of replicated alternatives can reduce (but does not eliminate) the tendency to underweight rare events. Experiment 3 shows that the relative importance of chasing and confusion is sensitive to the likelihood of realizing lower payoffs than the forgone payoffs. The main results are summarized with a simple model assuming that payoff sensitivity decreases with experienced regret. Copyright © 2006 John Wiley & Sons, Ltd.     

Stopping rule use during information search in design problems

Information search is critical in most decision-making tasks. An important aspect of information search is the stopping rule used by the decision maker to terminate information acquisition. Decision-making problems may be usefully decomposed into design problems and choice problems. The distinction is critical because the goals of stopping behavior in the two types of problems are quite different. In design problems, the focus is on the sufficiency of information obtained for problem structuring and generating alternatives, while choice problems focus on convergence toward a solution. While previous research has studied stopping behavior in choice problems, the present research is concerned with stopping rule use during information search in design problems. To build theory in this area, we conducted two experiments. In the first, we presented professional systems analysts with an information search problem in a systems development context and asked them to gather information for a proposed system. Protocols of the search sessions were analyzed, and stopping rules used and information gathered by the analysts were identified. Results indicated that the use of certain stopping rules resulted in greater quantity and quality of information gathered. Additionally, stopping rule use, rather than analyst experience, accounted for the quantity and quality of information elicited. Finally, stopping rule use differed between more experienced and less experienced analysts. To apply our findings about stopping rules from the first experiment, we performed a second experiment in which we investigated stopping rule use in a consumer information search task on the world wide web. Results indicated the use of the same stopping rules as in the first experiment, although in different proportions. Implications for information search theory and practice are discussed.     

Multi-attribute decision support and complexity: An evaluation and process analysis of aided versus unaided decision making

The present study addresses the effectiveness of a computerized decision aid (DECAID, Pitz 1987) based on the Multi-Attribute Utility Model (MAU). The effectiveness of DECAID was investigated for personnel selection problems varying in complexity (number of alternatives and number of attributes). Results show that the decision aid was most successful for problems of medium complexity. DECAID significantly affected the thoroughness of the decision, preference formation, and the perceived difficulty of the decision problem for medium complex problems. This was not the case for highly complex problems. Next, a process analysis is presented of unaided decision making focusing on the selection and evaluation of information. This process analysis of verbal protocols was based on the same decision problems as the first study. Results indicate that problem complexity had a significant impact on the decision process. Increasing complexity resulted in a less thorough evaluation, and selective information processing. Subjects tended to focus on one or two most promising alternatives. Furthermore, evaluation strategies diverged from those used in DECAID in terms of standard of comparison. The paper concludes with implications for the design of decision aids.