An automated test development of parallel tests from a seed test

Binary programming models are presented to generate parallel tests from an itembank. The parallel tests are created to match item for item an existing seed test and match user supplied taxonomic specifications. The taxonomic specifications may be either obtained from the seed test or from some other user requirement. An algorithm is presented along with computational results to indicate the overall efficiency of the process. Empirical findings based on an itembank for the Arithmetic Reasoning section of the Armed Services Vocational Aptitude Battery are given.The Office of Naval Research, Program in Cognitive Science, N00014-87-C-0696 partially supported the work of Douglas H. Jones. The Rutgers Research Resource Committee of the Graduate School of Management partially supported the work of Douglas H. Jones and Ing-Long Wu. A Thomas and Betts research fellowship partially supported the work of Ing-Long Wu. The Human Resources Laboratory, United States Air Force, partially supported the work of Ronald Armstrong. The authors benefited from conversations with Dr. Wayne Shore, Operational Technologies, San Antonio, Texas. The order of authors' names is alphabetical and denotes equal authorship.     

Robust interactive decision-analysis (RID): Behavioral results and implications

We have proposed a novel interactive procedure for performing decision analysis, called Robust Interactive Decision Analysis (RID), which permits a decision maker (DM) to voluntarily and interactively express strong (viz, sure) binary preferences for actions, partial decision functions, and full decision functions, and only imprecise probability and utility function assessments. These serve as INPUTS TO operators to prune the state probability space and decision space until an optimal choice strategy is obtained. The viability of the RID approach depends on a DM's ability to provide such information consistently and meaningfully. On a limited scale we experimentally investigate the behavioral implications of the RID method in order to ascertain its potential operational feasibility and viability. More specifically, we examine whether a DM can (1) express strong preferences between pairs of vectors of unconditional and conditional payoffs or utilities consistently; (2) provide imprecise (ordinal and interval) state probabilities that are individually as well as mutually consistent with the state probabilities imputed from the expressed strong preferences. The results show that a DM can provide strong individually and mutually consistent preference and ordinal probability information. Moreover, most individuals also appear to be able to provide interval probabilities that are individually and mutually consistent with their strong preference inputs. However, the several violations observed, our small sample size, and the limited scope of our investigation suggest that further experimentation is needed to determine whether and/or how such inputs should be elicited. Overall, the results indicate that the RID method is behaviorally viable.     

Psychological determinants of decision attitude

Decision attitude — an analog of risk attitude — is the propensity to make (or avoid making) a decision: in decision aversion, a person finds it more desirable to receive through fiat the better of two options than to have a choice between them; in decision seeking, the choice is more desirable, even though it can lead to nothing better than the best option. Both decision aversion and decision seeking were found in hypothetical scenarios. Experimental manipulations and subjects' justifications point to anticipated regret, fear of blame for poor outcomes, and desire for equitable distributions as sources of decision aversion. One source of decision seeking (for self) and decision aversion (when deciding for others) appears to be the desire for the self-determination of the affected parties. We consider the implications of our results for personal choice and public policy decisions.     

Stability criteria

Three approaches to the determination of behavioral stability were examined. In the first, a learning curve was fit to acquisition data (from Cumming and Schoenfeld, 1960), and the “experiment” stopped when the data approached sufficiently close to the theoretical asymptote. In the second, the data were analyzed for variability and linear and quadratic trend. In the third, the experiment was stopped when the magnitude of the daily changes in the data fell below a criterion. Accuracy was measured as deviation between the average value of the dependent variable when the experiment was stopped, and the average value over the last 100 sessions. The first approach was most accurate, but at the cost of requiring the most sessions and being the most difficult to apply. Both the second and third approaches provided acceptable criteria with a reasonable cost-accuracy tradeoff. The second approach permits a continuous adjustment of the criteria to accommodate the variability intrinsic in the experimental paradigm. The third, nomothetic, approach also takes into account the decreasing marginal utility of extended training sessions.     

Decisions,actions, and the appropriateness of confidence in knowledge

This research examines the appropriateness of confidence (i.e., subjective probability judgments) in knowledge associated with decisions and actions of social importance. One hundred and thirty seven participants completed a 50 item questionnaire assessing their knowledge of the two leading candidates in the 1988 presidential election in the U.S.A. Ninety one of the respondents completed the questionnaire one week prior to the election, whereas the other 46 completed the questionnaire on election day shortly after voting. After each item in the questionnaire, all respondents indicated whether or not the item content represented a reason why they voted (or intended to vote) for or against the candidate to whom the item referred. Within-person results indicated that, in comparison to items that were not cited as reasons for voting intentions or voting behavior, items endorsed as reasons were characterized by better accuracy and resolution, but worse overconfidence. Between groups, decision makers were significantly more accurate and better calibrated than those who had not made a decision between the candidates. Implications of inappropriate confidence on decision making effectiveness are discussed.     

Do medical decision analyses' largest gains grow from the smallest trees?

Some decision analysts recommend reserving their models for careful, complete study of complex problems. Others believe simple problem analyses provide the greatest gains. To clarify these issues, I tested: (1) whether simple published analyses compared to complex ones, produced results as non-intuitive; and (2) whether non-intuitiveness represented analytic gain (or equivalently, intuitive loss). Very limited data also examined (3) the gain from adding model intricacy for a given problem vs. selecting more complex problems for analysis. To assess two non-intuitiveness measures, 75 subjects (33 physicians and 42) non-physicians stated their intuitive preferences in 40 (22 published and 18 unpublished) medical dilemmas. For both physicians and non-physicians, simpler models (those with lees than 30 terminal branches in the decision tree) had about one third the non-intuitiveness of more complex ones. Three tests also supported the premise that analyses outperform intuition, therefore that on average the formers' non-intuitiveness reflects less analytic gain. In addition, for different models of the same problem, tree complexity did not correlate with gain. Thus, simpler trees may not generally gain less because they inadequately describe problems. Instead, simpler analyses may represent simpler problems, with more similar intuitive and formal solutions. If so, these findings may help us avoid some costly but unnecessary simple analyses.     

INTERACTIONS BETWEEN TEACHER GUIDANCE AND CONTINGENT ACCESS TO PLAY IN DEVELOPING PREACADEMIC SKILLS OF DEVIANT PRESCHOOL CHILDREN1

Token-mediated access to play and snacks was made contingent on completion of academic tasks in the Baseline Experiment. This contingency produced stable completion rates that were subsequently doubled, and then tripled, for four deviant children in a special preschool. A reversal design demonstrated that the contingency was functional in maintaining the children's rates of task completion. The Guidance Experiment examined the role of a social event, teacher guidance, in the acquisition of task-completion skills, in a multiple-baseline-across-tasks design (with reversals). The analysis demonstrated that teacher guidance was an important supplement to the token-mediated contingency in establishing significant increases in task completions for a second group of three deviant children in the special class. The importance of teacher guidance was related to the difficulty level of the children's tasks.     

Decision Support System and Multi‐Criteria Decision Aid: A State of the Art and Perspectives

The aim of this paper is to highlight the role of the Decision Support System within the field of multi‐criteria decision aid (MCDA). The MCDA tools have been incorporated into systems to create Multi‐Criteria Decision Support Systems (MCDSSs). In our literature review, we noticed that more than 100 papers have been written over a 20‐year period in which MCDSS was used as a decision‐making tool. The present paper describes some real applications of MCDSS in different fields, harmoniously combined with decision‐making methods such as analytic hierarchy process, Utility Additive, and Goal Programming. The present study proposes an integrative MCDSS evaluation through guidance on the tools most useful for a specific user with a particular decision problem. Copyright © 2014 John Wiley & Sons, Ltd.     

Discerning Mouse Trajectory Features With the Drift Diffusion Model

Mouse tracking, a new action-based measure of behavior, has advanced theories of decision making with the notion that cognitive and social decision making is fundamentally dynamic. Implicit in this theory is that people's decision strategies, such as discounting delayed rewards, are stable over task design and that mouse trajectory features correspond to specific segments of decision making. By applying the hierarchical drift diffusion model and the Bayesian delay discounting model, we tested these assumptions. Specifically, we investigated the extent to which the “mouse-tracking” design of decision-making tasks (delay discounting task, DDT and stop-signal task, SST) deviate from the standard “keypress” design of decision making tasks. We found remarkable agreement in delay discounting rates (intertemporal impatience) obtained in the keypress and mouse-tracking versions of DDT (ρ = 0.90) even though these tasks were given about 1 week apart. Rates of evidence accumulation converged well in the two versions (DDT, ρ = .86; SST, ρ = .55). Omission/commission error in SST showed high agreement (ρ = .42, ρ = .53). Mouse-motion features such as maximum velocity and AUC (area under the curve) correlated well with nondecision time (ρ = −.42) and boundary separation (ρ = .44)—the amount of information needed to accumulate prior to making a response. These results indicate that the response time (RT) and motion-based decision tasks converge well at a fundamental level, and that mouse-tracking features such as AUC and maximum velocity do indicate the degree of decision conflict and impulsivity.