A systematic review of algorithm aversion in augmented decision making

Despite abundant literature theorizing societal implications of algorithmic decision making, relatively little is known about the conditions that lead to the acceptance or rejection of algorithmically generated insights by individual users of decision aids. More specifically, recent findings of algorithm aversion—the reluctance of human forecasters to use superior but imperfect algorithms—raise questions about whether joint human-algorithm decision making is feasible in practice. In this paper, we systematically review the topic of algorithm aversion as it appears in 61 peer-reviewed articles between 1950 and 2018 and follow its conceptual trail across disciplines. We categorize and report on the proposed causes and solutions of algorithm aversion in five themes: expectations and expertise, decision autonomy, incentivization, cognitive compatibility, and divergent rationalities. Although each of the presented themes addresses distinct features of an algorithmic decision aid, human users of the decision aid, and/or the decision making environment, apparent interdependencies are highlighted. We conclude that resolving algorithm aversion requires an updated research program with an emphasis on theory integration. We provide a number of empirical questions that can be immediately carried forth by the behavioral decision making community.     

道德困境研究的范式沿革及其理论价值

通过评述道德困境研究范式的发展过程, 系统阐释了经典两难法、加工分离法、CNI模型法和CAN算法的优缺点和理论价值。后来的研究范式均在一定程度上克服了之前研究范式的局限。加工分离法克服了经典两难法的加工纯粹性假设等局限, CNI模型法在加工分离法基础上进一步分离了道德困境决策的多种心理过程, CAN算法则修正了CNI模型法的序列加工的不恰当预设。研究范式的沿革启示研究者综合应用新方法来解决研究争议和重新审视以往道德理论, 合理应用新方法来探索其他具有潜在冲突性的研究议题。总之, 本文为道德困境及相关研究提供了方法学参考。     

Gaussian variational estimation for multidimensional item response theory

Multidimensional item response theory (MIRT) is widely used in assessment and evaluation of educational and psychological tests. It models the individual response patterns by specifying a functional relationship between individuals' multiple latent traits and their responses to test items. One major challenge in parameter estimation in MIRT is that the likelihood involves intractable multidimensional integrals due to the latent variable structure. Various methods have been proposed that involve either direct numerical approximations to the integrals or Monte Carlo simulations. However, these methods are known to be computationally demanding in high dimensions and rely on sampling data points from a posterior distribution. We propose a new Gaussian variational expectation--maximization (GVEM) algorithm which adopts variational inference to approximate the intractable marginal likelihood by a computationally feasible lower bound. In addition, the proposed algorithm can be applied to assess the dimensionality of the latent traits in an exploratory analysis. Simulation studies are conducted to demonstrate the computational efficiency and estimation precision of the new GVEM algorithm compared to the popular alternative Metropolis–Hastings Robbins–Monro algorithm. In addition, theoretical results are presented to establish the consistency of the estimator from the new GVEM algorithm.     

Numerical approximation of the observed information matrix with Oakes' identity

An efficient and accurate numerical approximation methodology useful for obtaining the observed information matrix and subsequent asymptotic covariance matrix when fitting models with the EM algorithm is presented. The numerical approximation approach is compared to existing algorithms intended for the same purpose, and the computational benefits and accuracy of this new approach are highlighted. Instructive and real-world examples are included to demonstrate the methodology concretely, properties of the estimator are discussed in detail, and a Monte Carlo simulation study is included to investigate the behaviour of a multi-parameter item response theory model using three competing finite-difference algorithms.     

Trade‐off ranking method for multi‐criteria decision analysis

Multi‐criteria decision analysis presumes trade‐off between different criteria. As a result, the optimal solution is not unique and can be represented by the Pareto frontier in the objective space. Each Pareto solution is a compromise between different objectives. Despite a limited number of Pareto optimal solutions, the decision‐maker eventually has to choose only one option. Such a choice has to be made with the use of additional preferences not included in the original formulation of the optimization problem. The paper represents a new approach to an automatic ranking that can help the decision‐maker. In contrast to the other methodologies, the proposed method is based on the minimization of trade‐off between different Pareto solutions. To be realized, the approach presumes the existence of a well‐distributed Pareto set representing the entire Pareto frontier. In the paper, such a set is generated with the use of the directed search domain algorithm. The method is applied to a number of test cases and compared against two existing alternative approaches.     

Methodology to select solutions for multiobjective optimization problems: Weighted stress function method

The weighted stress function method is proposed here as a new way of identifying the best solution from a set of nondominated solutions according to the decision maker's preferences, expressed in terms of weights. The method was tested using several benchmark problems from the literature, and the results obtained were compared with those of other methods, namely, the reference point evolutionary multiobjective optimization (EMO), the weighted Tchebycheff metric, and a goal programming method. The weighted stress function method can be seen to exhibit a more direct correspondence between the weights set by the decision maker and the final solutions obtained than the other methods tested.     

改进的MCMC算法及其在估计IRT模型参数中的应用

本文首先简要的阐述了MCMC算法的思想及在IRT参数估计中的操作过程;其次,针对该算法存在的一些问题,提出相应的改进建议;然后,分别运用传统的和改进型的MCMC算法进行模拟数据分析和比较,结果显示新的方法表现更好;最后总结新方法的优点所在,并指出下一步的研究方向。     

Bayesian Analysis of Nonlinear Structural Equation Models with Nonignorable Missing Data

A Bayesian approach is developed for analyzing nonlinear structural equation models with nonignorable missing data. The nonignorable missingness mechanism is specified by a logistic regression model. A hybrid algorithm that combines the Gibbs sampler and the Metropolis–Hastings algorithm is used to produce the joint Bayesian estimates of structural parameters, latent variables, parameters in the nonignorable missing model, as well as their standard errors estimates. A goodness-of-fit statistic for assessing the plausibility of the posited nonlinear structural equation model is introduced, and a procedure for computing the Bayes factor for model comparison is developed via path sampling. Results obtained with respect to different missing data models, and different prior inputs are compared via simulation studies. In particular, it is shown that in the presence of nonignorable missing data, results obtained by the proposed method with a nonignorable missing data model are significantly better than those that are obtained under the missing at random assumption. A real example is presented to illustrate the newly developed Bayesian methodologies. This research is fully supported by a grant (CUHK 4243/03H) from the Research Grant Council of the Hong Kong Special Administration Region. The authors are thankful to the editor and reviewers for valuable comments for improving the paper, and also to ICPSR and the relevant funding agency for allowing the use of the data. Requests for reprints should be sent to Professor S.Y. Lee, Department of Statistics, The Chinese University of Hong Kong, Shatin, N.T., Hong Kong.     

Bayesian adaptive estimation: The next dimension

We propose a new psychometric model for two-dimensional stimuli, such as color differences, based on parameterizing the threshold of a one-dimensional psychometric function as an ellipse. The Ψ Bayesian adaptive estimation method applied to this model yields trials that vary in multiple stimulus dimensions simultaneously. Simulations indicate that this new procedure can be much more efficient than the more conventional procedure of estimating the psychometric function on one-dimensional lines independently, requiring only one-fourth or less the number of trials for equivalent performance in typical situations. In a real psychophysical experiment with a yes-no task, as few as 22 trials per estimated threshold ellipse were enough to consistently demonstrate certain color appearance phenomena. We discuss the practical implications of the multidimensional adaptation. In order to make the application of the model practical, we present two significantly faster algorithms for running the Ψ method: a discretized algorithm utilizing the Fast Fourier Transform for better scaling with the sampling rates and a Monte Carlo particle filter algorithm that should be able to scale into even more dimensions.