Reduction of pairwise comparisons in decision making via a duality approach

Although pairwise comparisons have been seen by many as an effective and intuitive way for eliciting qualitative data for multi‐criteria decision making problems, a major drawback is that the number of the required comparisons increases quadratically with the number of the entities to be compared. Thus, often even data for medium size decision problems may be impractical to be elicited via pairwise comparisons. The more the comparisons are, the higher is the likelihood that the decision maker will introduce erroneous data. This paper introduces a dual formulation to a given multi‐criteria decision making problem, which can significantly alleviate the previous problems. Some theoretical results establish that this is possible when the number of alternatives is greater than the number of decision criteria plus one. Copyright © 1999 John Wiley & Sons, Ltd.     

Pairwise multiple comparisons: A model comparison approach versus stepwise procedures

Researchers in the behavioural sciences have been presented with a host of pairwise multiple comparison procedures that attempt to obtain an optimal combination of Type I error control, power, and ease of application. However, these procedures share one important limitation: intransitive decisions. Moreover, they can be characterized as a piecemeal approach to the problem rather than a holistic approach. Dayton has recently proposed a new approach to pairwise multiple comparisons testing that eliminates intransitivity through a model selection procedure. The present study compared the model selection approach (and a protected version) with three powerful and easy‐to‐use stepwise multiple comparison procedures in terms of the proportion of times that the procedure identified the true pattern of differences among a set of means across several one‐way layouts. The protected version of the model selection approach selected the true model a significantly greater proportion of times than the stepwise procedures and, in most cases, was not affected by variance heterogeneity and non‐normality.     

An evaluation of exact methods for the multiple subset maximum cardinality selection problem

The maximum cardinality subset selection problem requires finding the largest possible subset from a set of objects, such that one or more conditions are satisfied. An important extension of this problem is to extract multiple subsets, where the addition of one more object to a larger subset would always be preferred to increases in the size of one or more smaller subsets. We refer to this as the multiple subset maximum cardinality selection problem (MSMCSP). A recently published branch‐and‐bound algorithm solves the MSMCSP as a partitioning problem. Unfortunately, the computational requirement associated with the algorithm is often enormous, thus rendering the method infeasible from a practical standpoint. In this paper, we present an alternative approach that successively solves a series of binary integer linear programs to obtain a globally optimal solution to the MSMCSP. Computational comparisons of the methods using published similarity data for 45 food items reveal that the proposed sequential method is computationally far more efficient than the branch‐and‐bound approach.     

Measurement scales and scale independence in the analytic hierarchy process

One approach to evaluate the relative performance of decision alternatives with respect to multiple criteria is provided by the analytic hierarchy process. The method is based on pairwise comparisons between attributes, and several numerical measurement scales for the ratio statements have been proposed. The choice of measurement scale is re‐examined, and new arguments supporting the measurement scale of geometric progression are derived. Separately from the measurement scale considerations, the effects of the scale parameter in geometric measurement scale are also studied. By using a regression model for pairwise comparisons data, it is shown that the statistical inference does not depend on the value of the scale parameter in the case of a single pairwise comparison matrix. It is also shown when the scale independence of statistical inference can be achieved in a decision hierarchy. This requires the use of the geometric‐mean aggregation rule instead of the traditional arithmetic‐mean aggregation. The results of the case study demonstrate that the measurement scale and the aggregation rule have potentially large impacts on decision support. Copyright © 2000 John Wiley & Sons, Ltd.     

Two efficient selection methods for high-dimensional CD-CAT utilizing max-marginals factor from MAP query and ensemble learning approach

Computerized adaptive testing for cognitive diagnosis (CD-CAT) needs to be efficient and responsive in real time to meet practical applications' requirements. For high-dimensional data, the number of categories to be recognized in a test grows exponentially as the number of attributes increases, which can easily cause system reaction time to be too long such that it adversely affects the examinees and thus seriously impacts the measurement efficiency. More importantly, the long-time CPU operations and memory usage of item selection in CD-CAT due to intensive computation are impractical and cannot wholly meet practice needs. This paper proposed two new efficient selection strategies (HIA and CEL) for high-dimensional CD-CAT to address this issue by incorporating the max-marginals from the maximum a posteriori query and integrating the ensemble learning approach into the previous efficient selection methods, respectively. The performance of the proposed selection method was compared with the conventional selection method using simulated and real item pools. The results showed that the proposed methods could significantly improve the measurement efficiency with about 1/2–1/200 of the conventional methods' computation time while retaining similar measurement accuracy. With increasing number of attributes and size of the item pool, the computation time advantage of the proposed methods becomes more significant.     

Preference‐based Learning of Ideal Solutions in TOPSIS‐like Decision Models

Combining established modelling techniques from multiple‐criteria decision aiding with recent algorithmic advances in the emerging field of preference learning, we propose a new method that can be seen as an adaptive version of TOPSIS, the technique for order preference by similarity to ideal solution decision model (or at least a simplified variant of this model). On the basis of exemplary preference information in the form of pairwise comparisons between alternatives, our method seeks to induce an ‘ideal solution’ that, in conjunction with a weight factor for each criterion, represents the preferences of the decision maker. To this end, we resort to probabilistic models of discrete choice and make use of maximum likelihood inference. First experimental results on suitable preference data suggest that our approach is not only intuitively appealing and interesting from an interpretation point of view but also competitive to state‐of‐the‐art preference learning methods in terms of prediction accuracy. Copyright © 2014 John Wiley & Sons, Ltd.     

Coursework selection: A frame of reference approach using structural equation modelling

Background. Choice behaviour has far‐reaching consequences on students' educational careers. Previous models on course selection for example, the model of achievement‐related choices ( Wigfield & Eccles, 2000 ) and of self‐efficacy theory ( Bandura, 1997 ), stress the importance of ability perceptions (self‐concept of ability) as major determinants of choice. Aim. This article suggests a model of course selection, which assumes that comparisons within an internal frame of reference (as proposed by Marsh, 1986 , for the association between achievement and self‐concepts) can also be applied to the association between self‐concepts and course selection. It is therefore hypothesized that course selection is not only positively influenced by the self‐concept with respect to the corresponding subject, but also negatively with respect to alternative subjects. Moreover, it should be tested whether the effects of previous achievement on course selection are completely mediated by the self‐concepts. Sample. These assumptions were empirically tested using a sample of 296 students from secondary school classes, who were specializing in, for example, chemistry or biology in the following school term. Method. Self‐concepts and course selection were assessed via questionnaire. The postulated models were tested using a structural equation modelling approach for ordinal variables. Results. The core assumption, that course selection is determined by dimensional comparisons, was supported by significant negative paths from self‐concepts on the selection of non‐corresponding subjects. Furthermore, the effects of previous achievement on selection were completely mediated by self‐concepts. Conclusion. Previous models of choice behaviour should be extended, by considering not only the selected alternatives, but also the unselected ones. The finding that the influence of achievement on choice is completely mediated by self‐concepts, demonstrates that subjective interpretations of previous achievement determine subsequent behaviour.     

Spatial and conjoint models based on pairwise comparisons of dissimilarities and combined effects: Complete and incomplete designs

In pairwise multidimensional scaling, a spatial representation for a set of objects is determined from comparisons of the dissimilarity of any two objects drawn from the set to the dissimilarity of other pairs of objects drawn from that set. In pairwise conjoint scaling, comparisons among the joint effects produced by pairs of objects, where the objects in a pair are drawn from separate sets, are used to determine numerical representations for the objects in each set. Monte Carlo simulations of both pairwise dissimilarities and pairwise conjoint effects show that Johnson's algorithm can provide good metric recovery in the presence of high levels of error even when only a small percentage of the complete set of pairwise comparisons are tested.     

An Optimized Design of Choice Experiments: A New Approach for Studying Decision Behavior in Choice Task Experiments

In this paper, we present a new approach for the optimal experimental design problem of generating diagnostic choice tasks, where the respondent's decision strategy can be unambiguously deduced from the observed choice. In this new approach, we applied a genetic algorithm that creates a one‐to‐one correspondence between a set of predefined decision strategies and the alternatives of the choice task; it also manipulates the characteristics of the choice tasks. In addition, this new approach takes into account the measurement errors that can occur when the preferences of the decision makers are being measured. The proposed genetic algorithm is capable of generating diagnostic choice tasks even when the search space of possible choice tasks is very large. As proof‐of‐concept, we used this novel approach to generate respondent‐specific choice tasks with either low or high context‐based complexity that we operationalize by the similarity of alternatives and the conflict between alternatives. We find in an experiment that an increase in the similarity of the alternatives and an increase in the number of conflicts within the choice task lead to an increased use of non‐compensatory strategies and a decreased use of compensatory decision strategies. In contrast, the size of the choice tasks, measured by the number of attributes and alternatives, only weakly influences the strategy selection. Copyright © 2014 John Wiley & Sons, Ltd.     

A novel machine learning approach for early detection of hepatocellular carcinoma patients

Liver cancer is quite common type of cancer among individuals worldwide. Hepatocellular carcinoma (HCC) is the malignancy of liver cancer. It has high impact on individual’s life and investigating it early can decline the number of annual deaths. This study proposes a new machine learning approach to detect HCC using 165 patients. Ten well-known machine learning algorithms are employed. In the preprocessing step, the normalization approach is used. The genetic algorithm coupled with stratified 5-fold cross-validation method is applied twice, first for parameter optimization and then for feature selection. In this work, support vector machine (SVM) (type C-SVC) with new 2level genetic optimizer (genetic training) and feature selection yielded the highest accuracy and F₁-Score of 0.8849 and 0.8762 respectively. Our proposed model can be used to test the performance with huge database and aid the clinicians.     

An adaptive approach for modelling multiattribute choice

An adaptive approach for modelling individual-level choice among multiattribute alternatives using the binary logit model is presented. The algorithm involves the collection of paired comparison data. In an effort to maximize the amount of information obtainable from each response, it is based on the experimental design criterion of D-optimality. A simulation study indicates that the proposed algorithm outperforms other sequential selection approaches in terms of estimation accuracy and predictive efficiency under certain circumstances. The results appear to encourage the use of such an adaptive algorithm for individual-level modelling in light of the potential reduction in data requirements without significant loss in predictive accuracy.     

On the limitations of fixed‐step‐size adaptive methods with response confidence

The family of (non‐parametric, fixed‐step‐size) adaptive methods, also known as ‘up–down’ or ‘staircase’ methods, has been used extensively in psychophysical studies for threshold estimation. Extensions of adaptive methods to non‐binary responses have also been proposed. An example is the three‐category weighted up–down (WUD) method (Kaernbach, 2001) and its four‐category extension (Klein, 2001). Such an extension, however, is somewhat restricted, and in this paper we discuss its limitations. To facilitate the discussion, we characterize the extension of WUD by an algorithm that incorporates response confidence into a family of adaptive methods. This algorithm can also be applied to two other adaptive methods, namely Derman's up–down method and the biased‐coin design, which are suitable for estimating any threshold quantiles. We then discuss via simulations of the above three methods the limitations of the algorithm. To illustrate, we conduct a small scale of experiment using the extended WUD under different response confidence formats to evaluate the consistency of threshold estimation.     

Computerized adaptive testing: A mixture item selection approach for constrained situations

In computerized adaptive testing (CAT), traditionally the most discriminating items are selected to provide the maximum information so as to attain the highest efficiency in trait (θ) estimation. The maximum information (MI) approach typically results in unbalanced item exposure and hence high item‐overlap rates across examinees. Recently, Yi and Chang (2003) proposed the multiple stratification (MS) method to remedy the shortcomings of MI. In MS, items are first sorted according to content, then difficulty and finally discrimination parameters. As discriminating items are used strategically, MS offers a better utilization of the entire item pool. However, for testing with imposed non‐statistical constraints, this new stratification approach may not maintain its high efficiency. Through a series of simulation studies, this research explored the possible benefits of a mixture item selection approach (MS‐MI), integrating the MS and MI approaches, in testing with non‐statistical constraints. In all simulation conditions, MS consistently outperformed the other two competing approaches in item pool utilization, while the MS–MI and the MI approaches yielded higher measurement efficiency and offered better conformity to the constraints. Furthermore, the MS–MI approach was shown to perform better than MI on all evaluation criteria when control of item exposure was imposed.     

Model selection for minimum‐diameter partitioning

The minimum‐diameter partitioning problem (MDPP) seeks to produce compact clusters, as measured by an overall goodness‐of‐fit measure known as the partition diameter, which represents the maximum dissimilarity between any two objects placed in the same cluster. Complete‐linkage hierarchical clustering is perhaps the best‐known heuristic method for the MDPP and has an extensive history of applications in psychological research. Unfortunately, this method has several inherent shortcomings that impede the model selection process, such as: (1) sensitivity to the input order of the objects, (2) failure to obtain a globally optimal minimum‐diameter partition when cutting the tree at K clusters, and (3) the propensity for a large number of alternative minimum‐diameter partitions for a given K. We propose that each of these problems can be addressed by applying an algorithm that finds all of the minimum‐diameter partitions for different values of K. Model selection is then facilitated by considering, for each value of K, the reduction in the partition diameter, the number of alternative optima, and the partition agreement among the alternative optima. Using five examples from the empirical literature, we show the practical value of the proposed process for facilitating model selection for the MDPP.     

Pairwise Likelihood Ratio Tests and Model Selection Criteria for Structural Equation Models with Ordinal Variables

Correlated multivariate ordinal data can be analysed with structural equation models. Parameter estimation has been tackled in the literature using limited-information methods including three-stage least squares and pseudo-likelihood estimation methods such as pairwise maximum likelihood estimation. In this paper, two likelihood ratio test statistics and their asymptotic distributions are derived for testing overall goodness-of-fit and nested models, respectively, under the estimation framework of pairwise maximum likelihood estimation. Simulation results show a satisfactory performance of type I error and power for the proposed test statistics and also suggest that the performance of the proposed test statistics is similar to that of the test statistics derived under the three-stage diagonally weighted and unweighted least squares. Furthermore, the corresponding, under the pairwise framework, model selection criteria, AIC and BIC, show satisfactory results in selecting the right model in our simulation examples. The derivation of the likelihood ratio test statistics and model selection criteria under the pairwise framework together with pairwise estimation provide a flexible framework for fitting and testing structural equation models for ordinal as well as for other types of data. The test statistics derived and the model selection criteria are used on data on ‘trust in the police’ selected from the 2010 European Social Survey. The proposed test statistics and the model selection criteria have been implemented in the R package lavaan.     

一种高效的CD-CAT在线标定新方法：基于熵的信息增益与EM视角

项目增补(Item Replenishing)对认知诊断计算机自适应测验(CD-CAT)题库的维护有着至关重要的作用, 而在线标定是一种重要的项目增补方式。基于数据挖掘中特征选择(Feature Selection)的思路, 提出一种高效的基于熵的信息增益的在线标定方法(记为IGEOCM), 该方法利用被试在新旧题上的作答联合估计新题的Q矩阵和项目参数。研究采用Monte Carlo模拟实验验证所开发新方法的效果, 并同时与已有的在线标定方法SIE、SIE-R-BIC和RMSEA-N进行比较。结果表明：新开发的IGEOCM在各实验条件下均具有较好的项目标定精度和项目估计效率, 且整体上优于已有的SIE等方法; 同时, IGEOCM标定新题所需的时间低于SIE等方法。总之, 研究为CD-CAT题库中项目的增补提供了一种更为高效、准确的方法。     

An Experiment on the Numerical Modelling of Verbal Ratio Statements

Verbal statements are intuitively attractive for preference elicitation. In the analytic hierarchy process (AHP) the verbal responses to pairwise comparisons of relative importance are converted into real numbers according to the nine-point integer scale. Several alternative scales have been proposed for the conversion, but sufficient empirical evidence has not been produced to support the choice among these scales. We performed a comparative study in which subjects were requested to quantify verbal ratio statements by adjusting the heights of visually displayed bars. Subjects were also asked to employ verbal expressions in pairwise comparisons of areas of figures with different shapes. The principal result of the experiment was that the perceived meaning of the verbal expressions varies from one subject to the next and also depends on the set of elements involved in the comparison. Our results indicate that there are alternative numerical scales which yield more accurate estimates than the usual 1-to-9 scale and reduce the inconsistency of the comparison matrices. Alternative ways of using verbal preference statements are suggested to overcome the difficulties that arise from the context dependence of verbal pairwise comparisons. © 1997 John Wiley & Sons, Ltd.     

A New VNS Metaheuristic Using MADS as a Local Optimizer

We propose a new hybrid metaheuristic VNSMADS that is an implementation of the Variable Neighbourhood Search (VNS) algorithm with the Mesh Adaptive Direct Search (MADS) as the local search. Because MADS is a Derivative‐Free Optimization method, the targeted problems may be non‐smooth on black box functions possessing several local optima. Constraints other than bounds on the variables are not considered. VNSMADS is applied on a real‐world problem from Finance: the portfolio selection problem using the investor preferences. Numerical results show the efficiency of our approach. Copyright © 2012 John Wiley & Sons, Ltd.     

多级评分的认知诊断计算机化适应测验

本文在0-1评分的CD-CAT基础上, 拓展出了适合多级评分CD-CAT (psCD-CAT)的认知诊断模型及选题策略, 为实现多级评分CD-CAT提供了方法支持。Monte Carlo模拟实验结果表明：本文拓展的多级评分CD-CAT具有较理想的属性诊断正确率及测验效率和题库安全性, 可以用于多级评分数据的CD-CAT; 模拟实验还表明, 整体来看PS-PWKL和PS-HKL两种选题策略具有较高属性判准率、题库安全性和高测验效率, 且均优于PS-KL选题策略。总之, 本研究对于进一步拓展CD-CAT在实践中的应用提供了认知诊断模型与选题策略等。