Rank disagreement: A comparison of multi-criteria methodologies

A number of multi-criteria decision support techniques have emerged in recent years that use varying computational approaches to arrive at the most desirable solution and thereby ‘recommend’ a course of action. Decision makers who use the results of this analytic work should be assured that the computational schemes used by their supporting analysts or decision support software produce the appropriate solutions. We conducted a series of simulation experiments that compared the top-ranked options resulting from the computational algorithms that support Multi-Attribute Value Theory (MAVT) and three methods that are reported in the literature that allow rank reversals, the change in rank order of two options when an unrelated option is added or deleted from the analysis: the Analytical Hierarchy Process (AHP), Percentaging and the Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). We also included a Fuzzy algorithm proposed by Yager to gauge its consistency with the other algorithms, even though it is not subject to rank reversals. These experiments demonstrated that the MAVT and AHP techniques, when provided with the same decision outcome data, very often identify the same alternative as ‘best’. The other techniques are noticeably less consistent with MAVT, the Fuzzy algorithm being the least consistent. The situations under which the most frequent and significant differences occurred were dependent upon the method. The results of our experiments indicate that other issues (e.g. the processes used for problem structuring and the elicitation of value weights) are likely to be of greater significance to problem outcome (based on our experience) than the choice between the computational algorithms of MAVT and AHP. The results cause us to be concerned about the use of the other methods.     

Time complexity and consistency issues in using the AHP for making group decisions

Two different ways of using the AHP in making group decisions are compared and evaluated. The first method combines different experts’ opinions before applying an eigenvalue method to obtain final weights for decision alternatives. The second, in contrast, derives each expert's rating for the decision alternatives before combining them. Both methods take into account the relative importance of different experts in making decisions. Comparison and evaluation of these two methods are made by using two criteria: time complexity and consistency indices. A numerical example is provided to illustrate the use of these two methods, and results of a mathematical simulation are presented for comparing the time complexity in different-sized problems. © 1998 John Wiley & Sons, Ltd.     

On the Proper Interpretation and Impact of the AHP Axiom of Independence in Single-Criterion and Multiple-Criteria Problems

Some authors have proposed that the analytic hierarchy process (AHP) axiom of independence be relaxed to accommodate observations drawn from (a) examples of pairwise comparisons of alternatives in clusters in single-criterion AHP problems and (b) examples of problems in which the criteria have the same underlying measurement and both the achievement of the goal and the alternatives are measured objectively. We show that the illustrations given are actually single-criterion problems according to the AHP. Thus the AHP axiom of independence is inapplicable in both situations and therefore not violated. We also consider the consequence of failure to distinguish between a criterion as an attribute of alternatives and cluster of alternatives, the two being different in the hierarchic structure. Finally, we discuss transformable problems, which look like multicriteria problems but are actually single-criterion problems, and how failure to recognize this fact may lead to incorrect syntheses and false conclusions.     

An investigation of the role of scale values in the DS/AHP method of multi‐criteria decision making

DS/AHP is a method of multi‐criteria decision making based on the Dempster–Shafer theory of evidence and the analytic hierarchy process. Central to the utilization of DS/AHP is the composing of preference judgements on identified groups of decision alternatives (DA) across a number of criteria against all the DA present in the problem in question. This paper exposits a series of results whose objectives are to aid in the development of an effective set of preference scale values for use within DS/AHP. These results relate directly to the concomitant level of ignorance (uncertainty) with the judgements made on a single criterion. Two particular directions of investigation are undertaken, firstly in determining the necessary number of scale values available and secondly finding the necessary differences between scale values, dependent on whether an arithmetic or geometric progression is the basis for the scale values. Through an example, the implications and utilization of these results within DS/AHP are illustrated. Copyright © 2003 John Wiley & Sons, Ltd.     

MCDM methods in strategic planning of forestry on state‐owned lands in Finland: applications and experiences

The forests in Finland have been under intensive planning for decades. Currently, mathematical programming is widely used in planning of wood production. Today's multi‐functional forestry, however, calls for more flexible decision support methods. MCDM tools have been used in responding to fresh planning challenges. For example, the Finnish Forest and Park Service, entrusted with the care of the vast majority of state‐owned natural resources in Finland, endeavours to produce large‐scale natural resource plans satisfying the needs of both economic, social, and ecological sustainability. Participatory approach is applied in the process. Several forestry applications of MCDM methods, particularly those making use of the AHP or the HIPRE program, have been presented. Also, the outranking methods ELECTRE and PROMETHEE have been tested. Due to the nature of forestry applications, statistical techniques for analysing uncertainties in pairwise comparisons and for utilizing interval judgement data have been developed to improve the usability of the AHP. Recently, a hybrid method called A'WOT, making use of the AHP and SWOT, was also introduced into strategic forest planning. This paper summarizes the experiences gained in applying a MAVT and two outranking methods in connection with a participatory natural resource planning process in Finland. In addition, some results of the method development work related to application needs are briefly presented. The details of the planning cases reviewed here have previously been presented in forestry journals. The purpose of this paper is not only to show how MCDM methods have been applied in forestry, but also to discuss the usability and usefulness of MCDM methods from the viewpoint of supporting forestry decision making—and how they might further be improved. Also, some perspectives for the future development work of MCDM applications in the field of natural resource management are focused on. As a conclusion, the use of more than just one MCDM method in a single planning process is seen usually recommendable. In addition, developing hybrid MCDM methods is regarded as a potential direction for future research. Also, closer co‐operation between method developers and appliers is called for to produce more useful applications. Copyright © 2002 John Wiley & Sons, Ltd.     

Integration of Analytic Hierarchy Process with Regression Analysis to Identify Attractive Locations for Market Expansion

Selection of new geographies in which to expand is a key decision for businesses aspiring to go beyond the opportunities in the existing markets. The conventional approaches of market selection can only provide a set of systematic steps for problem solving without considering the relationships between the decision factors. Decision models based on statistical techniques are able to examine the relationship between decision factors but are unable to effectively assist decision makers in identifying the most promising market, particularly in terms of prioritizing across decision factors. Analytic Hierarchy Process (AHP) is a commonly used approach for choosing alternatives by prioritizing across multiple decision factors. The typical AHP modelling requires knowledge of criteria and/or alternatives along with their relative weights, generally elicited from field experts. Quite often, firms encounter situations where decision makers are aware of only the overall objective and a set of earmarked geographies for setting up market locations while being relatively unaware of decision criteria and relative weights. This precludes using AHP to identify promising market locations. This paper conceptualizes a market selection decision model that integrates AHP with statistical modelling techniques to identify the attractive market locations for the purpose of expansion. The model first uses principal component analysis and multiple regression to determine significant decision criteria and their weights. Thereafter, it applies AHP to prioritize the market locations across the decision criteria. This integrative approach is illustrated for identifying the attractive locations in rural markets for a steel firm in India. The major advantage of this approach is that unlike the existing models, it works in situations when firms have not enough knowledge about factors for evaluating alternative market locations. Another key advantage of the proposed model is that of economizing resources for data collection on variables representing decision factors. Copyright © 2012 John Wiley & Sons, Ltd.     

Individual postdecision processes in group settings

In two experiments Differentiation and Consolidation Theory (Diff Con) (Svenson, 1992) was used to investigate individual postdecision making processes in three-member groups. It was predicted that in groups in which the subjects preferred different alternatives (conflict groups), subjects would consolidate their own preferred alternative, and not the group's final decision. A second hypothesis was that no consolidation would be indicated in groups in which all members preferred the same alternative (non-conflict groups). The results showed that in conflict groups, the members who gave up their preferred alternative (minority members) consolidated their own preference, thereby significantly regretting the group decision. In contrast, members who got their own will through in the majority decision (majority members) showed no consolidation of the group decision. The corresponding pattern of results was replicated in a second experiment, using a different decision situation. The results indicated that perceptions of social support, agreement in a group and decreasing responsibility for a group's decision, could all partly substitute consolidation by attractiveness restructuring.     

Two new cases of rank reversals when the AHP and some of its additive variants are used that do not occur with the multiplicative AHP

Many researchers have long observed some cases in which certain ranking irregularities can occur when the original analytic hierarchy process (AHP), or some of its variants, are used. This paper presents two new categories of ranking irregularities which defy common intuition. These ranking irregularities occur when one decomposes a decision problem into a set of smaller problems each defined on two alternatives and the same criteria as the original problem. These irregularities are possible when the original AHP, or some of its additive variants, are used. Computational experiments on random test problems and an examination of some real‐life case studies suggest that these ranking irregularities are dramatically likely to occur. This paper also proves that these ranking irregularities are not possible when a multiplicative variant of the AHP is used. Copyright © 2001 John Wiley & Sons, Ltd.     

Multiple Criteria Decision‐Making in Management Accounting and Control‐State of the Art and Research Perspectives Based on a Bibliometric Study

As an academic discipline as well as in practice, decision support is one of the most important functions of management accounting and control (MAC). Owing to decision settings' increasing complexity, suitable multiple criteria methods are becoming increasingly important for MAC. We, therefore, examine the extent to which certain multiple criteria decision‐making (MCDM) methods are used in the MAC context. In order to do so, we undertook an extensive bibliometric analysis of the management accounting field during the last three decades. The results indicate an increase in the importance of MCDM. Furthermore, recent research activities were classified into different areas of decision‐making within MAC since the majority of publications deal with the areas of strategic management, budgeting and performance management. More specifically, strategy and performance evaluation, strategic planning and the selection of alternatives are the most prominent MCDM applications. Our analysis reveals that the analytic hierarchy process (AHP) is the most popular analytical tool with which to support management in complex decision‐making situations. The results of the study prove that technical MAC literature does not cover a desirable range of MCDM applications and that the AHP, as well as the analytic network process (ANP) are of increasing importance in respect of the international scope of MAC. Therefore, the area of MCDM continues to be an active part of management science research and application, specifically of MAC. Copyright © 2011 John Wiley & Sons, Ltd.     

A Self‐regulation Perspective on Hidden‐profile Problems: If–Then Planning to Review Information Improves Group Decisions

In hidden‐profile (HP) problems, groups squander their potential to make superior decisions because members fail to capitalize on each other's unique knowledge (unshared information). A new self‐regulation perspective suggests that hindrances in goal striving (e.g., failing to seize action opportunities) contribute to this problem. Implementation intentions (if–then plans) are known to help deal with hindrances in goal striving; therefore, supporting decision goals with if–then plans should improve the impact of unshared information on group decisions. Indeed, in line with past research, control participants in two experiments rarely identified the best alternative despite monetary incentives and setting decision goals. In contrast, simply adding if–then plans to review advantages of the non‐preferred alternatives before making the final decision significantly increased solution rates. Process manipulations (Experiment 1) and measures (Experiment 2) indicate that conceptualizing HP problems as a self‐regulation challenge provides explanatory power beyond existing accounts. Copyright © 2014 John Wiley & Sons, Ltd.     

A Comprehensive Literature Review of the Rank Reversal Phenomenon in the Analytic Hierarchy Process

During the last few decades, several multi‐criteria decision analysis methods have been proposed to help in selecting the best compromise alternatives. Among them, analytic hierarchy process (AHP) and its applications have attracted much attention from academics and practitioners. However, since the early 1980s, critics have raised questions regarding its proper use. One of them concerns the unacceptable changes in the ranks of the alternatives, called rank reversal, upon changing the structure of the decision. Several modifications were suggested to preserve ranks. In this paper, a classification scheme and a comprehensive literature review are presented in order to uncover, classify and interpret the current research on AHP methodologies and rank reversals. On the basis of the scheme, 61 scholarly papers from 18 journals are categorized into specific areas. The specific areas include the papers on the topics of adding/deleting alternatives and the papers published in adding/deleting criteria. The scholarly papers are also classified by (1) year of publication, (2) journal of publication, (3) authors' geographic location and (4) using the AHP in association with other methods. It is hoped that the paper can meet the needs of researchers and practitioners for convenient references of AHP methodologies and rank reversals and hence promote the future of rank reversal research. Copyright © 2012 John Wiley & Sons, Ltd.     

The Effect of Decision Strategy on Deciding to Defer Choice

Recent research has examined consumer decision making when the option of not choosing any of the alternatives is also provided. The findings from this research suggest that the decision to defer choice is sensitive to the uncertainty of choosing the most preferred option from the set of alternatives provided. Building on this research, the author tests whether the decision to defer choice is also influenced by task variables that influence decision uncertainty. In the first experiment, this proposition is tested for choice problems in which information on three relatively equally attractive alternatives is presented either sequentially or simultaneously. As predicted, the preference for the defer-choice option was greater when the three alternatives were presented simultaneously. A second study forced subjects into using one of four decision strategies in order to choose between two non-dominated alternatives. The preference for the no-choice option was found to be higher when the rule required explicit attribute tradeoffs and lower when it simplified choice. These results suggest that choice uncertainty is influenced by the decision strategy used to determine the preference among alternatives. We conclude with a discussion of the implications of the results for marketers' communication strategies.     

Generating consensus priority interval vectors for group decision‐making in the AHP

Many practical and important decision‐making problems are complicated by at least two factors: (1) the qualitative/subjective nature of some criteria often results in uncertainty in the individual ratings; and (2) group decision‐making is involved and some means of aggregating individual ratings is required. Traditionally, both individual and group priorities have been represented as point estimates, but this approach presents severe limitations for accommodating imprecision in the decision‐making process. This paper examines the group decision‐making problem in the context where priorities are represented as numeric intervals. A set of techniques that could be used at some of the phases of an analytic hierarchy process (AHP)‐based group decision‐making process, which has the objective of generating a ‘consensus’ priority that represents the group's opinion with regards to the relative importance of a set of N objects (e.g. criteria, alternatives), is presented. Copyright © 2000 John Wiley & Sons, Ltd.     

An Interactive Approach for Multicriteria Decision Making Using a Tchebycheff Utility Function Approximation

Multicriteria decision making (MCDM) can provide an efficient mean for considering various and conflicting objectives to reveal the alternative that maximizes the decision maker's (DM) utility. In this paper, we propose a new interactive MCDM method for implicit alternatives to help a DM obtain a most preferred solution. We employ a Tchebycheff function to generate weights for objectives consistent with the DM's responses to pairwise comparisons between alternatives and present a mixed integer linear programming formulation to generate these weights. Thus, we approximate the DM's utility function by a Tchebycheff function and generate weights consistent with the DM's responses. We test our approach with different true utility functions on various sized multiple criteria linear programming problems. The computational results show that even with non‐Tchebycheff true utility functions, our method can generate alternatives very close to the optimal solution with few questions. The comparison of our results with other methods reveals its advantages. Copyright © 2013 John Wiley & Sons, Ltd.     

Attribute‐based differentiation of alternatives

An intermediate step is introduced to the dialogue decision process for decision analysis. Alternatives are refined after they have been generated within a strategy table but before they are subject to more detailed evaluation. Two or more judges create a subjective mapping from alternatives to attributes that will later be mapped to criteria. In strategy tables, each of the alternative strategies consists of a coherent set of choices made across several decisions that are to be coordinated. These strategic alternatives are modified so as to increase their differentiation in the attribute space, rather than in the decision space alone. When criteria weights are unknown, the best alternative from the modified set may be superior to the best alternative from the original set. Furthermore, analysis of the resulting alternatives may yield a better mapping of the value response surface for the action space, in the sense that this mapping leads to eventual construction of a higher value alternative. Results are reported for a consulting engagement incorporating the proposed step. Copyright © 2003 John Wiley & Sons, Ltd.     

A Field Experiment Comparing Anchored and Unanchored Criteria Weights in the Analytic Hierarchy Process

This study reports the results of a field experiment comparing the predictive validity of two approaches to multicriteria assessments: the absolute measurement mode of AHP and the absolute measurement mode of linking pin AHP. The questioning procedures of the two differ only in that the former employs unanchored criteria weight assessments and the latter anchored criteria weight assessments. The decision task required insurance agents to respond to a mailed questionnaire in which they evaluated non-monetary incentives (contests) according to (1) the public recognition received from winning, (2) the criteria for winning and (3) the nature of the reward. There were four levels for each dimension. A between subjects design was used, with each subject receiving one of the two methods. In addition, all subjects divided 100 points among four contests and these hold-out assignments were employed as a validity check. Linking pin AHP was found to be superior to conventional AHP in this experiment, lending weight to the argument that the use of unanchored criteria weights is problematic. © 1997 by John Wiley & Sons, Ltd. J. Multi-Crit. Decis. Anal. 6 : 140–149 (1997) No. of Figures: 3. No. of Tables: 5. No. of References: 24.     

The Characteristic Objects Method: A New Distance‐based Approach to Multicriteria Decision‐making Problems

Multicriteria decision‐making (MCDM) methods are concerned with the ranking of alternatives based on expert judgements made using a number of criteria. In the MCDM field, the distance‐based approach is one popular method for obtaining a final ranking. The technique for order preference by similarity to the ideal solution (TOPSIS) is a commonly used example of this kind of MCDM method. The TOPSIS ranks the alternatives with respect to their geometric distance from the positive and negative ideal solutions. Unfortunately, two reference points are often insufficient, especially for nonlinear problems. As a consequence of this situation, the final result ranking is prone to errors, including the rank reversals phenomenon. This study proposes a new distance‐based MCDM method: the characteristic objects method. In this approach, the preferences of each alternative are obtained on the basis of the distance from the nearest characteristic objects and their values. For this purpose, we have determined the domain and Fuzzy number set for all the considered criteria. The characteristic objects are obtained as the combination of the crisp values of all the Fuzzy numbers. The preference values of all the characteristic object are determined on the basis of the tournament method and the principle of indifference. Finally, the Fuzzy model is constructed and is used to calculate preference values of the alternatives, making it a multicriteria model that is free of rank reversal. The numerical example is used to illustrate the efficiency of the proposed method with respect to results from the TOPSIS method. The characteristic objects method results are more realistic than the TOPSIS results. Copyright © 2014 John Wiley & Sons, Ltd.     

Euclid: Strategic alternative assessment matrix

The vast amount of information that must be considered to solve inherently ill‐structured and complex strategic problems creates a need for tools to help decision makers (DMs) recognize the complexity of this process and develop a rational model for strategy evaluation. Over the last several decades, a philosophy and a body of intuitive and analytical methods have been developed to assist DMs in the evaluation of strategic alternatives. However, the intuitive methods lack a structured framework for the systematic evaluation of strategic alternatives while the analytical methods are not intended to capture intuitive preferences. Euclid is a simple and yet sophisticated multiobjective value analysis model that attempts to uncover some of the complexities inherent in the evaluation of strategic alternatives. The proposed model uses a series of intuitive and analytical methods including environmental scanning, the analytic hierarchy process (AHP), subjective probabilities, and the theory of displaced ideal, to plot strategic alternatives on a matrix based on their Euclidean distance from the ideal alternative. Euclid is further compared to the quantitative strategic planning matrix (QSPM) in a real world application. The information provided by the users shows that Euclid can significantly enhance decision quality and the DM's confidence. Euclid is not intended to replace the DMs, rather, it provides a systematic approach to support, supplement, and ensure the internal consistency of their judgments through a series of logically sound techniques. Copyright © 2003 John Wiley & Sons, Ltd.     

A Psychometric Analysis of the “Divide and Conquer” Principle in Multicriteria Decision Making

The principle of “Divide and Conquer” (DAC) suggests that: (1) complex decision problems should be decomposed into smaller, more manageable parts and (2) these smaller parts should be logically aggregated to derive an overall value for each alternative. Typically, decompositional procedures have been compared to holistic evaluations that require decision makers to simultaneously consider all of the relevant attributes in the evaluation of the objects under consideration. These comparisons between decompositional and holistic judgments have primarily used a variant of Multiattribute Utility Theory (MAUT).We presented a general experimental framework that allows for a more extensive assessment of the DAC principle, as well as the effects of decision complexity on both holistic and decompositional procedures. We illustrate this approach with a study that uses the Simple Multiattribute Rating Technique with Swing Weights (SMARTS; Edwards & Barron, 1994) and the Analytic Hierarchy Process (AHP; Saaty, 1980). We report data comparing the convergent validity (e.g the agreement between decompositional and holistic strategies) and the temporal stability for decompositional and holistic judgments on a variety of dependent measures. Decision complexity did not significantly affect the correspondence between decompositional and holistic judgments for both SMARTS and AHP judgments. Results from an ordinal measure of temporal stability indicated the DAC principle was violated for the AHP judgments. For a linear measure of temporal stability, trends in the data indicated that the predicted effects of decision complexity on the DAC principle was violated for the SMARTS judgments.     

A parameter analysis method for the weight‐set to satisfy preference orders of alternatives in additive multi‐criteria value models

In the context of the additive multi‐criteria value model, this paper investigates how the set of criteria weights (weight‐set hereafter) can be determined according to the preference orders of alternatives given by the decision maker. A construction method is proposed for the weight‐set for different intervals of β, where β is a differential amount of value between the preference information on two alternatives. The results of this paper are important for sensitivity analysis in multi‐criteria decision making (MCDM) problems and multi‐criteria group decision analysis. Copyright © 2000 John Wiley & Sons, Ltd.