Resolving Rank Reversal in TOPSIS: A Comprehensive Analysis of Distance Metrics and Normalization Methods

1Introduction

Multiple-Criteria Decision Making (MCDM) is a systematic approach used to make decisions in complex scenarios where multiple conflicting criteria or objectives need to be considered simultaneously. In many real-world situations, decisions cannot be made solely based on a single criterion; instead, multiple factors with different levels of importance and often incommensurable units need to be taken into account. MCDM provides a structured framework to evaluate and compare different alternatives in a way that reflects the preferences and priorities of decision makers.

During the MCDM process, the data representing different criteria might come in various formats, units, and scales. These criteria could include quantitative measurements like costs, benefits, or performance metrics, as well as qualitative factors like risk levels or expert opinions. Due to these differences, directly comparing and combining these criteria for decision-making purposes can be challenging and lead to misleading results. To deal with the problem, normalization has become a fundamental process in MCDM that enables decision makers to effectively compare and evaluate alternatives across multiple criteria (Pavličić, 2001). It generally supports the goal of making well-informed, objective, and consistent decisions that accurately reflect the preferences and priorities of decision makers.

Distance-based models in MCDM play a crucial role in quantifying the dissimilarity between alternatives, and the issue of rank reversal (RR) has drawn significant attention within this context. RR denotes a phenomenon where the rank order of alternatives undergoes a change due to alterations in the evaluated alternatives or related parameters. This phenomenon contradicts our intuitive expectations and is formally termed the axiom of independence of irrelevant alternatives (IIA) in the multi-attribute utility theory (von Winterfeldt and Edwards, 1986). Belton and Gear (1983) initially identified RR in the Analytic Hierarchy Process (AHP), attributing it to AHP’s relative measurement mode. Salo and Hämäläinen (1997) extended this finding, linking RR to AHP’s mode of measurement. The Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), applied to MCDM, was later reported to exhibit RR by Triantaphyllou (2000). Subsequent debates in major operations research journals, including works by Saaty (1990a, 1990b, 1991), Saaty and Vargas (1984), Triantaphyllou (2001), and Kulakowski et al. (2019), further underscored the significance of RR.

TOPSIS stands out as a straightforward and intuitive method that accommodates both the positive and negative aspects of each alternative, making it a widely adopted approach in decision-making processes dealing with multiple, conflicting criteria. It has been still well received until recently, e.g. Biswas et al. (2024) and Kousar et al. (2024). Despite its popularity, TOPSIS is not immune to RR, although its occurrence is notably less pronounced compared to AHP (Zanakis et al., 1998). The RR phenomenon in TOPSIS arises from the sensitivity of the distance metric to variations in the attribute values of alternatives. The normalization process in TOPSIS involves dividing the performance values by their respective weights. Furthermore, the choice of distance metrics and the determination of the positive-ideal solution (PIS) and negative-ideal or anti-negative solution (NIS) can introduce subjectivity, influencing the final rankings. Therefore, careful consideration of these factors is crucial to mitigate the RR problem and enhance the robustness of the TOPSIS method in decision-making processes.

To address the issue of RR, previous researchers such as Senouci et al. (2016) and Yang (2020) have suggested approaches like linear normalization and using absolute positive-ideal and negative-ideal solutions. However, many studies have relied on simplistic, fabricated examples or simulations to illustrate the RR phenomenon, leaving its real-world impact and significance unclear (Shih and Olson, 2022). This research is motivated by the need for a more comprehensive understanding of RR in the TOPSIS method, driving an in-depth mathematical analysis of the underlying formulae to better quantify and explain the occurrence of RR.

Distance metrics, such as Manhattan, Euclidean, and Chebyshev, measure the dissimilarity or proximity between alternatives in different ways, each introducing variations in rankings due to their sensitivity to the scale and distribution of data. These variations can trigger RR by affecting the computed distances between alternatives. To mitigate RR, it is crucial to examine how these distance metrics influence the final rankings, providing insight into the underlying mechanics of RR. This study aims to fill a gap in the literature by exploring the role of these distance measures in the TOPSIS framework, offering a more detailed understanding of their influence on ranking reversals.

While the Chebyshev distance in TOPSIS offers robust ranking results, it has limitations in capturing subtle performance differences across multiple criteria. This lack of sensitivity to nuanced variations can lead to less precise distinctions between alternatives. Additionally, although Euclidean distance has been shown to outperform Manhattan distance in reducing RR, it may not fully eliminate the phenomenon. The study supports these conclusions through a case example, providing empirical validation for the effectiveness of different distance metrics in preventing RR, while acknowledging the limitations inherent to each approach.

The rest of the paper runs as follows. Section 2 reviews the existing resolutions for RR in TOPSIS. Section 3 delves into an examination of the ranking index of TOPSIS, presenting a mathematical perspective on TOPSIS using different distance metric methods for handling RR. Section 4 examines a case study. Section 5 offers concluding remarks on RR solutions for TOPSIS.

2Literature Review

RR can occur in an MCDM process when a new alternative is introduced or an existing one is removed from the candidate list, all without any adjustments to the criteria weights. This phenomenon is rooted in the intricate interactions between alternatives and their relative positions in the ranking. With the addition of a new alternative, the dynamic relationships among all options might shift, causing some alternatives to be reevaluated in the context of the changed landscape. Similarly, the removal of an alternative can lead to redistributions in the ranking as the absence of that option could amplify the importance of others. Rank reversal serves as a reminder of the delicate balance within decision frameworks, highlights the non-intuitive nature of multi-criteria scenarios, and underscores the need for a thoughtful and adaptable approach to decision-making.

Building on the observations of the RR phenomenon in TOPSIS, Ren et al. (2007) presented pioneering research in addressing RR within TOPSIS. They introduced a novel modified synthetic evaluation method wherein two separation measures are transformed into a two-dimensional plane, and the ranking index is determined by the distance to PIS minus the minimum of all alternatives to PIS. However, it appears that their proposal lacks a systematic approach to avoiding the RR phenomenon, and the examples presented, such as those in the public health system and student evaluation, serve primarily for illustrative purposes.

Wang and Luo (2009) contributed to the literature by showcasing an example highlighting the occurrence of RR in the TOPSIS algorithm. However, they did not specifically identify the impact arising from non-dominated alternatives.

Zavadskas et al. (2006) studied the impact of vector and max-min normalization on TOPSIS ranking accuracy. Chakraborty and Yeh (2009) simulated comparisons of one vector and three linear normalization methods for TOPSIS, advocating for vector normalization based on ranking consistency and weight sensitivity. Çelen (2014) compared four normalization methods from Chakraborty and Yeh (2009) in evaluating the Turkish deposit banking market, affirming vector normalization’s reliability and noting that max-min and max normalization also produced consistent results. Kong (2011) aimed to mitigate RR by introducing linear normalization and incorporating extreme fictitious alternatives to stabilize PIS and NIS. However, Kong’s example was limited to five alternatives with two criteria.

García-Cascale and Lamata (2012) similarly considered the approaches introduced by Kong (2011). They further elaborated on numerous situations, presenting an example with four alternatives and two criteria.

İç (2014) explored the integration of design of experiments (DoE) into TOPSIS for the financial performance evaluation of companies. The study concluded with a five-alternative case involving five criteria to illustrate the DoE-TOPSIS method, aiming to preserve ranking. However, despite the belief that this combination could avoid RR, the method did not demonstrate improvement in the steps of TOPSIS.

Cables et al. (2016) proposed a reference ideal method, redefining the distance to the reference ideal as an interval and linearly normalizing performance data into a value. They also modified the separation measures, making the reference ideal as the vector of weights. The method was illustrated using a personnel selection case with five candidates and six criteria, claiming independence from the type of data. However, verification of this illustration is challenging due to the assumed interval values and reliance on a single example.

Senouci et al. (2016) suggested a modified TOPSIS for selecting a mobile network interface, incorporating linear normalization with four types and dynamically setting max/min values against RR. Their case involved seven alternatives and was evaluated by five criteria through simulation. Despite achieving a zero RR ratio with two types of normalization, we will later demonstrate that their case is unable to avert the RR phenomenon.

In addition to the developments above, Kuo (2017) investigated weighting on the elements of the TOPSIS ranking index, showing that the proposed modified ranking index outperforms the traditional TOPSIS in ranking consistency. Mufazzal and Muzakkir (2018) proposed a proximity index to overcome RR, utilizing Manhattan distance to count separation measures and using the sum of distances as the ranking index. Although they provided seven historical cases as evidence for the proposed approach, they were unable to clarify the ranking differences observed in their outcomes or elucidate the properties of the index.

Authors de Farias Aires and Ferreira (2019) introduced two forms of linear normalization to modify the traditional TOPSIS. They demonstrated the robustness of their approach through a student selection case involving twenty alternatives with three criteria and their subsets.

Yang (2020) made modifications to the approach proposed by de Farias Aires and Ferreira (2019), incorporating linear normalization of historical extreme values and assigning weights only to PIS. The study used Wang and Luo’s questionable example (Wang and Luo, 2009) to showcase the rank stability achieved by their approach.

Based on the studies mentioned above, the proposed solutions to mitigate RR are classified into the following five main categories.

3A Mathematical Exploration of Normalization and Distance Measurements in TOPSIS

This section provides a comprehensive mathematical explanation of how normalization and distance measurements contribute to the phenomenon of RR. This investigation aims to understand the factors that give rise to RR in MCDM, with a particular focus on TOPSIS. The goal is to enhance decision-making by addressing the occurrences of rank reversal, which are comparatively less frequent in TOPSIS. The common algorithm of TOPSIS for ranking and selection includes the following seven steps (Hwang and Yoon, 1981).

In the TOPSIS methodology, when comparing two alternatives, Ap and Aq, their respective relative closeness values are calculated as follows: Cp∗=Sp−/(Sp++Sp−) for alternative Ap and Cq∗=Sq−/(Sq++Sq−) for alternative Aq. If alternative Ap is preferred over alternative Aq in TOPSIS, then Cp∗ must be greater than Cq∗. This can be expressed as:

The mathematical models are based on a scenario in which a new alternative with the highest performance in a specific criterion, referred to as criterion k, is added to the candidate list. All criteria are considered as benefit criteria. Benefit criteria represent the positive attributes or qualities that decision-makers are trying to optimize or maximize in their decision-making process.

Similar to many MCDM methods, TOPSIS utilizes criteria weights in its aggregation process. These weights, assigned to criteria, play a crucial role in measuring the overall preferences of alternatives. Different sets of weights can significantly impact the ranks of alternatives (Kaliszewski et al., 2018). However, this study solely focuses on the distance measurements and normalization methods, thereby disregarding the weighting effect. Hence, all criteria weights are assumed to be identical and unchanged. Consequently, we assume wj=1/n for all criteria j. Figure 1 outlines the scope of this mathematical evaluation. The forthcoming exploration will delve into the mathematical aspects to identify the fundamental causes of RR in TOPSIS, particularly focusing on the impact of the normalization process and the use of different distance metrics. Initially, the max-min normalization method will be employed as a basis for analysis. The study will examine how different distance metrics—namely Euclidean, Manhattan, and Chebyshev—affect the ranking of alternatives, driving a detailed mathematical investigation into the underlying formulae to better quantify and explain the occurrence of RR. In the subsequent section, the analysis will extend to explore the effects of the max normalization method.

Fig. 1

Scope of the study.

3.2Euclidean Distance

The Euclidean distance is a straightforward and widely used measurement that calculates the straight-line distance between two points in a multi-dimensional space. In this case, following the TOPSIS method and utilizing Euclidean distance for separation measures, the initial two separation measures, Si+ and Si−, before introducing the new alternative are as follows:

(9)

Si+=1n∑j=1n(rij−1)2,Si−=1n∑j=1n(rij)2.

Within the candidate list, select any two alternatives, Ap and Aq, where Ap is preferred over Aq according to the TOPSIS method. It can then be observed that the expression Sp−Sq+−Sp+Sq−>0, or equivalently, Sp−Sq+>Sp+Sq−, holds true. Representing this expression using the Euclidean metric, the following result is derived:

(10)

∑j=1n(rpj)2·∑j=1n(rqj−1)2>∑j=1n(rpj−1)2·∑j=1n(rqj)2.

However, upon introducing the new alternative in this case, the separation measures are modified to:

(11)

Si+′=1n∑j=1n(rij−)2+(rik′−1)2−(rik−1)2,Si−′=1n∑j=1nrij2+rik′2−rik2.

Given that alternative Ap is initially preferred to Aq according to the TOPSIS method, when the new alternative is introduced to the list of candidates, it is imperative to uphold the following formula to avoid alterations in the ranking order.

(12)

Sp−′Sq+′>Sp+′Sq−′orSp−′Sq+′−Sp+′Sq−′>0.

In mathematical terms, this relationship is expressed as:

(13)

∑j=1n((rpj)2−rpk2(1−δk2))∑j=1n((rqj−1)2+(δkrqk−1)2−(rqk−1)2)>∑j=1n((rpj−1)2+(δkrpk−1)2−(rpk−1)2)∑j=1n((rqj)2−rqk2(1−δk2)),

where it is observed that rik2(1−δk2)>0 and (δkrik−1)2−(rik−1)2>0. To comprehend how the introduction of a new alternative, excelling in criterion k, to the candidate list influences the likelihood of RR, two distinct scenarios covering various conditions are explored.

Scenario 1: xpk>xqk.  In the given scenario, it is apparent that rpk>rqk and rpk′>rqk′. Furthermore, the following inequalities are valid:

(14)

rpk2(1−δk2)>rqk2(1−δk2)>0.

Moreover, it is ensured that if rpk+rqk<2/(δk+1), then

(15)

(δkrpk−1)2−(rpk−1)2>(δkrqk−1)2−(rqk−1)2>0;

otherwise,

(16)

(δkrqk−1)2−(rqk−1)2>(δkrpk−1)2−(rpk−1)2>0.

It is noted that rpk+rqk<1 and 1⩽2(δk+1)⩽2. Therefore, rpk+rqk<2(δk+1), which implies that (δkrpk−1)2−(rpk−1)2 is always larger than (δkrqk−1)2−(rqk−1)2.

The sum of squared values ∑j=1n(rij)2 in essence significantly surpasses rik2−rik′2, and ∑j=1n(rij−1)2 is notably larger than (δkrik−1)2−(rik−1)2. In most cases, the condition Sp−′Sq+′>Sp+′Sq−′ can be maintained when the new alternative is introduced. However, if Sp−Sq+≅Sq−Sp+, then the final ranking between alternatives p and q can be altered especially when rpk2(1−δk2)>>rqk2(1−δk2) and (δkrpk−1)2−(rpk−1)2≫(δkqik−1)2−(rqk−1)2. Let G1 represent the gap between rpk2(1−δk2) and rqk2(1−δk2), and G2 represent the gap between (δkrpk−1)2−(rpk−1)2 and (δkrqk−1)2−(rqk−1)2. It is obvious that Δk amplifies the difference between rpk2(1−δk2) and rqk2(1−δk2), leading to an increased gap. Similarly, the disparity between (δkrpk−1)2−(rpk−1)2 and (δkrqk−1)2−(rqk−1)2 also widens. Figures 2 and 3 illustrate the values of G1 and G2 for varying rpk−rqk and Δk, with rqk is fixed at 0.1. The parameter Δk plays a critical role in amplifying both gap measures. The impact is particularly strong at higher rpk−rqk values. For example, when rpk=0.9, increasing Δk from 1 to 10 results in a substantial growth in both G1 and G2. At lower rpk−rqk, the effect of Δk is less pronounced but still present.

Fig. 2

The gap G1 as a function of rpk and Δk.

Fig. 3

The gap G2 as a function of rpk and Δk.

Scenario 2: xpk⩽xqk.  Given rqk>rpk, rqk′>rpk′, it can be demonstrated that:

(17)

rqk2(1−δk2)>rpk2(1−δk2)>0and(δkrqk−1)2−(rqk−1)2>(δkrpk−1)2−(rpk−1)2>0.

As a result, the inequality Sp−′Sq+′>Sp+′Sq−′ still upholds. The possibility of RR occurring is negligible.

4Illustrative Case

Hwang and Yoon’s fighter selection problem is modified to consider only the first three benefit criteria for interpretation (Hwang and Yoon, 1981). Table 4 shows the decision matrix. Criterion X1 corresponds to maximum speed, X2 pertains to ferry range, and X3 relates to maximum payload. Table 5 shows the normalized decision matrix using the min-max normalization method. Utilizing the TOPSIS method, Tables 6, 7, and 8 showcases the ranking outcomes obtained by employing Manhattan distance, Euclidean distance, and Chebyshev distance to calculate separation measures, respectively, without taking criteria weights into account.

According to Tables 6 and 7, using Manhattan and Euclidean distances yields identical ranking results. The initial ranking result shows A2≻A4≻A3≻A1. However, when employing Chebyshev distance, the separation measures calculate the distance between the alternative and PIS/NIS, considering only the maximum absolute difference among the three criteria. Under such circumstances, the outcomes are determined solely by the performance of the alternative in a single criterion. Therefore, when an alternative excels in a particular criterion, it secures the optimal ranking. For example, A3 outperforms in criterion X3, resulting in a priority 1 ranking, or the same as A2. Because using Chebyshev distance in calculating separation measures does not simultaneously consider the influence of all criteria, the resulting ranking may be biased. Evaluating the performance of alternatives neglects other criteria and focuses solely on one criterion with extreme values. Therefore, adopting Chebyshev distance to compute separation measures in most MCDM problems might not be entirely appropriate. It could be challenging to differentiate rankings in the performance assessment of many candidate solutions.

To simulate a scenario where RR might arise, alternative A5 is introduced for evaluation based on the criteria [X1=2,X2=2,700,X3=21,000]. Despite the addition of A5, PIS remains unchanged. The new rankings, using Manhattan and Euclidean distances, are A5≻A2≻A4≻A3≻A1., However, when employing Chebyshev distance, the rankings are A5≻A2=A3≻A4≻A1. Importantly, the order of A1, A2, A3, and A4 remains consistent by using the same distance measurement, indicating the absence of RR. The achievement rating of alternative A5 on criterion X2 is gradually increased, leading to a change in PIS. In the current scenario, when the achievement rating of alternative A5 on criterion X2 continues to increase, the ratio of change ΔX2 is greater than 1 and increases synchronously. Tables 9, 10, and 11 illustrate the effect of increasing ΔX2 on alternatives A2 and A4 by using various distance measurements.