Non-Numerical Ranking Based on Pairwise Comparisons

Abstract

<p>The method of Pairwise Comparisons was first described by Marquis de Condorcet in 1785 [1]. At present, this method is identified with Saaty's Analytic Hierarchy Process (AHP, 1980) [25]. AHP is a formal method to derive ranking orders from pairwise comparisons. It is used around the world in a wide variety of decision making, in fields such as education, industry, and government.</p> <p>On the one hand, AHP has many respected practical applications. On the other hand, however, it is still considered by many researchers as a flawed procedure that produces arbitrary rankings [6].</p> <p>A non-numerical partial orders based method was proposed by Janicki and Koczkodaj [14] and developed by Janicki [9, 10, 11, 12, 13]. This model used the concepts of partial orders and rough sets, and emphasized the importance of indifference and weak ordering. However, the consistency rules of the model are incomplete. When the inconsistent pairs are found, the non-numerical ranking method manually changes the relationship among those pairs to satisfy the consistency rules.</p> <p>We extend the consistency rules to make them complete and compact. A consistency-driven algorithm by automatically enforcing "consistency" is presented. Property-driven algorithms by classical partial order approximations and refined partial order approximations are discussed. We present an algorithm using refined partial order approximations. A method of automatically converting AHP data to nonnumerical pairwise comparison ranking system is discussed, which ensures that the generated non-numerical pairwise comparison ranking system is consistent.</p> <p>We implement various ranking algorithms, including the AHP method, consistency-driven method, property-driven method and property/consistency-driven method. We test the experiments referenced in some non-numerical ranking papers, and give examples to compare how well the various non-numerical ranking methods solve the rank reversal problem.</p>