You are viewing a javascript disabled version of the site. Please enable Javascript for this site to function properly.
Go to headerGo to navigationGo to searchGo to contentsGo to footer
In content section. Select this link to jump to navigation

“Power ranking” professional circuit eSports teams using multi-criteria decision-making (MCDM)

Article type: Research Article

Authors: Pradhan, Seana; b; * | Abdourazakou, Yannc

Affiliations: [a] Department of Management, Menlo College, 1000 El Camino Real, Atherton, USA | [b] Center for Sports Management, Menlo College, 1000 El Camino Real, Atherton, USA | [c] Department of Management and Marketing, California State University – Dominguez Hills, 1000 E. Victoria Street, Carson, USA

Correspondence: [*] Corresponding author: Sean Pradhan, Tel.: +1 650 543 3988; Fax: +1 650 543 4496; E-mail: [email protected].

Keywords: Ranking, eSports, multi-criteria decision-making (MCDM), grey relational analysis (GRA), simple additive weighting (SAW), technique for order preference by similarity to the ideal solution (TOPSIS)

DOI: 10.3233/JSA-190420

Journal: Journal of Sports Analytics, vol. 6, no. 1, pp. 61-73, 2020

Published: 27 February 2020
Get PDF open access

Abstract

The utilisation of performance criteria to determine power rankings of various teams is useful for a vast number of audiences in many traditional sports, such as soccer, football, baseball, basketball, hockey, and the like. With market hyper-competition and viewership growth in the digital sector of eSports and related professional video gaming, the usage of performance evaluation in the scholarly literature is relatively scant. Thus, the current study provides a performance assessment and ranking structure in eSports. Applying a hybrid technique developed by previous research involving a series of Multi-Criteria Decision-Making (MCDM) tools, those being: Grey Relational Analysis (GRA), Simple Additive Weighting (SAW), and the Technique for Order Preference by Similarity to the Ideal Solution (TOPSIS), this study develops power rankings for professional teams in the popular multiplayer online battle arena (MOBA) game, Defense of the Ancients (Dota) 2. Further analyses offer a comparison of the produced ranking structure to other metrics commonly used in eSports evaluation, such as Elo and Glicko ratings. The findings of this study offer implications for players, teams, performance analysts, oddsmakers, consumers, among others. Recommendations and future directions of this research are also provided.

1Introduction

Over the course of more than 40 years, video games have advanced far beyond Pitfall!, Asteroids, and Frogger on the Atari 2600 gaming console, with some of the earliest gaming tournaments taking place in the United States during the 1970s (Bountie Gaming, 2018). Considering this, gameplay continues to be adapted by developers and organisers in order to garner the interest of millions of people around sporting events of a new kind, often termed eSports, filled with familiar features such as commentators, large screens, and stages. Since 2012, the number of fans and viewers in this domain has been steadily increasing (Statista, 2018). eSports figures are rather promising as they have also improved year-over-year. NewZoo, a market intelligence agency specializing in digital media and video gaming, estimates that eSports generated $345 million in revenue in North America in 2018, along with over $500 million abroad (Pannekeet, 2018). Furthermore, the most recent global revenue figures have experienced a year-on-year growth of over 26%, with revenue approaching over $1 billion during 2019 (Pannekeet, 2019).

Within this realm, Multiplayer Online Battle Arena (MOBA) titles are among the most played and watched digital games in the world (Gaudiosi, 2012). Some of the most popular MOBA games include League of Legends (LoL), Heroes of the Storm, Smite, Strife, and Defense of the Ancients (Dota) 2. Despite this, Ingraham (2018) points out that the North American eSports market pales in comparison to more traditional sports leagues, such as the National Football League (NFL), Major League Baseball (MLB), National Hockey League (NHL), and National Basketball Association (NBA), which each brought in over $4 billion in 2017 alone (Mathewson, 2019). In a more promising vein, viewership for the 2018 Dota 2 International Championship nearly matched that of the 2018 Kentucky Derby and exceeded figures for the Association of Tennis Professionals’ (ATP) 2018 Wimbledon tournament, National Association for Stock Car Auto Racing’s (NASCAR) Daytona 500 in 2018, the Professional Golfers’ Association (PGA) 2018 United States Open, and the Union Cycliste Internationale (UCI) 2017 Tour de France (Ingraham, 2018).

Produced by Valve Corporation, Dota 2 is an action real-time strategy (ARTS) video game, which sets two teams, named the “Radiant” and the “Dire”, of five characters (or “heroes”) against one another in a fictional fantasy world (Gaston, 2013). The objective of the game is to demolish the opposing team’s main building called the “Ancient”. Teams begin the game at their own base and move into enemy territory using three pathways called “lanes”. Several buildings called “Towers” are distributed within these lanes and will attack members of the opposing team. A river, positioned towards the centre of the map, divides the teams’ territories equally. Creatures called “Barracks” will spawn every 30 seconds within the lanes and proceed to the enemy base, attacking opposing players, Towers, and enemy Barracks along the way (Gaston, 2013). To strengthen the selected hero, players gain experience points by defeating enemies. Dota 2, considered an advanced form of ARTS, also contains an intricate item drop system that can be used to further enhance a player’s hero.

At this juncture, understanding and building statistical models can help predict players’ and teams’ performances in tournaments, particularly during high-stakes events where prize pools are expansive. Findings from the current study can better inform players, teams, oddsmakers, and consumers on how to plan respective in-game strategies and external wagers. Thus, we present a unique application of various Multi-Criteria Decision-Making (MCDM) techniques to eSports performance modelling using data obtained from the MOBA game, Dota 2. The remainder of the current paper is organised as follows: In section 2, a concise review of the literature and related scholarship is presented. A description of the data and relevant statistics are provided in section 3. The results of the current study are summarised in section 4. Finally, section 5 offers conclusions, recommendations, and directions for future work.

2Literature review

2.1MCDM and related techniques

Within operations research and the decision-making sciences, MCDM has emerged as a prevailing technique in providing solutions to a variety of issues involving multiple, even divergent, objectives (Triantaphyllou, Shu, Sanchez, & Ray, 1998). MCDM deals with a variety of decision problems in the presence of numerous criteria as a means to arrive at the ideal solution involving a series of alternatives. According to Triantaphyllou et al. (1998), there are three primary steps when numerically analysing and deciding between alternatives: 1) Collect the relevant criteria and alternatives; 2) Assign numerical weights to selected criteria; and 3) Compute numerical values to provide a ranking of each alternative. Scholars of MCDM have offered numerous methods to rank alternatives, such as analytic hierarchy process (AHP), ELimination Et Choix Traduisant la REalit e´ (ELimination and Choice Expressing Reality; ELECTRE), simple additive weighting (SAW), grey relational analysis (GRA), Technique for Order Preference by Similarity to the Ideal Solution (TOPSIS), among many others (Triantaphyllou, 2013). As Triantaphyllou (2000, p. 5) states: “with the continuing proliferation of decision methods and their variants, it is important to understand their comparative value.” Considering these assertions by Triantaphyllou (2000), several MCDM methods are expanded upon.

2.1.1Grey relational analysis

Deng (1982) proposed that solutions to problems may not always be easily attained, given that researchers typically do not have all the information required to resolve an inquiry. Thus, GRA was developed to aid in the MCDM process by utilising the available information to reach an ideal solution. The process of GRA involves first generating a matrix containing available data. The data is then normalised by a selected reference series according to the ideal direction of each performance indicator. For instance, indicators where higher values signal better performance (e.g., kills in Dota 2) utilise the highest data point as the reference series, while those in which lower values denote stronger play (e.g., deaths) employ the lowest value among the chosen indicator. Following an absolute value transformation, grey relational coefficients and grades are generated and then ranked in order to determine the ideal solution (see Kuo, Yang, & Huang, 2008 for a full description of GRA). In a previous study, Kuo et al. (2008) have illustrated the suitability and parsimony of the GRA method in calculations and optimisation in decision-making. GRA has been employed in a variety of different fields, such as in fire safety assessment (Yang, Qian, & Huang, 2012), product life strategy (Chan & Tong, 2007), and genetic algorithms (Huang, Chiu, & Chen, 2008).

Moreover, GRA has been utilised in many contexts within sports-based research, such as in ranking teams and individual athletes in the NFL (Chen et al., 2010) and NBA (Chen, Huang, Lin, Lin, & Kuo, 2006). More specifically, research by Sakinç (2014) assessed Turkish association football clubs, using GRA to rank the financial performance of four major clubs (i.e., Beşiktaş, Fenerbahçe, Galatasaray and Trabzonspor) listed on the Borsa İstanbul (BIST) from the period of 2009 to 2013. Using a variety of both traditional and advanced statistics from the NBA, Pradhan (2018) created a structured ranking system of NBA players in the Modern Era using GRA.

2.1.2Simple additive weighting

Much like GRA, SAW was developed by Hwang and Yoon (1981) as a way to solve MCDM problems. The underlying logic of SAW involves a similar sequence of calculations that produce standardised scores based on the ideal direction (i.e., higher or lower) of each performance indicator. Often, weights for these standardised scores are then selected based on the importance of each criterion. These scores are then summed to create weighted averages, which are then ranked in determining the ideal solution (Afshari, Mojahed, & Yusuff, 2010).

The usage of SAW has been widespread, having been previously exercised in circumstances assessing supply chain risk within pharmaceutical systems (Jaberidoost et al., 2015), the effects of cocoa combinations on various human sensory parameters (Dogan, Aktar, Toker, & Tatlisu, 2015), and wind energy resource sites (Georgiou, Polatidis, & Haralambopoulos, 2012). Although the scholarly application of SAW to sports has been very limited, it has been applied in dance member evaluations by Adela, Jasmi, Basiron, Huda, and Maseleno (2018). Researchers in this study examined several pre-selected criteria related to dance (e.g., physical flexibility, skill, deftness, and the like). Using SAW, Adela and colleagues provide findings that could aid in the selection of the “best” dancers for outlets such as professional organisations and schools.

2.1.3Technique for order preference by similarity to an ideal solution

Originally conceived by Hwang and Yoon (1981) and advanced further by Yoon (1987) along with Hwang, Lai, and Liu (1993), the TOPSIS approach attempts to locate the ideal solution by parsing out the best and worst alternatives using vector-type normalisation methods. In doing so, TOPSIS produces scores that can be ranked to determine the ideal solution of a given MCDM problem, akin to GRA and SAW. For instance, across multiple studies in the context of sports, Chen and colleagues (2012, 2014) developed an evaluation model using both AHP and TOPSIS to aid in the managerial decision-making of the optimal selection of a pitcher within Chinese professional baseball. Chen et al. employed AHP to determine patterns in starting pitcher selection and created numerical weights for the selected performance indicators. To this end, both AHP and TOPSIS were utilised in the final determination of pitcher rankings. Further research by Dadelo, Turskis, Zavadskas, and Dadeliene (2014) has also employed TOPSIS in the development of a player ranking system integrating several measured attributes, such as objective performance indicators as well as expert judgment assessments submitted by coaches, within basketball. Likewise, TOPSIS has been applied to situations involving ranking the performance of German Bundesliga association football teams (Kiani Mavi, Kiani Mavi, & Kiani, 2012), as well as various urban engineering applications (Oz, Mete, Serin, & Gul, 2018; Yuvaraj & Pradeep Kumar, 2015).

2.2Purpose

There are a vast number of methods implemented for sports ranking today. For example, Barrow, Drayer, Elliott, Gaut, and Osting (2013) estimate there are 101 different ranking techniques used in the field. ESPN, one of the largest global distributors of sports media, has compiled a library of different statistics to rate players and teams, such as Player Efficiency Rating (PER), Quarterback Rating (QBR), Basketball Power Index (BPI), and many others (Katz, 2015). Sports ranking techniques employ a variety of approaches for valuing teams, but one of the most prevalent methods is termed power ranking. This technique computes a team’s strength relative to other teams in the same league. Typically, these ratings are expressed as ranks, with the best teams being placed first overall. Within the literature, there are scores of studies that have evaluated different ranking methods and their predictive validity (Barrow et al., 2013; Burer, 2012; Langville & Meyer, 2012).

Performance analysis in eSports appears to be similar to modelling performance in traditional sports (e.g., soccer, basketball, baseball, and the like), as each rely on features related to the athletes involved and the game itself, wherein players’ performances impact the outcome. In this context, the purpose of our study is to gain a more comprehensive understanding of the underlying factors that determine various eSports teams’ performances within the realm of the popular MOBA game, Dota 2. eSports has a great advantage for game analytics over classic sport games due to the granularity in providing structured information about matches that are held online (Wagner, 2006). The use of MCDM techniques is also not foreign to the realm of eSports, or Dota 2 for that matter. Aryanata, Rahadi, and Sudarmojo (2017) utilised AHP in predicting performance of Dota 2 matches. The authors employed several metrics to predict match results, such as experience per minute, gold per minute, matchmaking ratio points, outcomes of the last three head-to-head matches, and the overall results of the last 10 matches. Findings from their AHP analysis provided accurate depictions of match prediction using the selected a priori parameters. Thus, our study seeks to investigate the influence of various criteria on Dota 2 teams’ performance by applying several MCDM models to offer insights into the performance analysis of eSports.

3Method

3.1Data collection and performance indicators

Data for the 2017–2018 Dota 2 Professional Circuit were collected from two primary sources, DOTABUFF (https://www.dotabuff.com) and datDota (https://www.datdota.com). We utilised a series of performance indicators in structuring our power ranking system for each team. Specifically, we employed all available, composite player statistics from the public Dota 2 community statistical database, DOTABUFF. In all, these comprised a total of 17 variables, which included: 1) number of matches played, 2) winning percentage, 3) kill-death-assist ratio, 4) kills, 5) deaths, 6) assists, 7) last hits, 8) denies, 9) gold per minute, 10) experience per minute, 11) points, 12) majors won, 13) majors qualified, 14) minors won, 15) minors qualified, 16) match wins, and 17) match losses. These statistics were compiled to create team averages for the selected metrics. All players who participated during the 2017-2018 Dota 2 ranked season were included in our analysis.

We also included additional indicators of team strength obtained from the professional Dota 2 statistical catalogue, datDota.

Specifically, we incorporated team ratings as determined by Elo (k = 32) and Elo (k = 64), as well as Glicko-1 and Glicko-2 methods. Elo ratings, developed by Arpad Elo (1961) in the context of chess, are computed using the results of a previous number of (k) matches. These ratings have been widely used in the context of sports in evaluating team strength ratings and assessing outcome predictions of NBA, NFL, MLB, and NHL franchises (FiveThirtyEight, 2019; Karminsky & Polozov, 2016). More recently, Glicko-1 ratings conceived by Glickman (1995) and described as a “special case of Elo” (p. 1) have also been implemented into the purview of sporting events. The primary differentiator of Glicko ratings is the accounting of a team’s standard deviation, termed ratings deviation (Glickman, 1995). The second iteration of these ratings, namely Glicko-2 ratings, attempts to further improve upon predictive capability by factoring in a team’s volatility (i.e., likelihood of inconsistent performance; Glickman, 2013). Teams with no earnings and where team strength data were unavailable were excluded from our dataset. A list of all 21 performance indicators used for our team ranking system is included in Table 1. A summary of statistics obtained from each player, as well as team strength markers are provided in Table 2 and illustrated in Fig. 1.

Table 1

Performance indicators used in the present study

Performance indicatorAbbreviationIdeal directionSource
MatchesMATHigherDOTABUFF
Winning PercentageWIN%Higher
Kill Death Assist RatioKDAHigher
KillsKHigher
DeathsDLower
AssistsAHigher
Last HitsLHHigher
DeniesDNHigher
Gold Per MinuteGPMHigher
Experience Per MinuteXPMHigher
PointsPTSHigher
Majors WonMJWHigher
Majors QualifiedMJQHigher
Minors WonMINWHigher
Minors QualifiedMINQHigher
Match WinsMATWHigher
Match LossesMATLLower
Elo Rating (k = 32)ELO32Higherdatdota
Elo Rating (k = 64)ELO64Higher
Glicko-1 RatingGLICKO1Higher
Glicko-2 RatingGLICKO2Higher
Table 2

Average performance statistics of included teams

TeamMATWIN%KDAKDALHDNGPMXPMPTSMJWMJQMINWMINQMATWMATLELO32ELO64GLICKO1GLICKO2
Virtus.pro85.710.684.184.954.8712.96133.048.76395.14445.8612,375.004814113601293.211456.722016.381977.45
Team Liquid111.670.603.284.965.3111.57126.147.85394.17417.839,461.301835127651028.951032.221233.161619.46
PSG.LGD97.000.393.084.385.3310.69152.497.75391.50441.707,266.00250598691171.371252.931867.961860.69
Team Secret65.000.583.304.295.5311.77212.378.87450.78485.785,136.001827104791303.051480.162027.61965.43
Mineski120.830.563.584.804.6611.21182.188.13436.83486.503,150.0017168175950.37923.02685.281331.97
Vici Gaming174.000.583.725.234.9112.23170.689.10414.20463.802,835.00060584731143.511209.531852.771882.96
Newbee128.570.483.194.035.5111.21133.146.67373.86416.292,445.00092691871001.29993.081527.491700.85
VGJ Storm76.670.684.365.844.4712.57195.289.72463.33503.172,010.00021142271157.931168.251823.721898.36
VGJ Thunder179.170.613.694.514.4311.30138.705.65396.00446.171,935.00030438481041.281069.181644.41798.12
OpTic Gaming139.290.623.484.194.8911.96131.906.25386.14435.431,800.00051253481037.321030.611247.921572.85
Evil Geniuses70.500.573.383.825.1510.65118.507.01371.10415.001,335.00071680571028.221018.521602.771825.56
Fnatic137.860.583.764.434.6111.90172.178.26405.29465.571,267.50040757611129.731171.851638.271832.63
Natus Vincere66.670.382.734.156.0211.29124.247.23350.44393.111,200.00050634551159.841247.921841.971863.84
paiN Gaming55.710.634.105.974.7411.49175.039.63445.93492.00675040218421088.161143.061637.791818.91
Winstrike75.000.684.166.315.1714.10169.219.80452.00481.80675010122251129.431213.811530.571806.05
TNC Predator140.000.573.755.084.9312.06171.897.87412.67458.17495040245451159.421196.261841.81848.55
OG116.430.624.025.274.3311.00178.3211.43449.29485.00420061570581122.961178.381839.611864.38
KEEN GAMING87.000.564.355.674.2811.13165.5410.12427.50465.60270020110181075.281150.61470.641716.38
LGD.Forever Young59.380.413.304.114.6010.57140.869.51381.25433.38200020318211014.391056.571457.761735.46
compLexity Gaming95.710.523.334.104.7811.18122.767.50376.57420.86135030630451027.421034.021391.231752.79
Immortals60.630.543.324.664.639.80158.308.30429.50463.639000037121031.381067.651321.091663.08
Infamous51.790.493.134.995.0810.00160.468.10438.07464.29900404938896.23813.991122.061494.61
Vega Squadron59.380.433.204.654.9810.21173.457.60415.75456.389002028241007.341019.71294.841672.63
Team Kinguin92.140.563.073.864.9310.42132.956.94370.43413.003001041624964.68969.091272.921672.33
Digital Chaos44.000.373.014.154.969.63132.827.30390.10421.900000114977.18983.781020.991591.32
Effect35.000.483.815.264.6611.82179.3711.42458.25467.750020019987.56977.151125.31605.53
GangSquad63.330.473.175.234.9610.02154.177.24425.56459.560000113952.62922.82976.361484.2
iG.Vitality45.000.503.234.954.779.71184.498.57438.75474.750010036993.46998.031386.31645.18
INVICTUS GAMING80.710.594.745.373.6911.24180.7711.31443.71499.290020011251049.721112.421528.041750.03
Rex Regum Qeon37.500.553.314.795.0711.33164.306.73414.17456.0000001041123.291230.471672.691701.33
SG e-sports team88.570.633.553.924.309.47124.087.11374.86394.430010610281302.111417.911992.021969.24
Team Empire60.500.583.815.164.8612.10192.3810.92440.40493.00001038241002.82975.611186.481580.99
Team Serenity13.750.342.213.564.917.57124.358.88384.63423.75000008111195.661311.041887.61908.59
Team. Spirit68.750.543.885.674.5110.97166.029.21431.63470.3800200891055.521072.431690.831710.32
The Final Tribe127.000.584.175.454.4312.08172.198.75431.60476.20001029171013.53992.421526.971659.59
Fig. 1

Ridgeline plots showing the distributions of standardised team statistics for the 2017–2018 Dota 2 Professional Circuit season. Black lines represent the median scores.

Ridgeline plots showing the distributions of standardised team statistics for the 2017–2018 Dota 2 Professional Circuit season. Black lines represent the median scores.

3.2Statistical analysis

To create our power ranking of Dota 2 teams, we used a revised version of the hybrid MCDM model proposed by Wang, Zhu, and Wang (2016). As mentioned previously, MCDM employs a series of performance attributes to determine the ideal solution out of a number of selected alternatives (Rolland, 2006). This is accomplished by aggregating the chosen criteria to produce a single numerical value that represents the ideal solution (Vu & Li, 2014). These values for alternatives are then ranked in MCDM to select the best alternative (Martinez & Herrera, 2006). Wang et al. have argued that a method that combines several MCDM methods, namely GRA, SAW, and TOPSIS, tends to produce a more accurate depiction and subsequent ranking of the alternatives presented. Wang et al.’s model, which brings together the aforementioned techniques, has been shown to produce similar outcomes as other singular forms of MCDM, such as AHP, ELECTRE, multi-objective optimisation on the basis of ratio analysis (MOORA), along with GRA, SAW, and TOPSIS among others. In the present study, we simplified Wang et al.’s hybrid model by only using the average numerical scores obtained from the output of the GRA, SAW, and TOPSIS analyses. This method of analysis was performed since we did not assign any specific weights to the performance indicators (i.e., all performance statistics were treated equally). A summary of each of the selected MCDM procedures is provided in Table 3.

Table 3

Review of MCDM Methods Used in Wang et al. (2016) Hybrid Model

MethodStepFormulaDescription
Grey relational1. Compile data matrix X = [xij] m×n, W = [wj] n X= Decision matrix
  analysis (GRA)1m= Alternatives
xij= Value for ith alternative of selected
  jth performance criterion
n = Number of criteria
W= Weight
wj= Determined weight of specific criterion
2. Data normalisation xi*(k)+=xi(k)-kminx0(k)kmaxx0(k)-kminx0(k) xi*(k)+ = Normalised value of selected performance criterion
  in which higher values are better
xi*(k)+ = Normalised value of selected performance criterion
  in which lower values are better
kminx0(k) = Maximum value among selected
  performance criterion
kmaxx0(k) = Maximum value among selected
  performance criterion
3. Transform to absolute differences Δxi(k)=|x0*(k)-xi*(k)| Δxi (k)= Absolute value of kth performance
criterion
xi*(k) = Normalised value of selected performance criterion
ξi (k)= Grey relational coefficient for kth performance criterion
4. Calculate grey relational coefficients ξi(k)=Δmin+ζΔmaxΔxi(k)+ζΔmax Δmin= Minimum value of selected criterion
Δmax= Maximum value of selected criterion
ζ= Distinguishing coefficient (i.e., usually a value of 0.50)
5. Calculate grey relational gradesΓi = ∑ [ξi (k) wi (k)]Γi= Grey relational grade
ξi (k)= Grey relational coefficient
wi (k)= Inverse of the number of performance indicators
Simple additive1. Compile data matrix X = [xij] m×n, W = [wj] n X= Decision matrix
  weighting (SAW)2m= Alternatives
xij= Value for ith alternative of selected
  jth performance criterion
n= Number of criteria
W= Weight
wj= Determined weight of specific criterion
2. Data normalisation rij={xijxj+,j∈Ωmaxxj-xij,j∈Ωmin rij= Normalised value
xij= Selected performance criterion
xj+ = Maximum value for selected criterion
xj- = Minimum value for selected criterion
Ωmax= Benefit criteria (i.e., criterion in which higher values are better)
Ωmin= Cost criteria (i.e., criterion in which lower values are better)
3. Transform normalised to weighted valuesW = [wn]W= Weighted value
w= Determined weight
n= Selected criterion
4. Calculate SAW score Si=∑j=1nwjrij Si= SAW score
wj= Weight of jth criterion
rij= Normalised value
Technique for order1. Compile data matrix X = [xij] m×n, X= Decision matrix
  preference by similarity to an ideal solutionW = [wj] nm= Alternatives
xij= Value for ith alternative of selected jth performance criterion
  (TOPSIS)2n= Number of criteria
W= Weight
wj= Determined weight of specific criterion
2. Data normalisation rij=xij∑k=1mxkj2 ,rij= Normalised value of performance criterion
i = 1, …, m; j = 1, …, nxij= Value for ith alternative of selected jth performance criterion
3. Transform normalisedvij = wjrij,vij= Weighted value of performance
  to weighted valuesi = 1, …, m; j = 1, …, n   criterion
wj= Determined weight
rij= Normalised value of performance criterion
3. Determine positive/ A+={v1+,…vn+} A+= Positive ideal solution (i.e., criterion
  negative  in which higher values are better)
  ideal solutions for A-={v1-,…vn-} vn+ = Maximum value of jth criterion
  performance criterion5- = Negative ideal solution (i.e., criterion in which lower values are better)
vn- = Minimum value of jth criterion
4. Calculate distance Di+=∑j=1n(vij-vj+)2 , Di+ = Distance between ith alternative
  from ideal solutioni = 1, …, m  and positive ideal solution
Di-=∑j=1n(vij-vj-)2 , Di- = Distance between ith alternative and negative ideal solution
i = 1, …, mvij= Weighted value of performance criterion
vj+ = Maximum value of jth criterion
vj- = Minimum value of jth criterion
5. Calculate TOPSIS Ci=Di-Di+-Di- Ci= TOPSIS score
  score Di+ = Distance between ith alternative and positive ideal solution
Di- = Distance between ith alternative and negative ideal solution

Note. 1Formulas and steps obtained from Pradhan (2018). 2Formulas and steps obtained from Wang et al. (2016).

4Results

Statistical analyses were performed in R Studio for macOS (Version 1.1.463) using the topsis package (Yazdi, 2013) and Microsoft Excel. Each MCDM model was generated separately and then aggregated using a modified version of Wang et al.’s (2016) procedure, as described previously, to produce the hybrid analysis. The full results of the hybrid MCDM analysis are presented in Table 4. Overall, the hybrid MCDM analysis ranked Virtus.pro as the top team during the 2017-18 Dota 2 Professional Circuit season, followed by Team Secret, Team Liquid, PSG.LGD, and OG, respectively. Digital Chaos was ranked as the last team in the hybrid analysis. Across each of the MCDM component analyses (i.e., GRA, SAW, and TOPSIS), Virtus.pro was ranked as the top team. Digital Chaos was categorised as the worst performer in all but the TOPSIS analysis, in which the team was ranked 34th (i.e., second-to-last; scoring slightly higher than the team, Vega Squadron, with a TOPSIS score of 19.31% compared to 18.74%). Among the actual placement of the teams based on seasonal performance, the hybrid model accurately identified the top four teams, albeit placing them in a different order apart from Virtus.pro.

Table 4

Hybrid MCDM ranking results for Dota 2 professional circuit 2017–2018 season

TeamGRA scoreGRA rankSAW scoreSAW rankTOPSIS scoreTOPSIS rankHybrid scoreHybrid rankActual season rank
Virtus.pro69.72%179.33%169.77%172.94%11
Team Liquid54.96%971.60%358.37%261.64%32
PSG.LGD51.35%1865.75%445.66%454.25%43
Team Secret68.22%275.58%248.95%364.25%24
Mineski51.58%1660.84%838.88%650.44%85
Vici Gaming56.95%764.59%632.43%751.32%66
Newbee49.23%2161.29%740.09%550.21%97
VGJ Storm62.95%360.35%1028.25%1250.52%78
VGJ Thunder50.42%2055.71%1526.78%1344.31%169
OpTic Gaming46.60%2654.50%1628.72%1043.27%1810
Evil Geniuses47.79%2457.13%1232.22%845.71%1111
Fnatic53.91%1160.40%928.69%1147.67%1012
Natus Vincere46.38%2752.79%2124.50%1641.22%2313
paiN Gaming55.10%855.89%1321.41%2344.14%17T-14
Winstrike58.92%555.80%1422.12%2045.61%12T-14
TNC Predator52.32%1357.29%1124.66%1444.76%1316
OG60.34%464.76%532.15%952.41%517
KEEN GAMING52.50%1252.83%2022.13%1942.49%2018
LGD.Forever Young44.95%3048.17%2720.59%2937.90%2919
omplexity Gaming45.39%2951.52%2323.25%1840.05%2520
Immortals46.97%2548.00%2820.91%2738.63%27T-21
Infamous44.45%3348.29%2619.98%3237.57%30T-21
Vega Squadron44.52%3247.18%3018.74%3536.81%32T-21
Team Kinguin43.02%3447.02%3120.94%2637.00%3124
Digital Chaos41.54%3540.86%3519.31%3433.90%35T-25
Effect51.56%1748.34%2521.04%2440.31%24T-25
GangSquad44.62%3143.72%3320.35%3136.23%33T-25
iG.Vitality47.81%2346.39%3220.45%3038.22%28T-25
INVICTUS GAMING58.52%653.63%1721.76%2244.64%14T-25
Rex Regum Qeon48.24%2247.41%2921.03%2538.89%26T-25
SG e-sports team54.93%1053.51%1824.55%1544.33%15T-25
Team Empire52.00%1551.78%2220.83%2841.53%21T-25
Team Serenity46.12%2842.54%3419.51%3336.06%34T-25
Team. Spirit51.21%1950.70%2422.09%2141.34%22T-25
The Final Tribe52.02%1453.00%1923.28%1742.77%19T-25

Supplementary analyses. To validate the results of the rankings generated by the hybrid MCDM analysis, we performed a series of Spearman rank-order correlations. These tests correlated the standings yielded by the hybrid model and the singular GRA, SAW, and TOPSIS rankings. We also sought to examine the association between the rankings produced by common team strength performance indicators, such as Elo (k = 32), Elo (k = 64), Glicko-1, and Glicko-2 ratings. Results from the Spearman rank-order correlations supported the validity of our findings (all Spearman rho [ρ] values > 0.45, all p values < 0.01; see Fig. 2). Follow-up Wilcoxon signed-rank tests with continuity correction were performed to further examine differences between the ranks produced by the hybrid MCDM method and the previously examined rankings. These tests did not reveal any significant differences arising from the use of the hybrid model, suggesting consistency with the other forms of ranking (all p values > 0.05; see Table 5).

Fig. 2

Spearman rank-order correlations between hybrid rankings and rankings produced by other MCDM methods and selected performance indicators. R values refer to Spearman Rs or rho (ρ) values. Coloured lines represent trend lines. Grey shaded regions display 95% confidence intervals.

Spearman rank-order correlations between hybrid rankings and rankings produced by other MCDM methods and selected performance indicators. R values refer to Spearman Rs or rho (ρ) values. Coloured lines represent trend lines. Grey shaded regions display 95% confidence intervals.
Table 5

Wilcoxon Signed-rank tests comparing hybrid MCDM model rankings and rankings produced by other methods and selected performance indicators

Ranking methodWilcoxon Vp95% CI
GRA250.500.72[–2.50, 3.00]
SAW126.001.00[–1.00, 1.50]
TOPSIS225.000.88[–1.50, 1.50]
Elo (k = 32)277.500.96[–3.00, 3.00]
Elo (k = 64)311.500.96[3.50, 3.00]
Glicko-1301.500.95[–2.50, 3.00]
Glicko-2294.000.82[–2.50, 3.00]

Note. n = 35. Wilcoxon V-statistic refers to the sum of positive ranks. 95% CI refers to confidence intervals based on Walsh averages. For further description of V values and Walsh averages, see Geyer (2003).

5Discussion

This study aimed at crafting a power ranking model for seasonal performance in the popular MOBA video game, Dota 2. Specifically, we employed a modified version of Wang et al.’s (2016) MCDM hybrid model, which utilised a combination of GRA, SAW, and TOPSIS, to rank teams during the 2017-18 Dota 2 Professional Circuit season. Results from the hybrid analysis are consistent with other forms of performance evaluation, such as Elo and Glicko ratings. With respect to top performing teams, our ranking system offered an accurate depiction of actual season standings. Nevertheless, our study did have several limitations.

Compared to physical sports, eSports prediction involves even more information, as the selection of characters could also be a factor that influences game results. For instance, each character in the game will have its own abilities, which could lead to different competitive advantages and disadvantages due to specific match-ups (Yang, Qin, & Lei, 2016). While the current study did not consider the impact of character selection, future research could examine how such choices influence eSports team performance. Although the present study utilised data from a MOBA-type game, there are other genres that might require different forms of power rankings based on their characteristics. For instance, games like Counter-Strike rely on other technical abilities as opposed to characters that one selects, such as the ability of players to know which weapons to use, how to control them, and where to be on the map relative to their teammates and opponents. These sets of skills involved in effective performance are what differentiates such titles from MOBA games like Dota 2.

Power ranking brings in many new opportunities to assist in detecting which players and teams have the most potential for performance, as well as marketability. In comparison to traditional sports athletes in leagues, such as the NBA, eSports athletes are quite different from a content standpoint. For example, LeBron James of the Los Angeles Lakers or Stephen Curry of the Golden State Warriors needed to first develop their performance on the court before acquiring sponsors and fans. In eSports, both performance and marketability content can be created equal. In many instances, an eSports athlete might have a better career streaming their games via Twitch and other related sites compared to competing and winning prize money with their team in league play. An eSports player does not have to wait to win titles in order to monetise their skills within games. In essence, the community defines what is relevant by promoting exciting content.

To that end, an influx of professional teams from traditional sports have joined the eSports market to further their already sizeable investments. The Sacramento Kings, Golden State Warriors, Philadelphia 76ers, Cleveland Cavaliers, and Houston Rockets have all endorsed their own LoL teams (Chikhani, 2018). As eSports continue to evolve, practitioners face managerial challenges that are similar to those in traditional sport, particularly in areas of in-game performance. Forecasting performance plays an important role in sports, with nearly every major professional sports team containing an analytics department. These analysts use player statistics to predict future games and attempt to enhance performance through draft selections or roster adjustments.

As Makarov, Savostyanov, Litvyakov, and Ignatov (2018) assert, the rapid maturity of eSports serves as a guide to gambling businesses and bookmakers that will inevitably turn their attention to such events. With thousands of bookmakers accepting wagers on sporting events around the world today, gambling in sports is more popular than ever before. For example, during the National Collegiate Athletic Association’s (NCAA) Basketball Tournament, March Madness, the American Gaming Association estimated that approximately 50 million Americans wagered $8.5 billion in 2019 (Axson, 2019). Similarly, eSports bookmakers need access to data in order to compile statistics to accurately set odds and make match predictions that facilitate gambling in real time. Several other issues identified by Grove (2016) also become apparent in the regulation of eSports gambling, such as specifically how to enforce a global industry like eSports, how skin gambling and in-game casinos will be treated, the involvement of minors in eSports, and equating eSports betting to traditional sports gambling.

In general, power rankings can apply to many sports, but may also be used for teams playing in specific tournaments, such as university championships. With the massive success of the eSports market, ESPN has begun broadcasting the Collegiate eSports Championship (CEC). Various college teams will compete in games like Overwatch, Hearthstone, among others (Wilson, 2019). There will also be a corollary pre-game show, called College eSports Championship: Countdown, that will air weekly. Consumers will be able to tune in through ESPN3, Twitch, YouTube, and Twitter. Considering the variety of other titles, the results of this study could also be adapted to other games in eSports. Future investigations could examine the relationships between the use of different parameters and actual championship results through Wang et al.’s (2016) hybrid model and other MCDM techniques.

In the light of recent development in the gambling industry, such inputs can help bookmakers and sponsors better assess the profitability of eSports athletes and increase brand visibility. For sponsorship and endorsements, in particular, such a data driven approach can further assist in assessing the potential return on interest (ROI) for different eSports teams. Moreover, these findings may even aid in team contracts and endorsement deals, in which eSports teams and sponsors can utilise a combination of MCDM techniques to compare and contrast players. eSports athletes careers often begin at a very early age and thus, accurate power rankings can potentially redefine the way such deals are negotiated in the long run. Organisations may even be able to craft well-informed prize pools based on in-game predictive performance. Such analyses are also vital for a team, its players, and management in terms of developing tactics and increasing performance. We hope our study provides future directions for research and insights for practitioners in guiding both in-game strategy and understanding Dota 2 performance.

References

1 

Afshari, A , Mojahed, M , & Yusuff, R.M. , (2010) , Simple additive weighting approach to personnel selection problem. International Journal of Innovation, Management and Technology, 1: (5), 511–515.

2 

Aryanata, G.A , Rahadi, P.S.A.D , & Sudarmojo, Y.P. , (2017) , Prediction of DOTA 2 match result by using analytical hierarchy process method. International Journal of Engineering and Emerging Technology, 2: (1), 22–25.

3 

American Gaming Association: $8.5 billion expected to be bet on NCAA Tournament. Retrieved from https://www.si.com/college-basketball/2019/03/18/ncaa-brackets-gambling-march-madness

4 

Barrow, D , Drayer, I , Elliott, P , Gaut, G , & Osting, B. , (2013) , Ranking rankings: An empirical comparison of the predictive power of sports ranking methods. Journal of Quantitative Analysis in Sports, 9: (2), 187–202.

5 

Bountie Gaming, 2018, The history and evolution of Esports. Retrieved from https://medium.com/@BountieGaming/the-history-and-evolution-of-esports-8ab6c1cf3257

6 

Burer, S. , (2012) , Robust rankings for college football. Journal of Quantitative Analysis in Sports, 8: (2). doi: 10.1515/1559-0410.1405.

7 

Chen, C.C , Lee, Y.T , & Tsai, C.M. , (2014) , Professional baseball team starting pitcher selection using AHP and TOPSIS methods. International Journal of Performance Analysis in Sport, 14: (2), 545–563.

8 

Chen, C.C , Lin, M.L , Lee, Y.T , Chen, T.T , & Huang, C.L. , (2012) , Selection best starting pitcher of the Chinese professional baseball league in 2010 using AHP and TOPSIS methods. In Frontiers in Computer Education (pp. 643–649). Berlin, DE: Springer.

9 

Chen, K.C , Huang, L.F , Hung, C.Y , Chen, Y.C , Wu, H.T , & Huang, W.K. , (2010) , Fight evaluation and managerial efficiency evaluation of NFL teams. Journal of Statistics and Management Systems, 13: (2), 309–325.

10 

Chen, K.C , Huang, L.F , Lin, H.Y , Lin, S.C , & Kuo, J.C. , (2006) , Fight evaluation of NBA teams. Journal of Information & Optimization Sciences, 27: (2), 481–498.

11 

Chikhani, R. , (2018) , The 3 types of esports investor: Sports teams, celebrities, and moguls. Retrieved from https://venturebeat.com/2018/05/15/the-3-types-of-esports-investor-sports-teams-celebrities-and-moguls/

12 

Dadelo, S , Turskis, Z , Zavadskas, E.K , & Dadeliene, R. , (2014) , Multi-criteria assessment and ranking system of sport team formation based on objective-measured values of criteria set. Expert Systems with Applications, 41: (14), 6106–6113.

13 

datDota, 2018, Ratings overview. Retrieved from https://www.datDota.com/ratings

14 

Deng, J. , (1982) , Control problems of grey systems. System and Control Letters, 5: , 288–294.

15 

Dogan, M , Aktar, T , Toker, O.S , & Tatlisu, N.B. , (2015) , Combination of the simple additive (saw) approach and mixture design to determine optimum cocoa combination of the hot chocolate beverage. International Journal of Food Properties, 18: (8), 1677–1692.

16 

DOTABUFF, 2018, Dota Pro Circuit 2017–2018 season. Retrieved from https://www.Dotabuff.com/procircuit/2017-2018

17 

Elo, A.E. , (1961) , The new U.S.C.F. rating system. Chess Life, 16: , 160–161.

18 

FiveThirtyEight, 2019, Elo ratings. Retrieved from https://fivethirtyeight.com/tag/elo-ratings/

19 

Gaston, M. , (2013) , Dota 2 review. Retrieved from https://www.gamespot.com/reviews/dota-2-review/1900-6411615/

20 

Gaudiosi, J. , (2012) , Riot Games’ League of Legends officially becomes most played PC game in the world. Retrieved from https://www.forbes.com/sites/johngaudiosi/2012/07/11/riot-games-league-of-legends-officially-becomes-most-played-pc-game-in-the-world/#572b3d0b718b

21 

Georgiou, A , Polatidis, H , & Haralambopoulos, D. , (2012) , Wind energy resource assessment and development: Decision analysis for site evaluation and application. Energy Sources, Part A: Recovery, Utilization, and Environmental Effects, 34: (19), 1759–1767.

22 

Geyer, C. , (2003) , Stat 5102 notes: Nonparametric tests and confidence intervals. Retrieved from http://www.stat.umn.edu/geyer/old03/5102/notes/rank.pdf

23 

Glickman, M.E. , (1995) , The Glicko system. Boston University: Boston, MA.

24 

Glickman, M.E. , (2013) , Example of the Glicko-2 system. Boston University: Boston, MA.

25 

Grove, C. , (2016) , Esports & gambling: Where’s the action? Irvine, CA: Eilers & Krejcik Gaming.

26 

Hwang, C.L , & Yoon, K. , (1981) , Methods for multiple attribute decision-making. In Multiple attribute decision-making (pp. 58–191). Berlin, DE: Springer.

27 

Hwang, C.L , Lai, Y.J , & Liu, T.Y. , (1993) , A new approach for multiple objective decision-making. Computers and Operational Research, 20: , 889–899.

28 

Ingraham, C. , (2018) , The massive popularity of esports, in charts. Retrieved from https://www.washingtonpost.com/business/2018/08/27/massive-popularity-esports-charts/?noredirect=on&utm term=.b02a82eead79

29 

Jaberidoost, M , Olfat, L , Hosseini, A , Kebriaeezadeh, A , Abdollahi, M , Alaeddini, M , & Dinarvand, R , (2015) , Pharmaceutical supply chain risk assessment in Iran using analytic hierarchy process (AHP) and simple additive weighting (SAW) methods. Journal of Pharmaceutical Policy and Practice, 8: (9), 1–10.

30 

Karminsky, A , & Polozov, A. , (2016) , Handbook of ratings. Cham, CH: Springer.

31 

Katz, S., 2015, How ESPN’s NFL Football Power Index was developed. Retrieved from http://www.espn.com/nfl/story/_/id/13539941/how-espn-nfl-football-power-index-was-developed-implemented

32 

Kiani Mavi, R , Kiani Mavi, N , & Kiani, L. , (2012) , Ranking football teams with AHP and TOPSIS methods. International Journal of Decision Sciences, Risk and Management, 4: (1–2), 108–126.

33 

Kuo, Y , Yang, T , & Huang, G.W. , (2008) , The use of grey relational analysis in solving multiple attribute decision-making problems. Computers & Industrial Engineering, 55: (1), 80–93.

34 

Langville, A.N , & Meyer, C.D. , (2012) , Who’s# 1?: The science of rating and ranking. Princeton, NJ: Princeton University Press.

35 

Makarov, I , Savostyanov, D , Litvyakov, B , & Ignatov, D.I. , (2017) , Predicting winning team and probabilistic ratings in “Dota 2” and “Counter-Strike: Global Offensive” video games. In International Conference on Analysis of Images, Social Networks and Texts (pp. 183–196). Cham, CH: Springer.

36 

Martínez, L , Pérez, L.G , Liu, J , Yang, J.B , & Herrera, F. , (2006) , A linguistic hierarchical evaluation model for engineering systems. In B. Bouchon-Meunier, G. Coletti, & R.R. Yager (Eds.), Modern information processing (pp. 295–305). Amsterdam, NL: Elsevier Science.

37 

Mathewson, T.J. , (2019) , TV is biggest driver in global sport league revenue. Retrieved from https://globalsportmatters.com/business/2019/03/07/tv-is-biggest-driver-in-global-sport-league-revenue/

38 

Oz, N.E , Mete, S , Serin, F , & Gul, M. , (2018) , Risk assessment for clearing and grading process of a natural gas pipeline project: An extended TOPSIS model with Pythagorean fuzzy sets for prioritizing hazards. Human and Ecological Risk Assessment: An International Journal, 1–18.

39 

Pannekeet, J. , (2018) , Newzoo: Global Esports economy will reach $905.6 million in 2018 as brand investment grows by 48%. Retrieved from https://newzoo.com/insights/articles/newzoo-global-esports-economy-will-reach-905-6-million-2018-brand-investment-grows-48/

40 

Pannekeet, J. , (2019) , Newzoo: Global Esports economy will top $1 billion for the first time in 2019. Retrieved from https://newzoo.com/insights/articles/newzoo-global-esports-economy-will-top-1-billion-for-the-first-time-in-2019/

41 

Pradhan, S. , (2018) , Ranking regular seasons in the NBA’s Modern Era using grey relational analysis. Journal of Sports Analytics, 4: (1), 31–63.

42 

Rolland, A. , (2006) , On bi-capacity-based concordance rules in multicriteria decision-making. In B. Bouchon-Meunier, G. Coletti, & R.R. Yager (Eds.), Modern information processing (pp. 231–243). Amsterdam, NL: Elsevier Science.

43 

Sakinç, I. , (2014) , Using grey relational analysis to determine the financial performance of Turkish football clubs. Journal of Economics Library, 1: (1), 22–33.

44 

Statista, 2018, eSports audience size worldwide from 2012 to 2022, by type of viewers (in millions). Retrieved from https://www.statista.com/statistics/490480/global-esports-audience-size-viewer-type/

45 

Triantaphyllou, E. , (2000) , Multi-criteria decision-making methods. In Multi-criteria decision-making methods: A comparative study (pp. 5–21). Boston, MA: Springer.

46 

Triantaphyllou, E , Shu, B , Sanchez, S.N , & Ray, T. , (1998) , Multi-criteria decision-making: An operations research approach. Encyclopaedia of Electrical and Electronics Engineering, 15: , 175–186.

47 

Vu, H.G , Beliakov, G , & Li, G. , (2014) , A Choquet integral tool-box and its application in customer preference analysis. In Y. Zhao & Y. Cen (Eds.), Data mining applications with R (pp. 247–272), Boston, MA: Academic Press.

48 

Wagner, M. , (2006) , On the scientific relevance of eSport. In Proceedings of the 2006 International Conference on Internet Computing and Conference on Computer Game Development (pp. 437–470). Las Vegas, NV: CSREA Press.

49 

Wang, P , Zhu, Z , & Wang, Y. , (2016) , A novel hybrid MCDM model combining the SAW, TOPSIS and GRA methods based on experimental design. Information Sciences, 345: , 27–45.

50 

Wilson, O. , (2019) , Week one programming to air on various digital platforms, hosted by streamers Goldenboy and LeTigress. Retrieved from https://espnpressroom.com/us/press-releases/2019/03/espn-to-debut-weekly-cec-countdown-friday-march-29/

51 

Yang, Y , Qin, T , & Lei, Y.H. , (2016) , Real-time eSports match result prediction. ArXiv e-prints:1701.03162.

52 

Yazdi, M.M. , (2013) , topsis: TOPSIS method for multiple-criteria decision-making (MCDM). R package version 1.0.

53 

Yoon, K. , (1987) , A reconciliation among discrete compromise situations. Journal of Operational Research Society, 38: (3), 277–286.

54 

Yuvaraj, N , & Pradeep Kumar, M. , (2015) , Multiresponse optimization of abrasive water jet cutting process parameters using TOPSIS approach. Materials and Manufacturing Processes, 30: (7), 882–889.

Show more