The LMI is limited in its current setting as it uses only playrates and winrates. By using Bo3 data we propose a way to expand the LMI with a banindex
Composite Indicators (CI) are a quantitative measure that aggregate multidimensional data into a single index. Differently from other aggregation methods as Principal Components Analysis (PCA) or Factorial Analysis (FA) they are not entirely data driven and they are compiled in order to communicate a concept. Mostly used in social or policy evaluation, they allows for a single and direct comparison between their units. Gamers commonly and intuitively use this tool when talking about tier list. In a card game like Hearthstone (HS) a simple but known example of CI is the Meta Score from viciousSyndicate (vS).
In Legends of Runeterra (LoR) we created a similar CI defined as LoRMetaIndex (LMI)^{1}. The index didn’t just try to replicate the vS Meta Score but tried to adjust it to the LoR data and its differences from HS. A limitation of the proposed index is it’s composed by only two variables: playrates and winrates. While they are the most important variables of performance of a deck, they don’t fully catch the complexities on the metagame performances. This can be solved by adding other variables to the index and a natural candidate is the banrate of a deck (in a contest of BoX matches).
An experiment to add the banrate was done earlier this year, for ‘Rise of the Underworld  Seasonal Tournament’ report^{2}.
The inclusion of such variable, was done without checking all the proper steps so that we wouldn’t compromise the quality of the CI. In this article we introduce in more detail the concept and framework of a CI and how to add the the information of the ban rates and their results.
Following the necessary steps, the proposed variation of the LMI partially contradicts the expected theoreticalframework requiring an higher level of complexity to reach the aimed .
Data are taken from the Seasonal tournaments ‘Guardian of the Ancient’ and ‘Rise of the Underworld’ Open Rounds.
The Open Rounds are the are organized in a series of nine Bo3 Matches with open lists and a ban phase before the start of the games. The smaller amount of games from the Asian shard/server is allegedly because of the fewer players taking part of it^{3}
Not all information about the tournament can be derived from the API. There is no direct data about the chosen ban deck or the entire lineup brought by a player at this have to be extracted by aggregating the metadata of games from a single match and all matches played during that day.
We only consider the decks that appears in the cases of a fulllineup and whose we can extrapolate the banned deck.
To evaluate winrates on the ladder, as additional source, we includes Ranked games from Master players (at least one of the two players being Master) from up to two weeks before the start of the first game of the Seasonal (because of the time zone, two weeks, before the start of the Asian Seasonal).
Bo3 Data  

Matches by Server  
Characteristic  Seasonal  Guardian of the Ancient, N = 20,637^{1}  Seasonal  Rise of the Underworld, N = 20,511^{1} 
server  
americas  9,126 (44%)  9,018 (44%) 
asia  2,457 (12%)  2,421 (12%) 
europe  9,054 (44%)  9,072 (44%) 
Bo3 Data from Seasonal Open Rounds  Rise of the Underworld Open Rounds Matches  games extracted with Riot API  
^{
1
}
n (%)

Example and part of the rawdata used can be found in Appendix ??)
The LMI is a composite indicator (CI) and in a previous article we introduced the tool we gave a brief explanation about how to create them. Here, we want to give a better and more complete overview of the tool.
For a manual on CI, a commonly referred guide is from the Joint Research Centre (JRC) of the European Commission: Handbook on Constructing Composite Indicators  METHODOLOGY AND USER GUIDE. In that guide building a composite indicator requires the following 10 steps:
Provide the basis for the selection and combination of variables into a meaningful composite indicator under a fitnessforporpuse principle (involment of experts and stakeholders is envisaged at this step)
What is the concept that the CI wants to convey? Of what characteristics is it made and what variable can express them? This steps is all about setting the definition of what needs to be created.
Should be based on the analytical soundness, measurability, coverage and relevance of the indicators to the phenomenon being measured and relationship to each other. The use of proxy variables should be considered when data are scarce (involvement of experts and stakeholders is envisaged at this step)
When defining the CI structure there is also the need to maintain a coherent structure. This means, among other, that values in the same subdimension should all follows the same direction. In an increase of a variable imply an increase in the final value of the subdimension index then all the other variables should be same.
Example: if we have a subdimension index related to “quality of life” containing life expectancy and child mortality, the higher the value of life expectancy the better and higher the final index should be. But, for the values of child mortality it is the opposite, the smaller the value, the better it is. In this case it’s not a problem as a common practise is to just use the opposite values by changing the sign as it is a linear transformation and the smaller the value of child mortality (with opposite sign) the better it is in evalutating “quality of life”.
The ‘polarity’ of an individual indicator is the sign of the relation between the indicator and the concept to be measured. For example, in the case of wellbeing, “Life expectancy” has positive polarity, whereas “Unemployment rate” has negative polarity. I
Is needed in order to provide a complete dataset (e.g by means or multiple imputation).
In the real world rarely the available data are complete (and this article is no exception). If the missing value are MCAR (Missing Completely At Random) then there would be no problem to just remove the rows with missing values but this is not always the case and sometimes we can’t just remove units as they too important and so we must try to impute the missing values without compromising the end results or estimate these values to be as near as possible as what we would have obtain as the “true” value (not compromising the quality and trying to estimate the best imputation are not the same problem).
Dealing with outliers is also part of this step. In our case it’s related to the minimum amount of games to require to a deck.
Should be used to study the overall structure of the dataset, assess its suitability, and guide subsequent methodological choices (e.g. weighting aggregation).
Sometimes the framework is not enough to guide our decision and variables may actually behave differently from our exceptions. This step is about checking the underlying structure of the data, helps identify groups and evaluate statistically evaluated structure of the data set to the theoretical framework and discuss potential differences.
In this article this will be done when looking at the correlations among our variables.
Should be carried out to render the variables comparable
As this steps was invested with care in the previous article, we will maintain the choice for using a quantile normalization for our data.
While it’s possible to use different normalization for the same CI at different groups of variables, it’s not a recommended practise.
Should be done along the lines of the underying theoretical framework
This steps mostly intertwined with the the Multivariate analysis as it can highlight
Like the normalization step, this step too was was invested with care in the previous article and we will maintain the choice for using an harmonic mean when aggregating our data. It is possible to use different aggregation method for the same CI if it’s justified (like for compensability)
Should be undertaken to assess the robustness of the composite indicator in terms of e.g. the mechanism for indulging or excluding an indicator, the normalization scheme, the imputation of missing data, the choice, the choice of weights, the aggregation method
This is an highly complex step that will be left for future articles.
Is needed to reveal the main drivers for an overall good or bad performance. Trasparency is promordial to good analysis
As it can be done better with the results of a uncertainty and sensitivity analysis, this steps too will be left for the future.
Should be made to correlate the composite indicator (or its dimension) with existing (simple or composite) indicators as well as to identify linkages through regressions.
This can be done when comparing the result of Balco’s MetaScore which applies the same algorithm by vS but is not part of this article as we would also include the opinion of thirdparty experts (some high level players) to evaluate the different CI performances.
Should receive proper attention, given that the visualization can influence (or help to enhance interpretability)
While not the only choice we will maintain for now the same plot strcture used up until now in the metareports and the previous LMI article.
When creating for the first time a CI these 10 steps aren’t done in a strict sequential order, so in this article we will sometimes return to what is are previous steps compared to the one we are mainly referring in a section of the article.
The basic LMI is made by aggregating playrates and winrates and its creation was was inspired by seeing the meta score on vS.
While the base raw data are from the same concept, they are combined in a different way.
From their F.A.Q.
Q: What is the meaning of the Meta Score and how do you compute it?
The Meta Score is a supplementary metric that measures each archetype’s relative standing in the meta, based on both win rate and prevalence, and in comparison to the theoretical “best deck”.
How is it computed?
…
We take the highest win rate recorded by a current archetype in a specific rank group, and set it to a fixed value of 100. We then determine the fixed value of 0 by deducting the highest win rate from 100%. For example, if the highest win rate recorded is 53%, a win rate of 47% will be set as the fixed value of 0. This is a deck’s Power Score. The range of 47% – 53%, whose power score ranges from 0 to 100, will contain “viable” decks. The length of this range will vary depending on the current state of the meta. Needless to say, it is possible for a deck to have a negative power score, but it can never have a power score that exceeds 100.
We take the highest frequency recorded by a current archetype in a specific rank group, and set it to a fixed value of 100. The fixed value of 0 will then always be 0% popularity. This is a deck’s Frequency Score. A deck’s frequency score cannot be a negative number.
We calculate the simple average of a deck’s Power Score and Frequency Score to find its vS Meta Score. The vS Meta Score is a deck’s relative distance to the hypothetical strongest deck in the game. Think of Power Score and Frequency Score as the coordinates (x, y) of a deck within a Scatter Plot. The Meta Score represents its relative placement in the plane between the fixed values of (0, 0) and (100,100).
If a deck records both the highest popularity and the highest win rate, its Meta Score will be 100. It will be, undoubtedly, the best deck in the game.
While for the LMI:
Only decks with at least 200 games are considered. A similar filter is most likely being applied by vS too, we just don’t know the values for the cutoff
Playrates and winrates are normalized with a quantile normalization into a FreqIndex and a WinIndex
FreqIndex and a WinIndex are aggregated by an harmonic mean of equal weights into the LMI
The reasoning behind these choices compared to other options can be found in the LMI  early concept article.
As the number of variables increase from the two of the baseLMI to three we now have more options as to combine them. Normally, this doesn’t mean that each possible choice should be evaluated, the definition we want to communicate should guide our choices and so the characteristics of our variables, e.g. we don’t add a variable of Life expectancy in a subdimension of ‘Infrastructure quality’
When creating the LMI we described it as a measure of performance of a deck and as definition of performance of a deck as:
The performance of a deck is defined by its own strength and popularity inside the metagame.
The definition of performance is probably not be as functional as it should as it seems a bit limiting in what it means. Yet, the term performance still remains appropriate.
What information can be added to the index? An easy inspiration can be taken from Riot’s main games: League of Legends (LoL)
In LoL a common value to describe the performance of a champion, in in addition to playrates and winrates is the banrate of a champion.
The most infamous value of the banrate was 95% banrate of Kassadin in S3. If we consider the definition of performance given earlier we can see that the banrate doesn’t fir perfectly strength or popularity in the metagame but it is more a case in the middle. It can seen as an aspect of strength as people don’t want to deal with it, so banning it, but it can be as an aspect of popularity or better yet a more general presence as while it may have not been played it sort of lingers in the match. The banned champion was not present directly in the match but with its spirit (the ban).
In the context of LoR such information can be added once we consider BoX data, currently only Bo3 with the easiest example being the Seasonal Tournaments and a possible way to define it is:
\[\begin{equation} BanRate = \frac{\#ban}{\#match} \end{equation}\]
Example: 2 LineUps contained a Teemo/Ezreal deck, both played all 9 matches and Teemo/Ezreal was banned respectively 3 and 6 times; the ban rate would be \(\frac{(3+6)}{(9+9)} = 50\%\)
As the variable is deemed appropriate for the LMI purpose the following step is to define the structure of the LMI to account for the new variable.
With just three variables the possible ways to combine them are exactly three as shown in Fig:1, Fig:2 and Fig:3
Each different structure correspond to a different way to see the banrate relationship with the other variables.
In the first structure it is considered a different characteristic altogether in comparison to playrates and winrates.
In the second structure the banrate of a deck is considered a part of its ‘strength’, the higher it is the more it means that players don’t want to deal with it be it for playpatterns, expected winrates, or other reasons one may have.
In the third structure the banrate of a deck is considered a part of its ‘presence’, it may have not been played, but like in the Kassadin example before, it’s lingering in the matches as an unseen factor that is still influential to the results. After all the ban or not of a deck, so if it takes an active or passive role in a match can heavily influence the remaining Match Ups.
While the second structure may seems the more intuitive choice, none of these strctures are the one proposed at the end. This is because of the results we found during the statistical analysis.
In this section we describes how the analysis was done in its entirely and not just the final version of the steps to do in order to create the LMI. This is so to highlight some of the results we found and how we add the account for them.
To assess which structure should be used we need to check the relationship between variables. This can be done by looking at their correlations.
Using all the data and calculating the correlation would be wrong, as, as showed in the previous LMI article it is better to limit the analysis to a smaller pool of decks with a sufficient amount of games.
At the Seasonal Tournament the number of games in total is overall small compared to the number of decks played so we tried a series of possible cutoff as min number of games played and find a compromise between not eliminating too many decks and having enough data to have quality results.
We tried to decide the min number of games required by looking at the amount of remaining decks we would have. The effect of different choices can be seen in Tab:1.
Note: it’s the number of decks with also no missing values, as in the 296 decks there is 1 with no mean banrate even at cutoff of zero, the overall number is reduced by one.
Table 1: Cutoff Table
minGames  #Deck 

0  295 
10  145 
30  79 
50  52 
100  37 
200  29 
Amount of remaining Decks depending on the required min amount of Games 
At the first glance, the 200 games used during the metareports seems excessive as it reduce the decks to 1/10th, 10 games is probably not enough as the winrates would be too unstable making us gravitating mostly on 30,50,100 but we will truly decide being guided by the correlation results as shown in Tab:2.
Table 2: Correlation Table at different cutoff
Correlation  0  10  30  50  100  200 

WR/playrate  0.07  0.12  0.18  0.16  0.13  0.08 
meanBan/playrate  0.09  0.24  0.29  0.33  0.28  0.24 
meanBan/WR  0.02  0.09  0.07  0.03  0.08  0.02 
Correlation between the three raw variables on different amount of required min amount of Games 
The resulting correlations took us by surprise in a first moment. Not only the variable with the higher correlation to the banrate is the playrate and not the winrate (the expected initial result) but the correlation with winrate seems to be negative. This is not strange, in Bo3 setting after the bans each player tries to enforce the best matchups among the remaining decks so it easier to have bad matchups even for highlyperforming decks on the ladder. This made us question whatever to aggregate the banrate with playrate, so the structure in Fig:3. What we actually did was to consider:
We tried to see if this is the case by looking at the correlation of the banrates with not just the winrates at the Seasonal but also the winrates from the ladder up to two weeks before the tournament starts. Since the aggregation is not done with the raw values but the normalized transformation the correlation is calculated after the normalization step using the quantile normalization as done in the previous iteration of the LMI. In addition, not to be affected by the particular tournament chosen this is why we also used the data from the ‘Seasonal Tournament  Guardian of the Ancient’ when the casestudy was aimed to compute the LMI for the ‘Rise of the Underworld’ edition. As seens in Fig:4 this was a crucial choice as the relationship of the banrates with the Seasonal winrates can change radically. In the June tournament the metagame was more polarized and this probably affected the correlations.
How to continue was the hardest part of the analysis. The correlation of ladder winrates have a strong positive orientation with the banrates while the correlation with the Sesonal winrates is more unstable and tends to be negative for certain value. If we wanted to use the structure of Fig:1 or Fig:2 than our problem would have to aggregate variables with different orientations. While this is not a rule one must always follows in the cases mentioned this would negatively impact the quality of the LMI. We would have for sure that an high value of a deck would decrease the winrate index and to have an high LMI we may not want an high banrates as it’s increase may decreae the winrate. It would be harder to have decks with high LMI and high values of both winrates and banrates. At the same time, can we really say the strength of the deck is the one we measured if the winrates we observe seems to be a consequence of the metagame from the ladder and the high performances in Bo1 that is reflected by high banrates as to suggest people don’t want to deal with decks they in the ladder they found are difficult to deal with unless the lineup can handle them?
The solution proposed is to use a winindex that is not created from just the Seasonal winrates but also the ladder winrates to include both informations and having the ladder value reduce if not remove the negative correlation of the Seasonal value. Both for this to work we have to check a couple of conditions:
Winrates from Seasonal and ladder needs to be correlated and the orientation shouldn’t change depending on the tournament, which is a legit worry after seeing the previous figure. A positive correlation can be found and seen in Fig:5.
The aggregated winrates have works better as a component to aggregate with banrates. The results can be seen in Fig:6.
Considering the overall results the final choices to the LMI frameworks are the following:
Aggregating variable into a single index this is always done by applying weights to each component, be them equal weights or another vector. The decision can be made by following different method not all being entirely datadriven. A common way to compute the weights is by using the normalized loading factors of the first eigenvalue from the principal components. With the proposed structure both the weights for the winindex and strength index resulted in a equal weights (0.50 and 0.50). As it is both the simplest case and being confirmed by a datadriven approach equal weights are the applied weights.
Note: if we used the 3subdimension structure the resulting weights would have been
PlayRate Index Strength Index BanRate Index
0.533 0.070 0.537
If we tried to apply the proposed structure for the LMI and create again the graph from the Seasonal  Rise of the Underworld report there is a clear difference from the past version. With the new framework Akshan/Sivir (DE/SH) is not in a league of its own while still being the best decks. We want to remember it is not like the previous or the current iteration is wrong and the other correct, they just measure the same/similar data in a different way. While the previous iteration created a more sensational graph we consider the current one more ‘realistic’ as the difference between Akshan/Sivir (DE/SH) and the other top performing decks was way too high. One may object that the difference is now too small but even it was the case we still consider it an improvement from the previous framework.
We started this article with the aim of adding the banrate to the LMI, contrary from out expectations this required to consider more the relationship with the selected variables and the possible causal relationship among than (in particular the banrates with the seasonal winrates). To solve this problem we proposed to add a forth variable in the ladder winrates from ranked games prior to the tournament.
A question that emerged during the analysis is about how to evaluate decks with a really small numbers of games because of an high banrate so if the selection filter should be adjusted to account for them. This could be done in future analysis that includes the uncertainty and sensitivity analysis.
As the new LMI doesn’t create what seems to be an outlier in Akshan/Sivir (DE/SH) a possible research question is how to determine is an even simpler way the strength of a deck. Meaning is we can translate the LMI into a tiervalue, for example from ‘tierD’ to ‘tierS’, the commonly system used by gamers.
If we tried used the cutoff values used by Balco we would have the following results.
As the cutoff values were chosen without any particular reason behind them, a future article will try to investigate different options for the cutoff values.
To expand the application of this version of the LMI an analysis that will follow will try to apply this framework to Bo3 taken from the gauntlet and friendly Bo3.
Lastly, as the variable we are using are a result of the state of the metagame of that time future article will try to check if we can add informations regarding the quota of counterdecks for a specific deck are played in a particular time of the meta/during the tournament.
Table 3: Quantile table
This Meta Report was created under Riot Games’ “Legal Jibber Jabber” policy using assets owned by Riot Games. Riot Games does not endorse or sponsor this project.
No official data are known at the moment of the writing. Supposition made from a series of points like the fewer amount of Master rank players when the cutoff takes place.↩︎