One MTGA Player Conceded Hundreds of Matches to Probe the Ranking System – Here are the Results.

One MTGA Player Conceded Hundreds of Matches to Probe the Ranking System - Here are the Results.

MTG: Arena player and Twitter user @Hareeb_AlSaq made waves on social media over the weekend by publishing a long and extremely detailed report about how they used multiple Arena accounts in an attempt to reverse engineer the game’s ranking system- an opaque system which has had very little official explanation on how it actually works.

The “k-value” refers to a factor that is used in the widespread Elo ranking system to determine the maximum change in player rating from any single game. Read on for details.

This announcement along with al-Saq’s published report (language warning) has not landed without controversy, with some players accusing al-Saq of making the game worse for players- both by instantly conceding so many matches, and also by exposing how the current system is highly exploitable. Conversely, al-Saq claims that they are only interested in improving how the ranking system in MTGA works, and that they hope their report may lead to improvements to the current system. Regardless of the motivations, this report makes a number of claims which seem to correspond with past sentiments from high playtime players including well-known streamers and MPL members. The full report is fascinating, but also lengthy and filled with mathematical jargon, so we have made an attempt to break it down to its primary points.

Before we proceed, it must be stated that al-Saq’s report is entirely unverified and NOT backed up by anything other than anecdotal evidence from other players. This report has not been verified or endorsed by Wizards of the Coast in any capacity, nor is it verified or endorsed by the MTGA Zone team.

A rating system exists and is used to pair players not just in Mythic, but in the lower ranks as well.

This is something that has been reported by other players who have noticed it in the past, as al-Saq themself mentions in their report. The reason that this is purported to be problematic or unfair has to do with the rank floors. If you play ranked ladder on Arena, then you already know that once you achieve a new rank (Silver, Gold, Mythic, etc), you cannot lose that rank until the season resets. However, if there is indeed a hidden rating system that is used for pairings, it exists separately from the shown ladder system. Therefore, if a player concedes or otherwise loses a large number of matches at the very bottom of a rank, their rating will continue to drop and drop while still staying at the same visible rank at the very bottom. After throwing a large number of games, the player would have a low enough rating that they will be paired against other players who are actually at a lower skill level, as opposed to starting out with a middling rating and facing equally skilled opponents. Theoretically, they would then have a much easier time climbing the ladder to the next rank since they’re still starting at the same place on the visible rank ladder even though their hidden player rating is much lower.

Interestingly, this hidden player rating is something that has been addressed by Wizard’s of the Coast in a State of the Beta post from 2018. WotC refers to the player ratings as MMR, or Matchmaking Rating. They even specifically reference the idea of players losing intentionally: “…it also creates a bit of a ‘safety net’ against players who are willing to sacrifice their Limited Ranking (or vice versa) to lower their MMR and ensure “easier” matches in their format of choice.” The report written by al-Saq directly disputes a few of the points in this official post from Wizards, especially the idea the MMR shields against intentional losses as it seems to do the opposite.

A player’s performance before entering Mythic has little impact on their placement upon entering Mythic.

In their experiment, al-Saq alleges to have found that their two burner accounts entered Mythic at roughly the same percentage level, even though the first account played matches normally, and the second “conceded hundreds of matches at Diamond 4 before trying to win.” According to al-Saq, this supposed fact, paired with a recent change to Arena that has allowed more players to get to Mythic, has caused issues that include the well-known severe rank decay at the end of the month.

Starting with the Ikoria ranked season, wins in the Gold rank award two pips on the ladder instead of one. This has caused more players to reach Mythic, and based on al-Saq’s findings, this has caused the players entering Mythic in the late season to have an over-inflated player rating when compared to the players who have already been in Mythic since they enter at roughly the same player rating. This causes the early-season Mythic players to “farm” the newcomers, who are presumably weaker players despite having similar player ratings. This then contributes to the rank decay at the end of the month, as more lower-skill players enter Mythic and are facing against the high skill, high playtime players who can achieve high winrates in those matches.

The K-value for Best-of-Three matches appears to be very high, almost the same as it was in high level tournament play of the past.

In ranking systems that are based on the well-established Elo system, the K-value refers to a mathematical constant that is applied to the ranking change for each match. It is used to essentially determine the maximum change in a player’s rank for each matchup, which helps to calibrate the effect that the most recent games played by that player have on their ranking over time. If al-Saq’s research and conclusions are correct, the K-value for a best-of-three match on Arena is somewhere around 45, while the K-value for best-of-one is only around 20. Wizards used to use a basic Elo system for ranking competitive play, where Pro Tour matches had a K-value of 48. That means that if al-Saq’s estimations are even close to correct, the effect on a player’s rank in best-of-three on Arena is almost the same as it was for tournament players at Magic’s highest levels of play.

What does it mean?

The findings of al-Saq’s report are very much based on estimations and mathematical modeling paired with observable changes in rank. There are many details and huge amounts of data that nobody has any access to outside of Wizards of the Coast. For this reason, it cannot be assumed that all of al-Saq’s findings are completely accurate- in fact, al-Saq would probably not claim that they are, and the report explains the full details of how its findings were reached if you would like to evaluate it for yourself. Instead, al-Saq was presumably trying to point out inconsistencies with the details that Wizards has provided about the ranking system, as well as to explain unexplained phenomena such as the insane rank decay at the top of the Mythic ladder.

It’s possible that more transparency from Wizards about how the ranking system works specifically would debunk some of al-Saq’s claims and prevent other players from feeling the need to do similar explorations in the future. It’s also possible that Wizards keeps most of the details about the ranking system private because they don’t want that information to be used to exploit it. Regardless of the accuracy of al-Saq’s findings, exploitation of the ranked ladder does seem to be possible at the moment. We can only hope that widely publicized reports like this encourage Wizards to make changes to the ladder in ways that will result in more fair gameplay and less opportunity for exploitation.

Paul

Dude from Vermont who likes to play Magic and Escape from Tarkov. Musician, writer, and gamer.

15 Responses

  1. Tom says:

    I noticed and abused it LONG ago (I was conceding a lot on Tier 4 for altruistic reasons). Easy Mythic for me! I told others abut it but few believed me. Too bad for them.

    • Albel says:

      Every system can be abused. What is worrying me is that with the help of mtgazone and other platforms you help spreading the word. This will easily break the ladder for a while and lead to masses of people conceding like crazy. That is, until they change the system. I’d probably enable tier 4 demotions (e. g platin to gold etc.) if I was WotC. I can already hear people complaining… 🙂

      • Ferseth says:

        The idea that pointing out flaws and lies about this is a bad thing is insane. It is WoTC’s job to make it ACTUALLY hard to exploit not to use lies and obfuscation to pretend. I hope more people abuse this it will force WoTC to do it right. You are a IDIOT. Exposing and reporting on the flaws in systems like this IS GOOD. Defending shotty and manipulative systems is BAD. Take WoTC’s dong out of your mouth lack of oxygen is causing you to sound muffled and dumb.

        • Albel says:

          Heyho

          While I do agree that the result is right (they amend it), they way of doing it (mass abuse) is in my opinion wrong due to massive negative player experience. There is a reason why media are sometimes considered weapons of mass destruction.

          Improving features of a game can be done in a much more structured and mannered way.

          Speaking of manners, why my different opinion from yours is causing you to insult me, I really fail to see. I wish you a great day anyways!

          • rando reader says:

            “much more structured and mannered way.”

            MTGA has a broken system that they do not comment on. There is no structured manner that they would address this unless pressured to do so.

      • Thane says:

        It’s far better for everybody to have access to that information, than only a privileged few.

        At least with the corner, you have a level playing field.

  2. Chrysologus says:

    Rock and a hard place. If they keep it the same, people can abuse it by conceding repeatedly at tier 4. If they freeze the Elo rating as well, so you can’t lower it, people may get frustrated who keep losing repeatedly at tier 4. If they abandon the tier freezing completely and let you drop back, people will complain that it’s too harsh and makes it too hard to get to Mythic!

    • casperionx says:

      I dont think making mythic is really the challenge, it’s making it to numbered mythic that is.

      And while people can manipulate the game to be in their favour making the jump from mythic to percentage is probably a lot harder.

      I do believe preventing to back a level is GOOD however the non visible ranking needs to be prevented from manipulation

    • Jhon says:

      Pretty simple solution… Make the matchmaking truely random.

  3. Max Zakharzhevskiy says:

    This is no different than people throwing game 1 of day 1 tournaments to get into the lower bracket and have easier matches. Winning a tournament x-0 versus x-1 is the same. It eventually will get harder.

  4. Paul Stevenson says:

    MTG Arena is DIRT and MANIPULATIVE.

  5. A says:

    This explains why I always made it out of the league in the last week of the season.

    Maybe this will finally cause Wizards to make fair Ranked. I am OK with MMR-based stuff in play, as well as automatcher pairing certain archetypes of decks, but in ranked this is utter nonsence. As soon as you get a new deck, after 8-10 wins you are paired against same 3-4 archetypes 80% of the time, and those are against which you lose most.

    I stopped spending any real money in-game, and stopped tryng to go above plat because it’s a waste of time.

    Ranked should be completely random based on 1 factor – League level. Until then I am going to exploit the system until they fix it.

  6. Chris says:

    The entire game is rigged. From “hand smoothing” your opening hand, to the opponents you face, to the sequence the cards appear even the free cards you get. The only truly random part of the game is when you play it.
    When I first started playing, I was playing catch up on cards. So I didn’t buy any core packs. Deciding to save my coins for Zendikar and use my wild cards on the previous sets. I never bought a core pack.
    When Zendikar was released, I had almost 4 copies all of the common cards in the core set except 5. One of each color. Each was a creature. Each could defeat the other in combat. And each had a property that fit with the theme of that color (4 power for green, gain life/lose life when it dies for black, does damage when etb for red, etcc.)
    That was not coincidental. And I plan to do the same for the next core set to prove my theory.
    This is a fun game as long as you don’t invest too much into it. Because its completely a cash grab

    • rt says:

      You cannot get any of those 5 M21 common creatures in boosters, you need to craft them. There are quite a lot of cards that cannot be found in boosters (e.g. Korvold / Kenrith), you can use filter “-?booster” to find all those cards (or “?booster” to filter cards that can be found in boosters).

  7. Jerome says:

    I was sure of that cause the level of players was very different after many win or loose.
    So one month after i beginned to play mtga,
    i conceded a lot while doing other things…
    I reach the bottom where around 1/4 of player don’t play or concede.
    Easy win… You can do your daily win reward without playing.
    Badly, there is not many player doing that, so in one week you can reach the medium tiers…

Leave a Reply

Your email address will not be published. Required fields are marked *