The Trouble with Tier Lists in MTG Limited

Compulsion Article

Tier lists have really taken over lately. While they have been a staple for several competitive video games over the last decade or so, people are starting to apply the concept to literally everything.

So it shouldn’t be too surprising that one of the most popular ways to assess the power of cards and inform draft picks in MTG is using such lists. With the impending release of Theros: Beyond Death, I fully expect dozens of them to be created over the next few weeks. Based on the title of the article you are probably expecting this to be purely a take down of tier lists, but I have to say…

Their popularity is understandable

  1. They are easy to comprehend – We are all too familiar with the grading scale (A to F), and we get the idea that 4 is greater than 3 on a numbered scale. Most tier lists are well organized and conveniently link directly to card images.
  2. They are easy to access – In a ‘real’ draft it would be frowned upon (or illegal) to whip out your phone and use outside resources to try to optimize your draft picks. But on Arena you are free to draft at your own pace, and if you find yourself stuck between a few choices you can utilize tier lists to quickly identify which card has the highest rating.
  3. They are easy to create – I should acknowledge that I actually made one for the Throne of Eldraine release. Once you decide on your scale, it is surprisingly easy to slot cards into the framework if you are an experienced player. While it did take a couple hours to rate every card, I spent significantly longer formatting the damn thing. Long story short I made a pretty sweet table in google sheets but then it wouldn’t play nice with the website, causing hours of troubleshooting frustration. I digress, but think most content creators will agree that rating the cards is not the most challenging aspect of presenting tier lists.

So what is the problem, then?

Like almost anything in life, the quality of content is generally proportional to the amount of time and effort put into it. Consider for example a stream vs. an edited video vs. a feature film. Streams can certainly be enjoyable, and watching things live has a unique excitement factor, but think about how much more densely packed a nicely edited video is. Hours of a stream can often be boiled down into a much shorter video with a little time and effort. Films take this to an extreme, with hundreds of people putting in countless hours to produce a two hour chunk of entertainment that is richly packed with content. Not to belabor the point, but music is another great example. Artists often work for months writing, recording, mixing, and mastering their work. It is crazy to think about how much time and effort can go into a 3-minute song.

Tier lists are easy, but you get what you give. So where do they fall short?

1. There isn’t a logical consistency between most lists – Nearly every tier list defines its ‘tiers’ slightly differently, making it difficult to directly compare them. Even the amount of tiers can be highly variable. On an A to F scale there are five tiers, while if you add +/-‘s there are 13. Some lists include an ‘S’ and/or ‘E’ category, other lists use a number scale with or without decimels. This results in a mess of card ratings that don’t correlate between lists. Even within a single list there are usually issues. I have seen players that deem their ratings ‘pack 1 pick 1,’ which is essentially saying their card ratings are done in a vacuum. But this results in ratings that become less and less useful as more cards are selected. When rating video game characters or even Constructed MTG decks that represent a complete package, this system makes more sense. But in Limited MTG each card is a piece of an overall puzzle, so rating them individually in a vacuum is only going to be accurate for the most straightforward cards.

2. The power level of a magic card is largely context dependent – A strong limited deck not only has strong individual cards, but also capitalizes on synergy between them while maintaining a good curve and creature/noncreature composition. Additionally, there should be consideration as to how the overall deck strategy fits into any existing metagame. Sure, some cards are always going to be stronger than other cards, but there is a large degree of uncertainty in card ratings generated by all of these factors. Let’s look at some examples! The gold standard for me has always been Luis Scott Vargas (LSV). His tier lists **ahem** I mean set reviews are about as good as it gets. LSV is a renowned magic player and does a fantastic job explaining the thought process behind his ratings. This is something that is sorely lacking from so many tier lists out there. But even a player as prolific as LSV can find themselves at a loss when trying to give a concrete rating to certain cards:

These 1.0 // 3.0 ratings amount to ‘I don’t really think these are good cards but I could see them being decent in the right deck.’ And really that is a proper assessment, but it isn’t too helpful for players trying to figure out if they should rare draft Emry, Lurker of the Loch or a 3.0 rated common, for example. LSV has been using these split ratings more and more recently. This may be due to mechanics such as Adventure and Adament that represent potential upside (or downside) to certain cards:

Edgewall Innkeeper is going to be one of the best cards in a dedicated Adventure deck, but playing a vanilla 1/1 is a horrible downside if it is drafted without much support. Bog Naughty can likewise be a huge threat, but without sufficient food enablers or room in the mana curve for an overcosted 3/3 flyer, could end up a liability. This is all without even considering metagame factors such as how fast or slow the format is. Cards like Revenge of Ravens are often too slow to get on the board and stabilize fast enough to gain value, but in Eldraine this card ended up being much better than expected. Since most tier lists come out before the set is even played by most players, there are bound to be oversights like this regardless of who is rating the cards. For instance, almost no one saw the Blue Mill decks coming when Eldraine released. Players tend to love Mill in limited, but there didn’t seem to be much support for the archetype besides Merfolk Secretkeeper. However, playing Limited on Arena presents an additional wrinkle:

3. On Magic Arena, the power level of cards is also dependent on the bots.

At the release of Eldraine the bots were seemingly programmed to always pass Merfolk Secretkeeper (and Lucky Clover). Due to this, Blue Mill decks became ubiquitous and completely twisted the metagame. Happenings like this really illustrate the uncertainty in card ratings. As the metagame unfolds and bots are tweaked it only gets more problematic. If the ratings were done in a vacuum, they quickly become obsolete due to the shifts. If the ratings were done based on the bots and metagame, then continual updates are needed for the list to remain relevant. Ultimately, the confounding factors of metagame, set mechanics, deck composition, and mana curve all impact the effective value of a given card. It isn’t that the players making tier lists are incompetent, they are just working with a deeply flawed premise.

What should be used instead of Tier Lists?

I want to first reiterate that tier lists definitely still have their usefulness. When a new set releases, they are a great way to get familiar with the cards and get a rough idea of how strong certain players think they are. Where things get shaky is when players tend to overuse them in their drafting and deckbuilding process. Unfortunately, a B rated card is not always going to be better than a C rated card in a given deck. When these sort of guides are used it can give players the illusion that they drafted perfectly because they drafted all of the higher rated cards.

It is important to own your draft picks so that you give yourself the opportunity to learn from them. It is easy to shift the blame of defeat to the tier list or luck, when so often there may have been missed opportunities in the drafting/deckbuilding process or line of play. Tier lists can be a great crutch to use to get started or reference when needed, but if you are always using a crutch you are effectively crippling yourself. I would urge you to make some difficult decisions in your next draft. Likely some of them will be ‘incorrect,’ but I guarantee you will learn more from it. Ideally you will also reflect on how your other potential picks could have played as well. Maybe you made a tough call between three cards in the draft. When you draw that card during your games, think about how the other options would have done.

Exercises like that are the experiences that help you construct your own understandings of the game. Playing a bad (or good) card provides evidence and helps build reasoning as to why it is a bad (or good) card. Those experiences help build your strength as a player because they are applicable to future drafts and games. Consulting a list and picking or passing a card based on its rating alone can cheat you out of that reflection. Guides can be a great tool, just make sure they aren’t doing all of the thinking for you or it will affect your growth as a player. You should also demand more from your guides than mere ratings.

A rating is just an assertion. Only when assertions are backed up with evidence and/or reasoning are they worth consideration. Try to seek out guides like LSV’s (or ours on mtgazone.com!) that actually discuss the cards. It is more time and effort for us, but results in a more useful product for you.

Speaking of, my next project will be a Limited Set Review of Theros beginning with White this Friday after the spoilers are complete. It will be my first collaboration with Drifter and should be an insightful first look at all of the cards broken down by color. I am really excited to learn and play Theros: Beyond Death with you all in a couple weeks! I am also looking forward to the upcoming Mythic Championship Qualifier this weekend. Good luck to anyone competing with me :). Lastly, I have decided to start streaming again, so if you would like to see me do some Theros drafts with commentary (when the time comes) you can follow me at twitch.tv/compulsion02. Thank you to everyone who supported me in different ways in 2019, and I am very optimistic about MTG in 2020!

3 Responses

  1. Chris says:

    Tier lists like LSV’s and Deathsie’s were instrumental in teaching me how to draft a few years ago, but as you stated – they can only take you so far. Great for new players and early in sets, but hardly to be taken as gospel. On LR Luis often mentions how dramatically a cards “rating” can change over the lifetime of a limited set, and sometimes you have to learn the hard way why deck synergy, proper land base, and a good curve are just as instrumental as picking cards with the biggest number or highest letter grade.

  2. VB says:

    As someone who has tried to pick up drafting in the last year or so, one wish list item for Tier Lists/Set Reviews, based on where I struggled:

    1) Definition of common archetypes of drafting. What does an Aggro deck look like in this set? What does a control deck look like? etc.

    This way, LSV’s dual rankings make more sense. One is the agnostic power level and the second is the power level when placed firmly within its preferred archetype.

Comment