My PvE approach, as I mentioned, was to copy the plan that worked best in a comparable game: train a model to predict deck success, feed it the target deck, then optimize the opposing deck for maximum success chance. I feel pretty good about how well this worked. If I’d allocated more time, I would have tried to figure out analytically why the local maxima I found worked (my model noticed Lotus Ramp as well as Sword Aggro but couldn’t optimize it as competently for some reason), and/or try multiple model types to see what they agree on (I used a GBT, which has high performance but doesn’t extrapolate well).
My PvP approach was a lot more scattershot. After some hilariously bad attempts to get an uncounterable deck by having two decks repeatedly optimize against each other, I decided to just recycle my PvE deck and hope I happened to win the rock-paper-scissors game. As it happened, Fate smiled on me, but if there had been any Good Tribal decks in play I wouldn’t be looking quite so clever right now.
Reflections on the challenge:
This was fun. I particularly like that it was superficially similar to Defenders of the Storm, while having profoundly different mechanics: I came in expecting another game that’s mostly about counters, and instead got a game that’s mostly about synergy. And, as everyone (including me) has already said, the premise and writing are hilarious.
My only problem with this game was the extra difficulty associated with approaching it analytically if you don’t happen to know about mtg-style card games (I remember looking at the comments on the main post late last week and wondering what a ‘ramp’ was). However, this issue is mitigated by the facts that:
.It (presumably) gave card game fans a chance to practice balancing-priors-against-new-evidence skills and not just ML/analysis skills. .It’s not unreasonable for card game knowledge to help pick cards in a game centered on card games. .I won despite lacking this background.
My only problem with this game was the extra difficulty associated with approaching it analytically if you don’t happen to know about mtg-style card games (I remember looking at the comments on the main post late last week and wondering what a ‘ramp’ was).
I actually considered this to be mostly a feature rather than a bug? I think real-world data science problems also benefit from having some knowledge of the domain in question.
It’s possible to apply data science techniques to a completely unfamiliar domain—you don’t need to know anything about card games to notice that ‘P’ and ‘S’ showing up together, or ‘L’ and ‘E’ showing up together, improves your payoff function, and to try submitting an answer that has lots of ‘P’s and lots of ’S’s in it.
But if you have some level of domain knowledge, you have more ability to guess what kind of patterns are likely to appear, and to extrapolate details. When you see that ‘L’ works well with ‘D’, ‘E’ and ‘A’ that doesn’t tell you much else: when you notice that ‘L’ works well with all three of the cards that have long and bombastic names, that lets you start guessing things like ‘there are some kind of costs to playing these powerful cards, and L helps you pay those costs to play them’. This lets you guess in turn things like ‘adding more Emperors might make the deck stronger against other decks like itself but weaker against faster decks’ that would be very hard to pull out of the data directly without some amount of domain knowledge to help.
This is part of the reason why I gave the cards names instead of just saying ‘Card ID 1’, ‘Card ID 2’, etc. (The other part is of course to sound cooler :P)
Reflections on my attempt:
My PvE approach, as I mentioned, was to copy the plan that worked best in a comparable game: train a model to predict deck success, feed it the target deck, then optimize the opposing deck for maximum success chance. I feel pretty good about how well this worked. If I’d allocated more time, I would have tried to figure out analytically why the local maxima I found worked (my model noticed Lotus Ramp as well as Sword Aggro but couldn’t optimize it as competently for some reason), and/or try multiple model types to see what they agree on (I used a GBT, which has high performance but doesn’t extrapolate well).
My PvP approach was a lot more scattershot. After some hilariously bad attempts to get an uncounterable deck by having two decks repeatedly optimize against each other, I decided to just recycle my PvE deck and hope I happened to win the rock-paper-scissors game. As it happened, Fate smiled on me, but if there had been any Good Tribal decks in play I wouldn’t be looking quite so clever right now.
Reflections on the challenge:
This was fun. I particularly like that it was superficially similar to Defenders of the Storm, while having profoundly different mechanics: I came in expecting another game that’s mostly about counters, and instead got a game that’s mostly about synergy. And, as everyone (including me) has already said, the premise and writing are hilarious.
My only problem with this game was the extra difficulty associated with approaching it analytically if you don’t happen to know about mtg-style card games (I remember looking at the comments on the main post late last week and wondering what a ‘ramp’ was). However, this issue is mitigated by the facts that:
.It (presumably) gave card game fans a chance to practice balancing-priors-against-new-evidence skills and not just ML/analysis skills.
.It’s not unreasonable for card game knowledge to help pick cards in a game centered on card games.
.I won despite lacking this background.
I actually considered this to be mostly a feature rather than a bug? I think real-world data science problems also benefit from having some knowledge of the domain in question.
It’s possible to apply data science techniques to a completely unfamiliar domain—you don’t need to know anything about card games to notice that ‘P’ and ‘S’ showing up together, or ‘L’ and ‘E’ showing up together, improves your payoff function, and to try submitting an answer that has lots of ‘P’s and lots of ’S’s in it.
But if you have some level of domain knowledge, you have more ability to guess what kind of patterns are likely to appear, and to extrapolate details. When you see that ‘L’ works well with ‘D’, ‘E’ and ‘A’ that doesn’t tell you much else: when you notice that ‘L’ works well with all three of the cards that have long and bombastic names, that lets you start guessing things like ‘there are some kind of costs to playing these powerful cards, and L helps you pay those costs to play them’. This lets you guess in turn things like ‘adding more Emperors might make the deck stronger against other decks like itself but weaker against faster decks’ that would be very hard to pull out of the data directly without some amount of domain knowledge to help.
This is part of the reason why I gave the cards names instead of just saying ‘Card ID 1’, ‘Card ID 2’, etc. (The other part is of course to sound cooler :P)
It’s definitely a feature as well; the exact tradeoff comes down to personal taste.