I like to read blog posts by people who do real statistics, but with a problem in front of me I’m very much making stuff up. It’s fun, though!
The approach I settled on was to estimate the success chance of a possible stat line by taking a weighted success rate over the data, weighted by how similar the hero’s stats are to the stats being evaluated. My rationale is that based on intuitions about the domain I would not assume linearity or independence of stats’ effects or such, but I would assume that heroes with similar stats would have similar success chances.
weightfactor(hero.stats, stats) = k ^ distance(hero.stats, stats)
(Assuming 0 < k < 1, and hero.succeeded is 1 if the hero succeeded and 0 otherwise)
I tried using both Euclidean and Manhattan distances, and various values for k as well. I also tried a hacky variant of Manhattan distance that added abs(sum(statsA) - sum(statsB)) to the result, but it didn’t seem to change much.
Lastly, I tried the replacing (hero.succeeded) with (hero.succeeded—linearprediction(sum(hero.stats))) to try to isolate builds that do well relative to their stat total. linearprediction is a simple model I threw together by eyeballing the data: 40% chance to succeed with total stats of 60, 100% chance with total stats >= 95, linear in between. Could probably be improved with not too much effort, but I have to stop somewhere.
I generally found two clusters of optima, one around (8, 14, 13, 13, 8, 16)—that is, +4 CHA, +2 STR, +4 WIS—and the other around (4, 16, 13, 14, 9, 16)—that is, +2 CON, +1 INT, +3 STR, +4 WIS. The latter was generally favored by low k values, as the heroes with stats closest to that value generally did quite well but those a little farther away got less impressive. So it could be a successful strategy that doesn’t allow too much deviation, or just a fluke. Using the linear prediction didn’t seem to change things much.
If I had to pick one final answer, it’s probably (8, 14, 13, 13, 8, 16) (though there seems to be a fairly wide region of variants that tend to do pretty well—the rule seems to be ‘some CHA, some WIS, and maybe a little STR’), but I find myself drawn towards the maybe-illusory (4, 16, 13, 14, 9, 16) niche solution.
ETA: Looks like I was iterating over an incomplete list of possible builds… but it turned out not to matter much.
ETA again (couldn’t leave this alone): I tried computing log-likelihood scores for my predictors (restricting the ‘training’ set to the first half of the data and using only the second half for validation. I do find that with the right parameters some of my predictors do better than simple linear regression on sum of stats, and also better the apparently-better predictor of simple linear regression on sum of non-dex stats. But they don’t beat it by much. And it seems the better parameter values are the higher k values, meaning the (8, 14, 13, 13, 8, 16) cluster is probably the one to bet on.
I like to read blog posts by people who do real statistics, but with a problem in front of me I’m very much making stuff up. It’s fun, though!
The approach I settled on was to estimate the success chance of a possible stat line by taking a weighted success rate over the data, weighted by how similar the hero’s stats are to the stats being evaluated. My rationale is that based on intuitions about the domain I would not assume linearity or independence of stats’ effects or such, but I would assume that heroes with similar stats would have similar success chances.
In pseudocode:
estimatedchance(stats) = sum(weightfactor(hero.stats, stats) * hero.succeeded) / sum(weightfactor(hero, stats))
weightfactor(hero.stats, stats) = k ^ distance(hero.stats, stats)
(Assuming 0 < k < 1, and hero.succeeded is 1 if the hero succeeded and 0 otherwise)
I tried using both Euclidean and Manhattan distances, and various values for k as well. I also tried a hacky variant of Manhattan distance that added abs(sum(statsA) - sum(statsB)) to the result, but it didn’t seem to change much.
Lastly, I tried the replacing (hero.succeeded) with (hero.succeeded—linearprediction(sum(hero.stats))) to try to isolate builds that do well relative to their stat total. linearprediction is a simple model I threw together by eyeballing the data: 40% chance to succeed with total stats of 60, 100% chance with total stats >= 95, linear in between. Could probably be improved with not too much effort, but I have to stop somewhere.
I generally found two clusters of optima, one around (8, 14, 13, 13, 8, 16)—that is, +4 CHA, +2 STR, +4 WIS—and the other around (4, 16, 13, 14, 9, 16)—that is, +2 CON, +1 INT, +3 STR, +4 WIS. The latter was generally favored by low k values, as the heroes with stats closest to that value generally did quite well but those a little farther away got less impressive. So it could be a successful strategy that doesn’t allow too much deviation, or just a fluke. Using the linear prediction didn’t seem to change things much.
If I had to pick one final answer, it’s probably (8, 14, 13, 13, 8, 16) (though there seems to be a fairly wide region of variants that tend to do pretty well—the rule seems to be ‘some CHA, some WIS, and maybe a little STR’), but I find myself drawn towards the maybe-illusory (4, 16, 13, 14, 9, 16) niche solution.
ETA: Looks like I was iterating over an incomplete list of possible builds… but it turned out not to matter much.
ETA again (couldn’t leave this alone): I tried computing log-likelihood scores for my predictors (restricting the ‘training’ set to the first half of the data and using only the second half for validation. I do find that with the right parameters some of my predictors do better than simple linear regression on sum of stats, and also better the apparently-better predictor of simple linear regression on sum of non-dex stats. But they don’t beat it by much. And it seems the better parameter values are the higher k values, meaning the (8, 14, 13, 13, 8, 16) cluster is probably the one to bet on.