If you didn’t account for selection effects, you may have correctly avoided boosting DEX because you thought it was actively harmful instead of merely useless.
I immediately considered a selection effect, but then I tricked myself into believing it did matter by a method that corrected for the selection effect but was vulnerable to randomness/falsely seeing patterns. Oops. Specifically I found the average dex for successful and failed adventurers for each total non-dex stat value, but had them listed in an inconvenient big column with lots of gaps. I looked at some differences and it seemed that for middle values of non-dex stats, successful adventurers consistently had lower average dex than failed ones, while that reversed for extreme values. When I (now—I didn’t at the time) make a bar chart out of the data it’s a lot more clear that there’s no good evidence for any effect of dex on success:
If you didn’t look for interactions, you may have dodged the WIS<INT penalty just because WIS seemed like a better place to put points than INT.
Yep. Thing is, I *did* look for interactions—with DEX. I had the idea that DEX might be bad due to such interactions, and when I didn’t find anything more or less stopped looking for such interactions.
And I’m pretty sure even the three people who submitted optimal answers on the last post (good job simon, seed, and Ericf) didn’t find them by using the right link function
For sure in my case. I calculated the success/fail ratios for each value of each stat individually (no smoothing), and found the reachable stat combo that maximized the product of those ratios. This method found the importance of reaching 8. I was never confident that this wasn’t random, though.
When I did later start simming guesses what I simmed would have given smoothed results: a bunch of stat checks with a D20, success if total number of passed stat checks greater than a threshold. The actual test would have been pretty far down in the list of things I would have checked given infinite time.
I immediately considered a selection effect, but then I tricked myself into believing it did matter by a method that corrected for the selection effect but was vulnerable to randomness/falsely seeing patterns. Oops. Specifically I found the average dex for successful and failed adventurers for each total non-dex stat value, but had them listed in an inconvenient big column with lots of gaps. I looked at some differences and it seemed that for middle values of non-dex stats, successful adventurers consistently had lower average dex than failed ones, while that reversed for extreme values. When I (now—I didn’t at the time) make a bar chart out of the data it’s a lot more clear that there’s no good evidence for any effect of dex on success:
Yep. Thing is, I *did* look for interactions—with DEX. I had the idea that DEX might be bad due to such interactions, and when I didn’t find anything more or less stopped looking for such interactions.
For sure in my case. I calculated the success/fail ratios for each value of each stat individually (no smoothing), and found the reachable stat combo that maximized the product of those ratios. This method found the importance of reaching 8. I was never confident that this wasn’t random, though.
When I did later start simming guesses what I simmed would have given smoothed results: a bunch of stat checks with a D20, success if total number of passed stat checks greater than a threshold. The actual test would have been pretty far down in the list of things I would have checked given infinite time.