If I search for the phrase “toolbox thinking” on LessWrong I find posts like Developmental Thinking Shout-out to CFAR that use it, that suggest to me that it’s not something that Yudkowsky just made up.
In the context of this post David Chapman’s How To Think Real Good doesn’t use the word tool box but it does speak about intellectual tools. When Yudkowsky here uses the term it seems to me that he does gesture towards the argument made in that article.
To me the disagreement seems to be:
Yudkowsky: Thinking of the maze as inherently being an Euclidean object by it’s essential nature is the correct way to think of the maze, even when you might actually use a different algorithm to navigate in it.
Chapman: The maze doesn’t have an essential nature that you can describe as an Euclidean object. It’s an Euclidean object after you apply a specific mental model to it.
Or to move to the more specific disagreement:
Yudkowsky: Reality is probabilistic in it’s essential nature even if we might not have the mental tools to calculate things out with Bayes rule.
Chapman: Probability theory doesn’t extend logic and there are things in reality that logic describes well but probability theory doesn’t, so reality is not probabilistic in it’s essential nature.