[Link] “Doing being rational: polymerase chain reaction” by David Chapman
This is a link post for:
I’m not going to quote the content at the link itself. [Should I?]
David Chapman – the author of the linked post – claims that “meta-rational” methods are necessary to ‘reason reasonably’. I, and I think a lot of other people broadly part of the greater rationalist community, have objected to that general distinction. I still stand by that objection, at least terminologically.
But with this post and other previous recent posts/‘pages’ that he’s posted at his site Meaningness, I think I’m better understanding the points he was gesturing or hinting at with what he describes as meta-rationality and I think that’s because ‘rationality’, in his understanding, is grounded in the actual behavior people perform. The notion that Richard Feynman is quoted as insisting on of having to “work on paper”, or the idea of ‘repair’ are almost certainly real and important facts about how we actually reason.
I think there’s less and less object-level disagreement between him and myself, even about something like ‘Bayesian reasoning’. His recent writing has crystalized the notion in me that he’s really on to something and dispelled the notion that we shared some kind of fundamental disagreement. It seems relatively unimportant whether I or others subsume ‘meta-rationality’ within ‘rationality’ (or not).
I’m less sure how much of this applies to arbitrary reasoners or artificial intelligence, e.g. an AI could maintain a checklist ‘internally’ instead of relying on an external physical device to perform the same function, but the ideas he’s discussing seem to be insightful and true to me about our own rational practices nonetheless.
The stuff on “cognitive prostheses” reminded me of Nova Organum.
Written before reading the linked post (but after reading the link post).
Contents:
Of Style
Rationality
AI
Footnotes
Of Style
In linkposts, often the content is quoted wholesale (beginning to end) or partially (First X paragraphs or so). Although the post starting with a video might have made these options more difficult than usual.
This is a reference to the linked article.
Rationality
This could be a disagreement about the name, or what the name should be.*
How we think people should reason. (Rationalism?)
How people should reason. (Empiricism?)
There’s disagreement around Bayesian reasoning on LW, some relating to logical inductors, which might be relevant (although it’s not clear) - the fact that ‘the credence you’d give to different hypothesis changes as you think about it’ has been mentioned in his work.
David Chapman’s thoughts on AI might differ from yours.
AI
AlphaStar: Impressive for RL progress, not for AGI progress could be loosely interpreted as “AI have memory problems”:
[emphasis mine] The whole post is worth reading.
Footnotes
*Perhaps the distinction is
-Rationality is what you should do.
-Meta-rationality is what you should do in order to “make rationality work”.
While these two things can be combined under one umbrella, making definitions smaller:
-Increases clarity (of discussion)
-Makes it easier to talk about components
-Makes it clear when all membership criterion for a category have been met.
-Might help with teaching/retention
I love this comment!
I didn’t know that – thanks for the feedback!
Should I edit my post to quote the referenced content?
At sufficient detail, everyone’s thoughts about anything (sufficiently complex) differ from everyone else’s. But I don’t think David Chapman and I have any fundamental disagreements about AI.
Ooooh! That’s a perfectly concise form of a criticism I’ve thought about neural network architectures for a long time. The networks certainly are a form of memory themselves, but not really a history, i.e. of distinct and relatively discrete events or entities. Our own minds certainly seem to have that kind of memory and it seems very hard for an arbitrary intelligent reasoner to NOT have something similar (if not exactly something like this).
The quoted text you included is a perfect example of this kind of thing too; thanks for including it.
Isn’t there evidence that human brains/minds have what is effectively a dedicated ‘causal reasoning’ unit/module? It probably also relies on the ‘thing memory’ unit(s)/module(s) too tho.
As I mentioned, or implied, in this post, I’m indifferent about the terminology. But I like all of your points and think they’re good reasons to make the distinction that Chapman does. I’m going to consider doing the same!
I’m not an expert on neuroscience. I’m not sure how continuous those things are between being modular and integrated. (I have a suspicion they’re both.)
It’s a particular style/format[1], using it isn’t required. There’s a couple styles around this that I’ve noticed:
1) A post which consists of a link, and maybe material on what it is. This is done by describing it, with an intended audience of people who haven’t read it. (The low-effort way of doing this is to copy the first few paragraphs.)
2) Writing a post with a (single) pre-requisite, which is linked to.
What makes the two different is that 1 is meant to be read after the linked post, while 2 is for reading before it [2].
I brought it up because I read your post, and then the linked post, and then I read your post again. 1 and 2 can be combined, but it wasn’t clear whether your post came before the link or after, and it wasn’t clearly split into two parts along those lines.
[1] I am not familiar with its origin.
[2] Except when 2 is a quote, and skipping to the link doesn’t miss anything. The discussion/comments on a post of either type may be indistinguishable, including discussions which require having read the linked post to understand. Sometimes a post which originally just consists of a link might get comments like “what’s the linked post about”.
Thanks!