Eliezer: “You could see someone else’s engine operating materially, through material chains of cause and effect, to compute by “pure thought” that 1 + 1 = 2. How is observing this pattern in someone else’s brain any different, as a way of knowing, from observing your own brain doing the same thing? When “pure thought” tells you that 1 + 1 = 2, “independently of any experience or observation”, you are, in effect, observing your own brain as evidence.”
Richard: “It’s just fundamentally mistaken to conflate reasoning with “observing your own brain as evidence”.”
Eliezer: “If you view it as an argument, yes. The engines yield the same outputs.”
Richard: “What does the latter have to do with rationality?”
Pure thought is something your brain does. If you consider having successfully determined a conclusion from pure thought evidence that that thought is correct, then you must consider the output of your brain (i.e. its, that is your, internal representation of this conclusion) as valid evidence for the conclusion. Otherwise you have no reason to trust your conclusion is correct, because this conclusion is exactly the output of your brain after reasoning.
If you consider your own brain as evidence, and someone else’s brain works in the same way, computing the same answers as yours, observing their brain is the same as observing your brain is the same as observing your own thoughts. You could know abstractly that “Bob, upon contempating X for 10 minutes, would consider it a priori true iff I would”, perhaps from knowledge of both of your brains compute whether something is a priori true. If you then found out that “Bob thinks X a priori true” you could derive that X was a priori true without having to think about it: you know your output would be the same (“X is a priori true”) without having to determine it.
Eliezer: “You could see someone else’s engine operating materially, through material chains of cause and effect, to compute by “pure thought” that 1 + 1 = 2. How is observing this pattern in someone else’s brain any different, as a way of knowing, from observing your own brain doing the same thing? When “pure thought” tells you that 1 + 1 = 2, “independently of any experience or observation”, you are, in effect, observing your own brain as evidence.”
Richard: “It’s just fundamentally mistaken to conflate reasoning with “observing your own brain as evidence”.”
Eliezer: “If you view it as an argument, yes. The engines yield the same outputs.”
Richard: “What does the latter have to do with rationality?”
Pure thought is something your brain does. If you consider having successfully determined a conclusion from pure thought evidence that that thought is correct, then you must consider the output of your brain (i.e. its, that is your, internal representation of this conclusion) as valid evidence for the conclusion. Otherwise you have no reason to trust your conclusion is correct, because this conclusion is exactly the output of your brain after reasoning.
If you consider your own brain as evidence, and someone else’s brain works in the same way, computing the same answers as yours, observing their brain is the same as observing your brain is the same as observing your own thoughts. You could know abstractly that “Bob, upon contempating X for 10 minutes, would consider it a priori true iff I would”, perhaps from knowledge of both of your brains compute whether something is a priori true. If you then found out that “Bob thinks X a priori true” you could derive that X was a priori true without having to think about it: you know your output would be the same (“X is a priori true”) without having to determine it.