Not only is “What do we believe?” a theoretically distinct question from “What do I do about it?”
“What do we believe?” is a distinct question; and asking it is comitting an error of rationality. The limitations of our minds often force us to use “belief” as a heuristic; but we should remember that it is fundamentally an error, particularly when the consequences are large.
You don’t do the expected-cost analysis when investigating a theory; you should do it before dismissing a theory. Because, If someday you build an AI, and hardcode in the many-worlds assumption because many years before you dismissed the one-world hypothesis from your mind and have not considered it since, you will be committing a grave Bayesian error, with possibly disastrous consequences.
(My cost-of-error statements above are for you specifically. Most people aren’t planning to build a singleton.)
I can’t speak for Eliezer, but if I was building a singleton I probably wouldn’t hard-code my own particular scientific beliefs into it, and even if I did I certainly wouldn’t program any theory at 100% confidence.
“What do we believe?” is a distinct question; and asking it is comitting an error of rationality. The limitations of our minds often force us to use “belief” as a heuristic; but we should remember that it is fundamentally an error, particularly when the consequences are large.
You don’t do the expected-cost analysis when investigating a theory; you should do it before dismissing a theory. Because, If someday you build an AI, and hardcode in the many-worlds assumption because many years before you dismissed the one-world hypothesis from your mind and have not considered it since, you will be committing a grave Bayesian error, with possibly disastrous consequences.
(My cost-of-error statements above are for you specifically. Most people aren’t planning to build a singleton.)
I can’t speak for Eliezer, but if I was building a singleton I probably wouldn’t hard-code my own particular scientific beliefs into it, and even if I did I certainly wouldn’t program any theory at 100% confidence.