If the AI actually ends up with strong evidence for a scenario it assigned super-exponential improbability, the AI reconsiders its priors and the apparent strength of evidence rather than executing a blind Bayesian update, though this part is formally a tad underspecified.
I would love to have a conversation about this. Is the “tad” here hyperbole or do you actually have something mostly worked out that you just don’t want to post? On a first reading (and admittedly without much serious thought—it’s been a long day), it seems to me that this is where the real heavy lifting has to be done. I’m always worried that I’m missing something, but I don’t see how to evaluate the proposal without knowing how the super-updates are carried out.
That hyperbole one. I wasn’t intending the primary focus of this post to be on the notion of a super-update—I’m not sure if that part needs to make it into AIs, though it seems to me to be partially responsible for my humanlike foibles in the Horrible LHC Inconsistency. I agree that this notion is actually very underspecified but so is almost all of bounded logical uncertainty.
I would love to have a conversation about this. Is the “tad” here hyperbole or do you actually have something mostly worked out that you just don’t want to post? On a first reading (and admittedly without much serious thought—it’s been a long day), it seems to me that this is where the real heavy lifting has to be done. I’m always worried that I’m missing something, but I don’t see how to evaluate the proposal without knowing how the super-updates are carried out.
Really interesting, though.
That hyperbole one. I wasn’t intending the primary focus of this post to be on the notion of a super-update—I’m not sure if that part needs to make it into AIs, though it seems to me to be partially responsible for my humanlike foibles in the Horrible LHC Inconsistency. I agree that this notion is actually very underspecified but so is almost all of bounded logical uncertainty.
Using “a tad” to mean “very” is understatement, not hyperbole.
One could call it hypobole.
Specifically, litotes.