I don’t think it should be particularly surprising that physics gets more weird and unintuitive to humans the more we inspect it at scales further and further away from the ones we evolved to operate in. And domain experts being wrong about AI timelines is, to me, not surprising whatsoever, both because of technological advancement timelines being notoriously hard to predict and AI itself being a difficult subject.
More generally, how exactly would we differentiate human error because of a task or subject being difficult and human error because of living in a simulation? I feel like in the case of the second one, it would be more like having a completely inconsistent hole in our world. Like, if everything else played out the same before November 3, 2020 but Kanye West got elected instead, every domain expert, even after the fact, would still be absolutely baffled.
In the case of AI and physics, so far, we seem to be able to readjust our internal knobs into something consistent (because, as expected, no matter how outlandish the rules of the micro world are, they still need to add up to what we experience in the macro world).
The Minecraft thought experiment is interesting, but you would have to play by the actual rules of the Minecraft world, so whatever intelligent agent is born there would have to be instantiated according to those rules.
I’m not really sure how life could begin naturally by Minecraft rules. If an intelligence just sort of pops in there suddenly, then I feel like, to it, that would be good evidence of outside interference once it learns more about standard Minecraft world rules.
Ok, I see what you’re saying. If our world’s rules align with what we consider to be computationally efficient, then that is at least moderate evidence for living in a simulation. I’m not sure whether that is consistently the case, though.
As for your Minecraft point, what I was saying was that you would have to instantiate the intelligence within the Minecraft world, running by its rules. And I can’t see how anything like that could arise naturally in there. It would have to just sort of appear suddenly.
I guess you could also have it running externally and only feed it input from Minecraft, but that seems like even further evidence for a simulation from its perspective (“everything else in the world fits into relatively simple rules, but I’m a super complex black box that also just suddenly popped in here”).