I think, early on in the first conversation, the metaphor of territory has more legs and could maybe help explain things. For example, if there’s a town and there’s GeoWizard doing a straight line challenge, he might not know how to get there, but you can fairly confidently predict that if you post up in that town you’re gonna see him roll in some time afternoon fairly reliably. Whereas if you’re awaiting the summiting of Mount Everest there’s an unpleasantly good chance that whoever is going up there is going to come down in a bodybag or not at all. And in fact you don’t actually need to know anything particularly about the terrain, if you just look at a map and this person says “I will get to there or there” and you can see them make steady progress day after day, you’re probably looking at a map of the British countryside instead of the Himalayas. And there might of course still mountains and cliffs and barbed wire fences and military areas, and you may not know how to get there and even GeoWizard may not know how to get there. But the thing about being an experienced expert is you can navigate around obstacles in a reliable, repeatable manner, and this makes the entire story a lot more predictable than one would think.
So the point is, so far it looks like AI development is certainly not easy but on net more on the level of somewhat impassable countryside than alpine range. And maybe the destination will turn out to lie behind a cliff, but maybe not. So if your safety depends on GeoWizard not getting to Bamburgh or whatever, the fact that even he may not know how to get there until he tries it is simply not of any comfort.
I think, early on in the first conversation, the metaphor of territory has more legs and could maybe help explain things. For example, if there’s a town and there’s GeoWizard doing a straight line challenge, he might not know how to get there, but you can fairly confidently predict that if you post up in that town you’re gonna see him roll in some time afternoon fairly reliably. Whereas if you’re awaiting the summiting of Mount Everest there’s an unpleasantly good chance that whoever is going up there is going to come down in a bodybag or not at all. And in fact you don’t actually need to know anything particularly about the terrain, if you just look at a map and this person says “I will get to there or there” and you can see them make steady progress day after day, you’re probably looking at a map of the British countryside instead of the Himalayas. And there might of course still mountains and cliffs and barbed wire fences and military areas, and you may not know how to get there and even GeoWizard may not know how to get there. But the thing about being an experienced expert is you can navigate around obstacles in a reliable, repeatable manner, and this makes the entire story a lot more predictable than one would think.
So the point is, so far it looks like AI development is certainly not easy but on net more on the level of somewhat impassable countryside than alpine range. And maybe the destination will turn out to lie behind a cliff, but maybe not. So if your safety depends on GeoWizard not getting to Bamburgh or whatever, the fact that even he may not know how to get there until he tries it is simply not of any comfort.