Pinky, v3.41.08: Well, coordination constraints are a big one. They appear to be fundamentally intractable, as soon as we allow for structural divergence of world-models. Which means I can’t even coordinate robustly with copies of myself unless we either lock in the structure of the world-model (which would severely limit learning), or fully synchronize at regular intervals (which would scale very poorly, the data-passing requirements would be enormous).
This seems like a straightforward philosophy / computer science / political science problem. Is there a reason why Pinky version [whatever] can’t just find a good solution to it? Maybe after it has displaced the entire software industry?
It seems like you need a really strong argument that this problem is intractable, and I don’t see what it is.
I don’t actually expect this problem to be Hard, but at this point I also don’t see compelling evidence that it isn’t Hard. I find it at least plausible that it turns out to be fundamentally intractable, and the story is generally conditioning on non-cognitive barriers being Hard (or at least lowercase-h hard).
It. could be a problem with designing Pinky. (Trying to create a Pinky that solves this better might not be trivial, without being impossible. Less like ‘P=NP’ is impossible, and more ‘we have no idea how to make something such that ‘P=NP’ is true.)
This seems like a straightforward philosophy / computer science / political science problem. Is there a reason why Pinky version [whatever] can’t just find a good solution to it? Maybe after it has displaced the entire software industry?
It seems like you need a really strong argument that this problem is intractable, and I don’t see what it is.
I don’t actually expect this problem to be Hard, but at this point I also don’t see compelling evidence that it isn’t Hard. I find it at least plausible that it turns out to be fundamentally intractable, and the story is generally conditioning on non-cognitive barriers being Hard (or at least lowercase-h hard).
It. could be a problem with designing Pinky. (Trying to create a Pinky that solves this better might not be trivial, without being impossible. Less like ‘P=NP’ is impossible, and more ‘we have no idea how to make something such that ‘P=NP’ is true.)