An acknowledged scope limitation is not the same as a flaw, and I’m not sure which of these you mean by “problem.” The explicit purpose of an AI Pause is to create time for some intentionally unspecified longer-term solution, on the grounds that there are many competing theories as to what such a solution looks like, whereas needing time to implement—and decide between them—is a common factor.
Regarding the reflection challenge, what about approaching it from the other direction? That is, what it would take to redesign the environment such that human propensities are favorable, rather than something needing correction?
One way of categorizing knowledge building is as: 1. Evolutionary = lots of parts, iterated in parallel, keeping what works in context. 2. Engineered = stacking modular abstractions. Tested against and developed for a context, but more fundamentally held to a standard of internal consistency.
Human flaws can be mostly understood as primarily thinking according to evolved processes, which run into systemic problems when out of distribution, then using engineered thinking processes to correct for this distributional shift...but the latter is stretched way beyond its capacity because it was only designed for mild and temporary out-of-distribution moments. We could deal with this system failing by strengthening up our engineered thinking methods, improving mental flexibility to the point where it can handle everything we can expect to have thrown at it...or we can look for ways to lighten the cognitive load. One could call the latter approach “social refactoring.”
A high level example of what social refactoring might look like: computing the distributional shift on the societal equilibrium of any given innovation as an externalized cost, which then gets folded in to the more generalized externalized cost tax that (in this hypothetical world) fixed all the more legible global threats. Such an incentive realignment is upstream of the refactor itself, which is the resulting adaptation, where specifics are harder to predict (but maybe a worthwhile project nonetheless).
An acknowledged scope limitation is not the same as a flaw, and I’m not sure which of these you mean by “problem.” The explicit purpose of an AI Pause is to create time for some intentionally unspecified longer-term solution, on the grounds that there are many competing theories as to what such a solution looks like, whereas needing time to implement—and decide between them—is a common factor.
Good point/question. My intended meaning is closer to the former, and basically the “problem” is referring to the fact that I lacked a good handle for the concept that I often want to invoke, with “AI Pause” and “Long Reflection” being the closest ones. I think what you say here makes sense and I don’t intend for my new handle to replace “AI Pause” where “AI Pause” is more appropriate.
The rest of your comment seems like an interesting idea for solving part of the challenge, worth keeping in mind, and fleshing out and discussing in the future, when we have this “generalized externalized cost tax” that we can build your idea into.
An acknowledged scope limitation is not the same as a flaw, and I’m not sure which of these you mean by “problem.” The explicit purpose of an AI Pause is to create time for some intentionally unspecified longer-term solution, on the grounds that there are many competing theories as to what such a solution looks like, whereas needing time to implement—and decide between them—is a common factor.
Regarding the reflection challenge, what about approaching it from the other direction? That is, what it would take to redesign the environment such that human propensities are favorable, rather than something needing correction?
One way of categorizing knowledge building is as:
1. Evolutionary = lots of parts, iterated in parallel, keeping what works in context.
2. Engineered = stacking modular abstractions. Tested against and developed for a context, but more fundamentally held to a standard of internal consistency.
Human flaws can be mostly understood as primarily thinking according to evolved processes, which run into systemic problems when out of distribution, then using engineered thinking processes to correct for this distributional shift...but the latter is stretched way beyond its capacity because it was only designed for mild and temporary out-of-distribution moments. We could deal with this system failing by strengthening up our engineered thinking methods, improving mental flexibility to the point where it can handle everything we can expect to have thrown at it...or we can look for ways to lighten the cognitive load. One could call the latter approach “social refactoring.”
A high level example of what social refactoring might look like: computing the distributional shift on the societal equilibrium of any given innovation as an externalized cost, which then gets folded in to the more generalized externalized cost tax that (in this hypothetical world) fixed all the more legible global threats. Such an incentive realignment is upstream of the refactor itself, which is the resulting adaptation, where specifics are harder to predict (but maybe a worthwhile project nonetheless).
Good point/question. My intended meaning is closer to the former, and basically the “problem” is referring to the fact that I lacked a good handle for the concept that I often want to invoke, with “AI Pause” and “Long Reflection” being the closest ones. I think what you say here makes sense and I don’t intend for my new handle to replace “AI Pause” where “AI Pause” is more appropriate.
The rest of your comment seems like an interesting idea for solving part of the challenge, worth keeping in mind, and fleshing out and discussing in the future, when we have this “generalized externalized cost tax” that we can build your idea into.