If engineering grade solutions to the problem are acceptable, a design for nanotech or biotech that vastly increases the amount of conventionally controllable matter/energy and doesn’t come with a backdoor that instantiates another intelligence seems like straight empowerment.
The messy dilemma only come up when elements of decision-making are outsourced to AI. Tooling that makes a specific baseline human comprehensible course of action possible in the first place only gives the AI control in that the AI can limit the performance of that tooling and so predictably shape what humans can do with it.
Obviously we’re not on track for that future but it is a viable solution.
If an untrustworthy boxed ASI hands you nanobot plans in the form of a generic toolkit of understandable parts then there seem like ways to audit-ably/randomly assemble those parts in a way that minimizes risk the ASI breaks out through that causal channel. Hacking human brains is a risk but the toolkit and bootstrapping workflow space itself seems somewhat benign with the right engineering discipline. Not so easy with biotech mind you given all the potential interactions between parts, but dry nanotech is fairly understandable and biotech based bootstrap workflows can be isolated enough in practice to be fairly safe.
If engineering grade solutions to the problem are acceptable, a design for nanotech or biotech that vastly increases the amount of conventionally controllable matter/energy and doesn’t come with a backdoor that instantiates another intelligence seems like straight empowerment.
The messy dilemma only come up when elements of decision-making are outsourced to AI. Tooling that makes a specific baseline human comprehensible course of action possible in the first place only gives the AI control in that the AI can limit the performance of that tooling and so predictably shape what humans can do with it.
Obviously we’re not on track for that future but it is a viable solution.
If an untrustworthy boxed ASI hands you nanobot plans in the form of a generic toolkit of understandable parts then there seem like ways to audit-ably/randomly assemble those parts in a way that minimizes risk the ASI breaks out through that causal channel. Hacking human brains is a risk but the toolkit and bootstrapping workflow space itself seems somewhat benign with the right engineering discipline. Not so easy with biotech mind you given all the potential interactions between parts, but dry nanotech is fairly understandable and biotech based bootstrap workflows can be isolated enough in practice to be fairly safe.