Assuming the person making the decision is perfect at estimating risk.
Or merely aware of the same potential weakness that you are. I’d be overwhelmingly uncomfortable with someone developing a super-intelligence without the awareness of their human limitations at risk assessment. (Incidentally ‘perfect’ risk assessment isn’t required. They make the most of whatever risk assessment ability they have either way.)
“Of the steps of the smallest size that still advances things, which of those steps has the lowest risk?”
I consider this a rather inferior solution—particularly in as much as it pretends to be minimizing two things. Since steps will almost inevitably be differentiated by size the assessment of lowest risks barely comes into play. An algorithm that almost never considers risk rather defeats the point.
If you must artificially circumvent the risk assessment algorithm—presumably to counter known biases—then perhaps make the “small steps” a question of satisficing rather than minimization.
Since steps will almost inevitably be differentiated by size the assessment of lowest risks barely comes into play. An algorithm that almost never considers risk rather defeats the point.
If you must artificially circumvent the risk assessment algorithm—presumably to counter known biases—then perhaps make the “small steps” a question of satisficing rather than minimization.
Or merely aware of the same potential weakness that you are. I’d be overwhelmingly uncomfortable with someone developing a super-intelligence without the awareness of their human limitations at risk assessment. (Incidentally ‘perfect’ risk assessment isn’t required. They make the most of whatever risk assessment ability they have either way.)
I consider this a rather inferior solution—particularly in as much as it pretends to be minimizing two things. Since steps will almost inevitably be differentiated by size the assessment of lowest risks barely comes into play. An algorithm that almost never considers risk rather defeats the point.
If you must artificially circumvent the risk assessment algorithm—presumably to counter known biases—then perhaps make the “small steps” a question of satisficing rather than minimization.
Good point.
How would you word that?