Hm. Solid point regarding how the counter-narrative to this would look like, I guess. Something to prepare for in advance, and shape the initial message to make harder.
Basic point: Not all things in the second category are created equal. AIs turning out to belong to the second category is just one fact about them. A fact that makes them less safe, but not as safe as any other thing belonging to that category. Updating on this information should provoke a directional update, not anchoring onto the baseline safety level of that category.
As I’d mentioned in the post, I think a lot of baseline safety-feelings towards AIs are based on the background assumption that they do belong to the first category. So taking out this assumption wouldn’t be just some irrelevant datum to a lot of people – it would significantly update their judgement on the whole matter.
Incidentally this is among my favorite theses, with a beautiful elucidation of ‘weird machines’ in chapter two. Recommended reading if you’re at all interested in computers or computation.
Hm. Solid point regarding how the counter-narrative to this would look like, I guess. Something to prepare for in advance, and shape the initial message to make harder.
Basic point: Not all things in the second category are created equal. AIs turning out to belong to the second category is just one fact about them. A fact that makes them less safe, but not as safe as any other thing belonging to that category. Updating on this information should provoke a directional update, not anchoring onto the baseline safety level of that category.
As I’d mentioned in the post, I think a lot of baseline safety-feelings towards AIs are based on the background assumption that they do belong to the first category. So taking out this assumption wouldn’t be just some irrelevant datum to a lot of people – it would significantly update their judgement on the whole matter.
Computer viruses belong to the first category while biological weapons and gain of function research to the second.
For now.
See Urschleim in Silicon: Return-Oriented Program Evolution with ROPER (2018).
Incidentally this is among my favorite theses, with a beautiful elucidation of ‘weird machines’ in chapter two. Recommended reading if you’re at all interested in computers or computation.