Could AgI safety overhead turn out to be more harmful in terms of actual current human lives, as well as their individual ‘qualities’? (I don’t accept utilitarian or longtermism couter-arguments since my question solely concerns the present which is the only reality, and existing ‘individualities’ as the only ones capable of experiencing it. Thank you)
Yes. If it turns out that alignment is trivial or unneeded whenever AGIs appear. The same as with all excessive safety requirements. The problem here being discerning between excessive and necessary. An example would be hand sanitizers etc. during the pandemic. In the beginning it wasn’t stupid, as it wasn’t known how the virus spread. Later, when it became obvious that it was airborne, then keeping antibacterial gels everywhere in the name of ending the pandemic was stupid (at least in the context of Covid) and wasteful. We’re currently in the phase of having no idea about how safe AGI will or won’t be, so it’s a good idea to check for potential pitfalls.
It comes down to uncertainty. It seems wiser to go slowly and carefully, even if that will result in loss, if doing so can limit the potential dangers of AGI. Which could potentially involve the eradication of everything you hold dear.
I’m not sure how literally you treat the present—if you only care about, say the next 5 years, then yes, the safety overhead is harmful in that it takes resources from other stuff. If you care about the next 30 years, then the overhead is more justified, given that many people expect to have an AGI by then.
Could AgI safety overhead turn out to be more harmful in terms of actual current human lives, as well as their individual ‘qualities’? (I don’t accept utilitarian or longtermism couter-arguments since my question solely concerns the present which is the only reality, and existing ‘individualities’ as the only ones capable of experiencing it. Thank you)
Yes. If it turns out that alignment is trivial or unneeded whenever AGIs appear. The same as with all excessive safety requirements. The problem here being discerning between excessive and necessary. An example would be hand sanitizers etc. during the pandemic. In the beginning it wasn’t stupid, as it wasn’t known how the virus spread. Later, when it became obvious that it was airborne, then keeping antibacterial gels everywhere in the name of ending the pandemic was stupid (at least in the context of Covid) and wasteful. We’re currently in the phase of having no idea about how safe AGI will or won’t be, so it’s a good idea to check for potential pitfalls.
It comes down to uncertainty. It seems wiser to go slowly and carefully, even if that will result in loss, if doing so can limit the potential dangers of AGI. Which could potentially involve the eradication of everything you hold dear.
I’m not sure how literally you treat the present—if you only care about, say the next 5 years, then yes, the safety overhead is harmful in that it takes resources from other stuff. If you care about the next 30 years, then the overhead is more justified, given that many people expect to have an AGI by then.