Presumably many people are already hard at work trying to undo what safety precautions were instilled into Llama-2, and to use various techniques to have it do everything you are imagining not wanting it to do
There are now easily available “uncensored” versions of Llama-2. I imagine the high false refusal rate is going to increase the use of these among non-malicious users. It seems highly likely that, in the context of open source LLMs, overly strict safety measures could actually decrease overall safety.
Consciousness might not matter for alignment, but it certainly matters for moral patienthood.