Broken link. pls fix.
Luo Ling
Karma: 17
While price gouging can quickly mobilize forces to satisfy the emergency demand, they can also have problematic second-order effects. If a price gouge is too high, then this allows certain agents to benefit from the disaster. This creates a perverse incentive that disincentivizes disaster prevention, and potentially even incentivizing artifically intensifying / creating disasters.
Some vague idea: Alignment can be fragile. Can capabilities be made fragile too?
I think fragile capabilities can be potentially useful in situations that needs to prevent tampering the model, eg finetuning a model to jailbreak / learn dangerous bioweapon capabilities.
For the banning of these weapons, how much does effectiveness weigh against moral concerns? If usefulness weighs a lot, then these examples won’t generalize to TAI.
Unless there are very clear, convincing evidence that TAI isn’t controllable with current paradigm, then it will still be perceived as a highly useful tech. (Even if such evidence exists, IMO there’s high possibility that they’ll just cope harder.)
Biochemical weapons: These are only useful against civilians and pre-modern armies. Modern armies can easily afford equipments to protect against these.
https://acoup.blog/2020/03/20/collections-why-dont-we-use-chemical-weapons-anymore/
(I saw this article mentioned somewhere in another LW post. When I see TsviBT’s shortform I immediately recalled this article, so I wrote this post.)
Space nukes and LEO missiles: In space there’s no cover, they’re easily detectible. Without air, dodging maneuver cost significant dv. This means overall less survivability than ground / sea based nukes.
Deploying missiles in LEO also requires a more complicated trajectory than traditional ground / sea based missiles, which cost more dv. If they need to stay in space for a long time, then reliability and maintainence also becomes a serious problem.
Obedience in this post looks like corrigibility which has been discussed a lot.
Zachtronics games does part of that. In those games the player doesn’t do the tasks directly, instead they need to program a bot (or other systems) to do the task. While mirroring the player is impossible, it should be possible to mirror the bots programmed by the player.
https://www.lesswrong.com/posts/TyusAoBMjYzGN3eZS/why-i-m-not-a-bayesian
In most cases, when we mention probability, we’re using bayesian framework. The lack of description power of a scalar probability may be caused by the limits of bayesianism.