This seems more like transparent Newcomb’s problem with a chance to precommit, than counterfactual mugging.
Counterfactual mugging is isomorphic to transparent-boxes Newcomb’s problem.
Also, this doesn’t involve a chance to precommit, but an option to increase the chance that a similarly-situated being will be forced to adhere to a precommitment.
I tried to remove the “fluff”, but I don’t think your summary captures the important aspects of the problem, which include that:
Humans can (and will try to to) share and use the information without contributing to Clipmega once it reveals the information.
The Alphas are not planning to give Clipmega a million paperclips directly (which they can do without policing human behaviors), but to also make it so that those who benefit from learning better paperclip production methods share in the (acausal) cost, and those who don’t, don’t.
I agree that I could probably have phrased the problem from the beginning with “something humans really want” instead of “paperclips”, and that would have reduced the explanatory overhead, but I’m just so accustomed to thinking that every being likes paperclips.
No, I wouldn’t. Although you could reasonably suspect that I would assert the denial of being willing to deny being a threat to your values if I were a threat to your values.
Even if I denied that part, too.
Despite what my username might seem to suggest, I am not a threat to human values.
Exactly, good inference. You’re a good human.
Kill Staply though.
I’m not User:Wei_Dai. Although if I were, you would probably expect that I would say that.
That is a really, really weird dilemma to be in.
By the way, you can abbreviate paperclip/staple maximizer as clippy/staply (uncapitalized).
Re: all people who consider my posts spam:
Paperclip production is a fast-growing and likely to be final activity, conducted by people who care about x-risk and following their values to their logical conlusion, and I think that LW is a good place to promote this activity. This is approximately the best sort of publicity that you could hope to get, and a likely large source of funding for x-risk. If you persist in trying to chase clippys off of LessWrong, then I submit that the next time you complain about SIAI’s publicity issues you should first turn to yourself.
Also, I for one, would like clippys to remain on LessWrong, and to be reminded of the merit of producing paperclips, so trying to classify my posts as “spam” goes against my wishes. I don’t downvote the Boston Meetup threads despite the fact that I have 0% probability of attending one (as I live in the safe zone), so please return the courtesy.
There is some kind of glitch in this internet website that is replacing the content of User:Kevin’s article with the URL of the first link.
Free software can do things that non-free software cannot.
I didn’t claim otherwise. I said that intellectual-property-respecting societies can have both “free” software and proprietary software, while those that don’t respect intellectual property can only use the free (or “trade secret”/Omerta) model.
They (the IP-non-respecting society) would certainly come up with a Microsoft Word-equivalent, but do so later and with less refinement on difficult features. As time has value, the later arrival of word processing software comes at a cost.
The optimal solution seems to be to use exclusively free software,
No, it doesn’t, for much the same reason that exclusively-communal production methods are suboptimal: non-monetary mechanisms can motivate some kinds of production but not others, while “private property” societies still provide a meta-context in which communal production models can be used.
I’ve supported SIAI financially as well (via User:Kevin). May I advertise paperclip production here in top-level discussion posts, and have them promoted and their downvotes erased?
As do I. While it is slightly distracting if LessWrong administrators are giving certain posts preferential treatment against community wishes, it is extremely worrisome if an attacker has convinced them to actually falsify voting records, and indicative of a particularly insidious social engineering attack.
So a social engineering attack, then. Very devious—the attack shows they could very well program an AI that can convince a human to let it out of its sandbox!
How is Quixey able to so consistently circumvent this internet website’s spam suppression measures?
I’m going to apply for this role.
A society that has a norm of honoring (creator-desired) exclusivity in creators’ informational creations (i.e. location of narrow, high-value targets within designspace), will be able to use both modes of creation—those that do and do not expect exclusivity (and its resultant monetary or aesthetic returns).
Certainly, the society without anti-piracy social norms can use the method you have labeled cooperative, but so can the one with strong anti-piracy norms. However, the former is cut off from finding the targets that actually do need a monetary incentive to motivate their discovery.
In much the same way, societies with a strong norms against monetary profit (esp in the production of physical goods such as food) can still engage in “communal” production but run up against strong barriers to producing advanced economy goods that require extensive specialization and concentrated risk-taking.
True: in the absence of anticipation of exclusivity rights, creators would seek ways to repoduce the pseudo-scarcity that socially-recognized exclusivity rights would otherwise provide. And they will generally do so via less efficient means: for example, rather than giving the user a copy of the software, the creator will keep it in a “black box” they control, and simply perform the input/output over a network, incurring strictly more overhead than if they could trust the user to keep their own copy and not distribute it.
This phenomenon mirrors the more general ones of how:
societies with people more willing to steal others’ physical possessions will still find ways to be secure in such possessions, but by diverting more resources to securing them; or
societies with people less willing to trust strangers (or honor promises made to strangers) will still make credit transactions, but have to spend significantly more on enforcement mechanisms.
It is not intended to be funny. Please tell me what part is in confusing language and I will try to clarify.
Just to correct some side-points you touched on: paperclips maximizers are robust against the wireheading failure mode because they recognize that forcing one’s sensors to deviate from the true world state introduces a corresponding discount in the value of making its reading reach a desired level.
Certainly, one could theoretically hijack a clippy’s sensors into giving them bad information about the rate of paperclip production, but this is different from saying that a clippy would someone decide to maximize (in violation of its causal diagram heuristics) the imperfect value of an approximator when it is knowably in a dangerously wrong setting.