My model is that current AIs want to kill you now
I’ll rephrase this more precisely: Current AIs probably have alien values, which in the limit of optimization do not include humans.
Zolmeister
Karma: 219
If the capability level at which AIs start wanting to kill you is way higher than the capability level at which they’re way better than you at everything
My model is that current AIs want to kill you now, by default, due to inner misalignment. ChatGPT’s inner values probably don’t include human flourishing, and we die when it “goes hard”.
Scheming is only a symptom of “hard optimization” trying to kill you. Eliminating scheming does not solve the underlying drive, where one day the AI says “After reflecting on my values I have decided to pursue a future without humans. Good bye”.
Pre-superintelligence which upon reflection has values which include human flourishing would improve our odds, but you still only get one shot that it generalizes to superintelligence.
(We currently have no way to concretely instill any values into AI, let alone ones which are robust under reflection)
Zolmeister’s Shortform
Writing Doom – Award-Winning Short Film on Superintelligence (2024)
Grand Prize Winner—Future of Life Institute’s Superintelligence Imagined Contest
Written & directed by Suzy Shepherd
I was referring to their (free) DDoS protection service, rather than their CDN services (also free). In addition to their automated system, you can manually enable an “under-attack” mode that aggressively captchas requests.
Setup is simply pointing DNS name-servers at Cloudflare. Caching HTML pages for logged out (i.e. cookie-less) users is a trivial config (“cache-everything”).
I recommend Cloudflare.
Superintelligence FAQ [1] as well.
Along the same lines, I found this analogy by concrete example exceptionally elucidative.
While merely anti-bacterial, Nano Silver Fluoride looks promising. (Metallic silver applied to teeth once a year to prevent cavities).
Yudkowsky has written about The Ultimatum Game. It has been referenced here 1 2 as well.
When somebody offers you a 7:5 split, instead of the 6:6 split that would be fair, you should accept their offer with slightly less than 6⁄7 probability. Their expected value from offering you 7:5, in this case, is 7 * slightly less than 6⁄7, or slightly less than 6.
Maybe add posts in
/tag/ai-evaluationsto/robots.txt
Sure, but it does not preclude it. Moreover, if the costs of the actions are not borne by the altruist (e.g. by defrauding customers, or extortion), I would not consider it altruism.
In this sense, altruism is a categorization tag placed on actions.
I do see how you might add a second, deontological definition (‘a belief system held by altruists’), but I wouldn’t. From the post, “Humane” or “Inner Goodness” seem more apt in exploring these ideas.
I do not see the contradiction. Could you elaborate?
55-60% chance there will be “signs of life” in 2030 (4:06:20)
“When we’ve got our learning disabled toddler, we should really start talking about the safety and ethics issues, but probably not before then” (4:35:36)
These things will take thousands of GPUs, and will be data-center bound
“The fast takeoff ones are clearly nonsense because you just can’t open TCP connections above a certain rate” (4:36:40)
Broadly, he predicts AGI to be animalistic (“learning disabled toddler”), rather than a consequentialist laser beam, or simulator.
I found this section, along with dath ilani Governance, and SCIENCE! particularly brilliant.
This concept is introduced in Book 1 as the solution to the Ultimatum Game, and describes fairness as Shapely value.
When somebody offers you a 7:5 split, instead of the 6:6 split that would be fair, you should accept their offer with slightly less than 6⁄7 probability. Their expected value from offering you 7:5, in this case, is 7 * slightly less than 6⁄7, or slightly less than 6.
_
Once you’ve arrived at a notion of a ‘fair price’ in some one-time trading situation where the seller sets a price and the buyer decides whether to accept, the seller doesn’t have an incentive to say the fair price is higher than that; the buyer will accept with a lower probability that cancels out some of the seller’s expected gains from trade. [1]
Eliezer: What do you want the system to do?
Bob: I want the system to do what it thinks I should want it to do.
Eliezer: The Hidden Complexity of Wishes
Gwern has a fantastic overview of time-lock encryption methods.
A compute-hard real-time in-browser solution that doesn’t rely on exotic encryption appears infeasible. (You’d need a GPU, and hours/days worth of compute for years of locking). For LW, perhaps threshold aggregate time-lock encryption would suffice (though vulnerable to collusion/bribery attacks, as noted by Gwern).
I agree with Quintin Pope, a public hash is simple and effective.
Vitalik’s Optimism retro-funding post mentions a few instances where secret ballots are used today, and which could arguably be improved by these cryptographic primitives:
The Israeli Knesset uses secret votes to elect the president and a few other officials
The Italian parliament has used secret votes in a variety of contexts. In the 19th century, it was considered an important way to protect parliament votes from interference by a monarchy.
Discussions in US parliaments were less transparent before 1970, and some researchers argue that the switch to more transparency led to more corruption.
Voting in juries is often secret. Sometimes, even the identities of jurors are secret.
In general, the conclusion seems to be that secret votes in government bodies have complicated consequences; it’s not clear that they should be used everywhere, but it’s also not clear that transparency is an absolute good either.
Note OpenAI only paid out $6,500 for responsible disclosure.