Is it better to have a rationality meetup with a baseline level of unskilled ad-hoc conflict resolution, or no rationality meetup? (You can ask the same question about any social event potentially open to the public.) I imagine the answer would depend on the number of people expected to attend—informal methods work better for groups of 10 than for groups of 100.
CronoDAS
Karma: 16,389
[Question] Possible AI regulation emergency?
I actually have found an example of a strategy that doesn’t incentivize someone else to self-modify into Hawkbot: https://www.lesswrong.com/posts/TXbFFYpNWDmEmHevp/how-to-give-in-to-threats-without-incentivizing-them
Basically, when you’re faced with a probable extorter, you play Cooperate some of the time (so you don’t always get nuked) but either Defect or Nuke back often enough that Hawkbot gets a lower expected value than Cooperate/Cooperate.
There’s a fairly simple and common conflict that I find explains a lot of procrastination:
-
I want X to have been done.
-
I do not want to be doing X.
So you struggle to convince yourself to “voluntarily” do X even though it’s unpleasant, or you don’t do X and struggle with worrying about the consequences of not doing X. :/
-
Does any process in which they ended up the way they did without considering your decision procedure count as #2? Like, suppose almost all the other agents it expects to encounter are CDT agents that do give in to extortion, and it thinks the risk of nuclear war with the occasional rock or UDT agent is worth it.
Especially in the age of AGI, leaders may no longer need to respect the values of most people because they’re not economically relevant.
Or militarily relevant. Traditionally, if you were a ruler, you had to at least keep your army happy. However, if you command an entirely automated army that doesn’t have any actual people in it, there’s no risk of the army turning against you. You have the robot weapons and nobody else does, so you can do whatever the hell you want to people without having to care what anyone else thinks.
Most pivotal acts I can easily think of that can be accomplished without magic ASI help amount to “massively hurt human civilization so that it won’t be able to build large data centers for a long time to come.” I don’t know if that’s a failure of imagination, though. (An alternative might be some kind of way to demonstrate that AI existential risk is real in a way that’s as convincing as the use of nuclear weapons at the end of World War II was for making people consider nuclear war an existential risk, so the world gets at least as paranoid about AI as it is about things like genetic engineering of human germlines. I don’t actually know how to do that, though.)
[Question] Game theory of “Nuclear Prisoner’s Dilemma”—on nuking rocks
Indeed, but I don’t actually know how to do that.
Honestly? By going to the list of doctors that my health insurance will pay for, or some other method of semi-randomly choosing among licensed professionals that I hope doesn’t anti-correlate with the quality of their advice. There are probably better ways, but I don’t know what they are offhand. ::shrug::
If you were accused of a crime and intended to plead not guilty, how would you choose a defense attorney, assuming you weren’t going to use a public defender?
And of course the right answer is “absolutely everyone”. It should be fully public. If your setup is such that it even makes sense to ask this question of “who should be allowed to know what cryptographic algorithm we use”, then your security system is a complete failure and nobody should trust you with so much as their mother’s award-winning recipe for potato salad, much less any truly sensitive data.
This makes sense for computer security, but for biosecurity it doesn’t work, because it’s a lot harder to ship a patch to people’s bodies than to people’s computers. The biggest reason there has never been a terrorist attack with a pandemic-capable virus is that, with few exceptions (such as smallpox), we don’t know what they are.
A: My understanding is that the U.S. Government is currently funding research programs to identify new potential pandemic-level viruses.
K: Unfortunately, yes. The U.S. government thinks we need to learn about these viruses so we can build defenses — in this case vaccines and antivirals. Of course, vaccines are what have gotten us out of COVID, more or less. Certainly they’ve saved a ton of lives. And antivirals like Paxlovid are helping. So people naturally think, that’s that’s the answer, right?
But it’s not. In the first place, learning whether a virus is pandemic capable does not help you develop a vaccine against it in any way, nor does it help create antivirals. Second, knowing about a pandemic-capable virus in advance doesn’t speed up research in vaccines or antivirals. You can’t run a clinical trial in humans on a new virus of unknown lethality, especially one which has never infected a human — and might never. And given that we can design vaccines in one day, you don’t save much time in knowing what the threat is in advance.
The problem is there are around three to four pandemics per century that cause a million or more deaths, just judging from the last ones — 1889, 1918, 1957, 1968 and 2019. There’s probably at least 100 times as many pandemic-capable viruses in nature — it’s just that most of them never get exposed to humans, and if they do, they don’t infect another human soon enough to spread. They just get extinguished.
What that means is if you identify one pandemic-capable virus, even if you can perfectly prevent it from spilling over and there’s zero risk of accidents, you’ve prevented 1⁄100 of a pandemic. But if there’s a 1% chance per year that someone will assemble that virus and release it, then you’ve caused one full pandemic in expectation. In other words, you’ve just killed more than 100 times as many people as you saved.
you can’t very well trust someone else more than you trust yourself
In certain domains, I absolutely can and will do this, because “someone else” has knowledge and experience that I don’t and could not conveniently acquire. For example, if I hire lawyers for my business’s legal department, I’m probably not going to second-guess them about whether a given contract is unfair or contains hidden gotchas, and I’m usually going to trust a doctor’s diagnosis more than I trust my own. (The shortfalls of “Doctor Google” are well-known, so although I often do “do my own research” I only trust it so much.)
I think the amount of cash a bank loses in a typical armed robbery really isn’t that large compared to the amounts of money the bank actually handles—bank robbers are a nusiance but not an existential threat to the bank.
The actual big danger to banks comes from insiders; as the saying goes, the best way to rob a bank is to own one.
There’s a way to test this. If you put a piece of paper next to the HVAC filters around the fan, does it stick to them or fall down?
So RL “narrows the reasoning boundary”— the region of problems the model is capable of solving sometimes.
This seems useful if you don’t want your model answering questions about, say, how to make bombs.
His worst defeat came at the hands of General Winter.
A story of ancient China, as retold by a book reviewer:
In 536 BC, toward the end of the Spring and Autumn period, the state of Zheng cast a penal code in bronze. By the standards of the time, this was absolutely shocking, an upending of the existing order—to not only have a written law code, but to prepare it for public display so everyone could read it. A minister of a neighboring state wrote a lengthy protest to his friend Zichan, then the chief minister of Zheng:
“In the beginning I expected much from you, but now I no longer do so. Long ago, the former kings consulted about matters to decide them but did not make penal codes, for they feared that the people would become contentious. When they still could not manage the people, they fenced them in with dutifulness, bound them with governance, employed them with ritual propriety, maintained them with good faith, and fostered them with nobility of spirit. They determined the correct stipends and ranks to encourage their obedience, and meted out strict punishments and penalties to overawe them in their excesses. Fearing that that still was not enough, they taught them loyalty, rewarded good conduct, instructed them in their duties, employed them harmoniously, supervised them with vigilance, oversaw them with might, and judged them with rigor. Moreover, they sought superiors who were sage and principled, officials who were brilliant and discerning, elders who were loyal and trustworthy, and teachers who were kind and generous. With this, then, the people could be employed without disaster or disorder. When the people know that there is a code, they will not be in awe of their superiors. Together they bicker, appeal to the code, and seek to achieve their goals by trying their luck. They cannot be governed.
“When there was disorder in the Xia government, they created the ‘Punishments of Yu.’ When there was disorder in the Shang government, they created the ‘Punishments of Tang.’ When there was disorder in the Zhou government, they composed the ‘Nine Punishments.’ These three penal codes in each case arose in the dynasty’s waning era. Now as chief minister in the domain of Zheng you, Zichan, have created fields and ditches, established an administration that is widely reviled, fixed the three statutes, and cast the penal code. Will it not be difficult to calm the people by such means? As it says in the Odes,
Take the virtue of King Wen as a guide, a model, a pattern;
Day by day calm the four quarters.
And as it says elsewhere,
Take as model King Wen,
And the ten thousand realms have trust.In such an ideal case, why should there be any penal codes at all? When the people have learned how to contend over points of law, they will abandon ritual propriety and appeal to what is written. Even at chisel’s tip and knife’s edge they will contend. A chaotic litigiousness will flourish and bribes will circulate everywhere.
“Will Zheng perhaps perish at the end of your generation? I have heard that ‘when a domain is about to fall, its regulations are sure to proliferate.’ Perhaps this is what is meant?”
Zichan wrote back:
“It is as you have said, sir. I am untalented, and my good fortune will not reach as far as my sons and grandsons. I have done it to save this generation.”
I don’t know if this is true, but I’ve heard that when the Nazis were having soldiers round up and shoot people (before the Holocaust became more industrial and systematic), as part of the preparation for having them execute civilians, the Nazi officers explicitly offered their soldiers the chance to excuse themselves (on an individual basis) from having to actually perform the executions themselves with no further consequences.
https://thethreevirtues.com/
Three Virtues
According to Larry Wall(1), the original author of the Perl programming language, there are three great virtues of a programmer; Laziness, Impatience and Hubris
Laziness: The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful and document what you wrote so you don’t have to answer so many questions about it. Impatience: The anger you feel when the computer is being lazy. This makes you write programs that don’t just react to your needs, but actually anticipate them. Or at least pretend to. Hubris: The quality that makes you write (and maintain) programs that other people won’t want to say bad things about.
This is a lot to think about. My late wife definitely lived in a much darker world than I have, and I really hope I managed to be a ray of light in hers.
Billy Joel describes the struggle well.