Congratulations. Now when the bomb is attached the the CPU running the malevolent AI, the AI can hack the Pi and prevent the bomb from going off.
Sometimes the world needs dangerous things, like weapons.
If you don’t want to build dangerous things, don’t become a munitions engineer. (But be aware that someone else will take that role.)
If you are a munitions engineer, be a good one. Build a bomb that goes ‘bang’ reliably when it’s supposed to, and not otherwise. Keep it simple.
Very clear writeup—thank you for doing it. (I”m not sure if it says anything much about cognition either, but hey).
It would be great to see this incorporated in a Wikipedia article, but that’s probably an uphill battle.
Your two principle goals—maximise total utility and minimize utility inequality—are in conflict, as is well-known. (If for no other reason, becuase incentives matter.) You can’t have both.
A more reasonable goal would be Pareto efficiency-limied utility inequality.
Karl Popper said “optimisim is a duty”, because only optimists work to improve the world. Those who think we’re already doomed and that effort is hopeless, don’t try.
Optimism is a duty. The future is open. It is not predetermined. No one can predict it, except by chance. We all contribute to determining it by what we do. We are all equally responsible for its success.
I think that’s from The Open Society and its Enemies (1945).
A desire to understand the arguments is admirable.
Wanting to actually be convinced that we are in fact doomed is a dereliction of duty.
Karl Popper wrote that
Optimism is a duty. The future is open. It is not predetermined. No one can predict it, except by chance. We all contribute to determining it by what we do. We are all equally responsible for its success.
Only those who believe success is possible will work to achieve it. This is what Popper meant by “optimism is a duty”.
We are not doomed. We do face danger, but with effort and attention we may yet survive.
I am not as smart as most of the people who read this blog, nor am I an AI expert. But I am older than almost all of you. I’ve seen other predictions of doom, sincerely believed by people as smart as you, come and go. Ideology. Nuclear war. Resource exhaustion. Overpopulation. Environmental destruction. Nanotechnological grey goo.
One of those may yet get us, but so far none has, which would surprise a lot of people I used to hang around with. As Edward Gibbon said, “however it may deserve respect for its usefulness and antiquity, [prediction of the end of the world] has not been found agreeable to experience.”
One thing I’ve learned with time: Everything is more complicated than it seems. And prediction is difficult, especially about the future.
Re your postscript, Neal Stephenson said “gold is the corpse of value” (quoting banker Walter Wriston), expressing much the same thing.
Contrary to popular belief, rich people almost never hoard wealth. They invest it.
“Commuication requires consent”. There’s a road sign on the highway that strongly, strongly, disagrees.
What is and isn’t stupid to believe depends on the general state of knowledge.
Given what Newton’s society knew at the time, alchemy wasn’t stupid—not enough was known about the nature of matter to know that alchemy was hopeless.
150 years ago nobody had researched smoking—it had some obvious positive effects and no obvious bad effects. Nothing stupid about smoking, then. Almost the same can be said for seatbelts and asbestos (they had obvious pros and cons, of which the magnitudes were not understood.)
Environmental destruction is not bad at small scales, nor is burning fossil fuels. It’s scale that make them bad.
On the other hand, today we know enough about the way the universe works to say that pyramid power, crystal healing, and runes on A4 paper are stupid. As is your landlord.
In competitive situations where there’s lots of optimization experience this tends to be a good strategy. People have been selling and buying used cars for 100 years—all the tricks and counter-tricks have been worked out, and pretty much cancel each other out.
So by not making any special attempt to signal and just being honest, you save the costs of all that signaling. And the other party saves costs protecting themselves from the false signals. Putting you both ahead.
As I recall most of the interest in in the first book in the series, “Wizards Bane” (1989). I don’t think it’s a spoiler under the circumstances—guy builds a spell generation VM using Forth (because he has no actual computers at hand, he needs to keep things really simple).
It’s implied in later books that he bootstraps more complex systems from there.
I do think we’re in one of the few worlds that made it thru the Cold War intact. I consider our existence weak evidence for Many Worlds and quantum immortality.
Gives a reason to be subjectively optimistic about the future. But, I hope, not complacent. I find it hard to shake off the notion that the number of worlds in which civilization survives matters. But I also find it hard to explain why.
What’s with the $10k/year limit? It’s almost as if they want to discriminate against rich people.
Cherry picking. How many of those stoves were made 100 years ago? How many are around now?
Some of our 2021 stoves will be around in 100 years, too. Not many. Will someone point at one and say “back in 2021 they built to last”?
(Same reason old European cities are prettier than new American ones—the ugly buildings got torn down and replaced, the pretty ones didn’t. After a while you have a lot of pretty buildings.)
It’s been done. https://en.wikipedia.org/wiki/Decimal_time
That aside, what’s so special about base 10?
Wouldn’t it be more logical to celebrate passage of, say, 2^24 seconds (194 days)? Once we’re no longer constrained to human bodies, we can have as many fingers on our hands as we want, so we may as well have 8 or 16.
Your social circle is very different from any I’ve ever been in. Sorry to hear that.
I assume these drugs take effect gradually, and that the effects are dramatic enough that the victim is (eventually) aware of them.
If so, mightn’t it be better to warn everyone about the problem (be on the lookout for drug effects!), and advise victims to inform others about the scumbags doing this? (Under the assumption that most of them time when people are drugged it’ll be fairly obvious who did it.)
I’d think it would be better to drive the scumbags out of your community than to just find a way to prevent one attack vector. (Surely those who’d stoop to this will find other equally objectionable ways to achieve their ends.)
It’s cool—a little too cool; I wonder how much was the effect from your cherry-picking answers.
Even so, I’d love to ask the simulation a few questions of my own.
We don’t know, true. But given the possible space of limiting parameters it seems unlikely that humans are anywhere near the limits. We’re evolved systems, evolved under conditions in which intelligence was far from the most important priority.
And of course under the usual evolutionary constraints (suboptimal lock-ins like backward wired photoreceptors in the retina, the usual limited range of biological materials—nothing like transistors or macro scale wheels, etc.).
And by all reports John von Neumann was barely within the “human” range, yet seemed pretty stable. He came remarkably close to taking over the world, despite there being only one of him and not putting any effort into it.
I think all you’re saying is there’s a small chance it’s not possible.
Yes, I think some web portals, and some software, are designed poorly because of malice. Not (usually) malice against users, but malice against managers and those setting requirements, when those people and their instructions are perceived as stupid and unreasonable.
One reaction to such demands is to deliver exactly what was requested—something stupid and unreasonable, in order to vividly demonstrate the stupid and unreasonable nature of the managers and requirements.
Sometimes professionalism, ethics, and dedication to user experience manage to overcome the natural human reaction to unreasonable requests. The more the developers are in an organization that rewards obedience over quality, the more likely the result will be due to malice.
For example, in Year 1 that useless letter “c” would be dropped to be replased either by “k” or “s”, and likewise “x” would no longer be part of the alphabet. The only kase in which “c” would be retained would be the “ch” formation, which will be dealt with later. Year 2 might reform “w” spelling, so that “which” and “one” would take the same konsonant, wile Year 3 might well abolish “y” replasing it with “i” and Iear 4 might fiks the “g/j” anomali wonse and for all.
Jenerally, then, the improvement would kontinue iear bai iear with Iear 5 doing awai with useless double konsonants, and Iears 6-12 or so modifaiing vowlz and the rimeining voist and unvoist konsonants. Bai Iear 15 or sou, it wud fainali bi posibl tu meik ius ov thi ridandant letez “c”, “y” and “x”—bai now jast a memori in the maindz ov ould doderez—tu riplais “ch”, “sh”, and “th” rispektivli.
Fainali, xen, aafte sam 20 iers ov orxogrefkl riform, wi wud hev a lojikl, kohirnt speling in ius xrewawt xe Ingliy-spiking werld.
Based solely on personal experience (N=1), don’t exceed 2 cups/day.
More than that and I used to get headaches on weekends when I didn’t drink it. At 2 or less cups/day, no problem.