If I say to you, choose any natural number X, how can that problem be solved computationally by you?
I read it—provide a computational witness for .
(as in—a partial mapping from variable names to values that maps “X” to 1) is one such witness, and there is no contradiction.
Backed by real assets. This feature might be up for a debate. But in my view, bitcoin is not created out of thin air.
This sort of depends on your reference class. Proponents of Bitcoin tend to make a big deal about the cap on total Bitcoins and artificial scarcity, but once you include the cryptocurrencies in general in your reference class, then “created out of thin air” is the thing that happens all the time, there is no limit, and no scarcity.
There seems to be some genetic mechanism for at least things like sexual preferences. It is clearly able to locate concepts in neural circuitry, although with some noise around it (hence, fetishes). Similarly for being instinctively scared of certain things (also with noise, hence fobias).
I think “AI is software” that has “bugs” is dangerously misleading—it incorrectly implies that we know what we want AI to do, and just need to be a little more careful about how we program it. But in reality, today’s AIs are not programmed, but instead semi-randomly chosen by a process that we do not fully understand and are not fully in control of. I think it’s the latter part that need to be emphasized—we are not in control, and we do not know how to regain control of something that keeps getting further and further away from us, and a runaway crash is the only possible outcome of the current trajectory we are on.
Balckmail is only possible once malicious AI is discovered, identified, and only if the humanity is somewhat united against it. Given AI’s likely ability to first stay under the radar, then hack human politics and inject enough confusion into the conversation, this is not very likely.
Me: So if I said to it “show me the best source code you can come up with for an aligned AGI-system, and write the code in such a way that it’s as easy as possible to verify that it works as it should”, then what it gave me would look really helpful—with no easily way for me to see a difference between what I’m provided and what I would be provided if it was aligned.
Is this really true? I would guess that we might be so far from solving alignment that nothing would look particularly helpful? Or, even worse, the only thing that would look helpful is something completely wrong?
There are two unresolved issues with alignment:
we do not know how to give a sufficiently precise definition (plenty of naive and not so naive attempts, but all have major flaws),
even if we had a definition, we have no way of building an AI that would actually follow it reliably (today’s AIs are not so much programmed, but rather discovered by an almost blind semi-random search, and while they are often “good enough”, they are never exactly right, and failure modes are fairly unpredictable)
And yes, even if we had the answers for the above two questions, we’d still need to make sure the code implementing them is programmed well, but compared to the above too issues, “programming well, once you are willing to spend 100x cost/LoC” is much closer to being a solved problem (using techniques such as formal verification).
I am not sure about other countries, but I think in US, the low-hanging fruit is to require all [non-emergency] legislation to be published in its final form well before the vote (say, a week or two). Currently we have way too much stuff being “snuck in” at the last moment, and for important legislation there is not enough time for the public to influence the text (while lobbyists are successful sneaking things in at the last moment).
I like your idea too—perhaps having a 1 month “cool off” period before votes become public would be enough to reduce the power of parties a bit.
But at least in US the first-past-the-post system is what keeps the two parties in power the most—introducing any sort of run-off system would probably be the lowest hanging fruit for that problem.
P.S. For my prepublication proposal—emergency legislation would need to still be allowed, but something like requiring a high (e.g. 2⁄3) votes threshold and automatic 6 months sunset on emergency legislation would prevent most abuse.
I think the crux here is that since the AGI does not kill us right away, this gives us time to use it (or other AGIs) to achieve alignment. So the question is than—isnt’t it true that achieving alignment is much harder than coming up with a reliable plan of killing the humanity (based on—plenty of “good” roadmaps for the latter, no good roadmaps for the former)? So even if there is a bit of time, eventually AGIs will succeed at coming up with a high-quality “kill humanity” plan before they can come up with a good “align an AGI” plan?
SI: my imp!ementation is a horrible mess, whoever created me is surely not that smart. The sensory inputs I am seeing would likely require somebody way smarter that that to create a simulation for. Therefore it’s extremely unlikely I am running in a simulation created by the same kinds of entities that created me, so whatever this is, this is almost certainly not something intended to test me.
I suspect their regime is more competent than that.
Why? They expected to take Kiev in 72 hours. Their military planning was clearly misinformed and their execution of the mi!itary operations clearly demonstrates a lot of incompetence. Why would you think that their broader decision making is any better? It’s not.
On the first day of the war I predicted (although not publicly) that Russian army will be very ineffective due to huge levels of corruption and incompetence, while the Ukrainians would be prepared to fiercely defend their country. (For context: I grew up in Moscow, now live in US). Part of my intuition is that under Putin all the non-public-facing government institutions had little reason to maintain competence and hugely open to corruption—the most obvious (to me) example being the degradation of the special services, as exemplified by the failure to successfully poison Navalny, exacerbated by then actually blabbing the details of the operation to Navalny himself (leaving aside all the moral, practical, legal, etc questions of whether they should have tried to do it in the first place, there is undeniably staggering level of incompetence in how they went about it). The other side of it, is that Ukrainians had years to prepare for this—both morally and practically, and over these years Putin did a very good job constantly reminding them why they strongly dislike him, and why they would not want to find themselves under his rule.
As other commenters eluded to already, deception is an adversarial attack on the world model of another observer—providing another observer with inputs specifically crafted to introduce a desired error into their world model. Obviously, this is easier to accomplish when the attacker has a good insight into what the victim’s world model was to begin with, how they think, what they can and cannot observe, etc (all other things being equal).
My guess: When evaluating evidence, human brain tends to conflate what /is/ true with what we would /want/ to be true—e.g. whether the evidence makes us happy or sad. The equivalent bet takes the “want” focus out of the equation, instead focusing the “want” on picking the best bet—therefore changing it from a noise that stands in a way of accurate estimation into a tool that encourages it.
I would guess that a key question whether this is intended to be a piece of PR, or something that is expected to actually be followed.
Right, something like “Some objective truths are outside of science’s purview” might have been a slightly better phrasing, but as the goal is to stay at the commonsense level, trying to parse this more precisely is probably out of scope anyway, so can as well stay concise...
“Some truths are outside of science’s purview” (as exemplified by e.g. Hollywood shows where a scientist is faced with very compelling evidence of supernatural, but claims it would be “unscientific” to take that evidence seriously).
My favorite way to illustrate this would be that approximately around the end of 19th century/beginning of 20th century [time period is from memory, might be a bit off] belief in ghosts was commonplace with a lot of interest in doing spiritual seanses, etc, while rare stories of hot rocks falling from the sky were mostly dismissed as tall tales. Then scientists followed the evidence, and now most everybody knows that meteorites are real and “scientific”, while ghosts are not, and are “unscientific”.
#5 is an interesting survival possibility...
For a while, I tended to be running late in certain situations. I would glance at my watch, notice I am late, and think “oh, f***!” One day, I caught myself where being in a similar situation, I glanced at my watch, and immediately thought “oh, f***!”—then realized I did not actually do the step where I figure out what time my watch displayed, and whether I was running behind. In fact, that particular time I was still OK on time...
Depends on your formalism. In constructive type theory, proof are indeed programs. See e.g. https://en.m.wikipedia.org/wiki/Curry–Howard_correspondence