But don’t you see—those infections are a second wave, so do not have to be counted. The model is almost tautologically true that way. But terribly misleading, and very irresponsibly so.
Anon User
Karma: 455
I wonder whether you may be conflating two somewhat distinct (perhaps even orthogonal) challenges not modeled in the CIDR model:
Human actions may be reflecting human values very imperfectly (or worse—can be an imperfect reflection of inconsistent conflicting values).
Some actions by AI may damage the human, at which point the human actions may stop being meaningfully correlated with the value function. This is a problem that would have still be relevant if we somehow found an ideal human capable of acting on their values in a perfectly rational manner.
The first challenge “only” requires the AI to be better at deducing the “real” values. (“Only” is in quotes because it’s obviously still a major unsolved problem, and “real” is in quotes because it’s not a given what that actually means.). The second challenge is about AI needing to be constrained in its actions even before it knows the value function—but there is at least a whole field of Safe RL on how do do this for much simpler tasks, like learning to move a robotic arm without breaking anything in the process.
I think you might be trying to apply the concept at a wrong granularity. Yes, there is often an iterative combination of the fundamental and applied, but then you need to be classifying each iterative step, rather than the white sequence, and the point is that it’s a “Pasteur-Edison” iteration, not a “Bohr-Edison” one. Almost any new fundamental advance has to go through the “Edison” phase as the technology readiness grows, before it becomes practical. This is true whether the advance came from “Bohr” quadrant, it “Pasteur” one. The distinction is whether you are mindful of the potential applications when you were embarking on doing the fundamental part (“Pasteur”), or whether the practical implications were only figured out after the fact (“Bohr”). The distinction becomes particularly pronounced when the research effort is only proposed, and you are asking for funding.
Your world descriptions and your objections seem to focus on HRAD being the only prerequisite to being able to create an aligned AGI, rather than simply one of them (and is the one worth focusing on because of a combination of factors, such as—which areas of research are the least attended to by other researches, which areas could provide insights useful to then attack other ones, which ones are the most likely to be on a critical path, etc). It could very well be an “overwhelming priority” as you stated the position you are trying to understand, without the goal being “to come up with a theory of rationality [...] that [...] allows one to build an agent from the ground up”.
I am thinking of the following optimization problem. Let R1 be all the research that we anticipate getting completed by the mainstream AI community by the time they create an AGI. Let R2 be the smallest amount of successful research such that R1+R2 allows you to create an aligned AGI. What research questions we know to formulate today, and have a way to start attacking today that are the most likely to be in R2? And among the top choices, which ones are also 1) more likely to produce insights that would help with other parts of R2, and 2) less likely to compress the AGI timeline even further? It seems possible to believe in HRAD being such a good choice (working backwards from R2) without being in one of your world’s (all of which work forward from HRAD).
Perhaps one aspect of minimum wage that you are missing is that this is different from price control of fungible goods is several important aspects, that everything else being equal:
Higher minimum wage means higher demand for goods consumed by minimum wage employees.
Higher minimum wage incentivises employers to invest more in their employee productivity (training, better work conditions, etc)
Same employees may be more productive if you pay them higher wages, and you may be able to get better employees.
In some cases 2+3 might means that there may be several equilibrium points that are roughly equally good for the employers—either hire high-turnover low-productivity people with lower wages, or hire lower-turnover higher-productivity people for higher wages, and effect #1 is enough for the higher minimum wage to just be a win-win (which is perhaps why some employers actually support minimum wage laws).
What are the “unnatural” deaths—are they things like car accidents? For those I’d expect them to actually go down pretty significantly because of the significantly reduced mobility.
First, poor have lower savings rate, and consume faster, so money velocity is higher. Second, minimal wages are local, and I would imagine that poor people on average spend a bigger fraction of their consumption locally (but I am not as certain about this one).
More of an anecdote than research, but I recently became aware of Dr. A.J Cronin’s novel “The Citadel” published in 1937 and the claim that the book prompted new ideas about medicine and ethics, inspiring to some extent the UK NHS and the ideas behind it. Did not look into this much myself, but certainly a very fascinating story, if true.
Well, this is Committee on Armed Services—obviously the adversarial view of things is kind of a part of their job description… (Not that this isn’t a problem, just pointing out that they are probably not the best place to look for a non-adversarial opinion).
In mathematics, axioms are not just chosen based of what feels correct—instead, the implications of those axioms are explored, and only if those seem to match the intuition too, then the axioms have some chance of getting accepted. If a reasonably-seeming set axioms allows you to prove something that clearly should not be provable (such as—in the extreme case—a contradiction), then you know your axioms are no good.
Axiomatically stating a particular ethical framework, then exploring the consequences of the axioms in the extreme and tricky cases can serve a similar purpose—if simingly sensible ethical “axioms” lead to completely unreasonable conclusions, then you know you have to revise the stated ethical framework in some way.
Random data point—https://ftx.com/trade/TRUMPFEB (“Trump is the President on Feb 1st, 2021”) is currently at 0.142 (14.2% probability it will happen)...
I think your analysis of “you’re only X because of Y” is missing the “you are doing it wrong” implicit accusation in the statement. Basically, the implied meaning, I think, is that while there are acceptable reasons to X, you are lacking any of them, but instead your reason for X is Y, which is not one of the acceptable reasons. Which is why your Z is a defense—claiming to have reasons in the acceptable set. And another defense might be to respond entirely to the implied accusation and explain why Y should be an OK reason to X. “You’re only enjoying that movie scene because you know what happened before it”—“Yeah, and what’s wrong with that?”
I do not see the connection. The gist of Newcomb’s Problem does not change if the player is given a time limit (you have to choose within an hour, or you do not get anything). Time-limited halting problem is of course trivially decidable.
Right, which is why the claim is immediately more suspect if Xavier is a close friend/relative/etc.
The way you described the chess/marriage/etc market, it’s a bit vulnerable. Imagine there is a move that appears to be a very strong one, but with a small possibility of a devastating countermove that is costly for market participants to analyze. There is an incentive to bet on it—if the countermove exists, hopefully somebody will discover it, heavily bet against the move, and cause the price to drop enough that it is not taken, and the bets are refunded. If no countermove exists, the bet is a good one, and is profitable. But if nobody bothers to check for the countermove, and it exists, everybody (those who bet on the move, and the decision makers who made the move) are in trouble, but it could still be the case that no bettors have enough incentive to check for countermove (if it exists, they do not derive any benefit from the significant mispricing of the move, as you just refund the bets).
Note that Kelly is valid under the assumption that you know the true probabilities. II do not know whether it is still valid when all you know is a noisy estimate of true probabilities—is it? It definitely gets more complicated when you are betting against somebody with a similarly noisy estimate of the same probably, as at some level you now need to take their willingness to bet into account when estimating the true probability—and the higher they are willing to go, the stronger the evidence that your estimate may be off. At the very least, that means that the uncertainty of your estimate also becomes the factor (the less certain you are, the more attention you should pay to the fact that somebody is willing to bet against you). Then the fact that sometimes you need to spend money on things, rather than just investing/betting/etc, and that you may have other sources of income, also complicates the calculus.
It seems you are actually describing a 3-algorithm stack view for both the human and AGI. For human, there is 1) evolution working on the genome level, there is 2) long-term brain development / learning, and there is 3) the brain solving a particular task. Relatively speaking, evolution (#1) works on much smaller number much more legible parameters than brain development (#2). So if we use some sort of genetic algorithm for optimizing AGI meta-parameters, then we’d get a very stack that is very similar in style. And in any case we need to worry about “base” optimizer used in the AGI version of #1+#2 producing an unaligned mesa-optimizer for AGI version of the #3 algorithm.
Here is yet another reason this trade may be irrational. If souls were real, then I’d expect the value of a soul to be quite high. For the sake of the argument, let’s posit the value of a soul (if it existed) at $1M. Now the question is—can you make 100,000 statements that you are about as certain of being true as the statement “souls do not exist” and not make even a single mistake? If the answer is “no” (and it’s probably “no” for all but the most careful people), then the habit of selling souls for $10 is a bad habit to have—sooner or later you’d mess up and sell something way too valuable.
OK, sorry, but can somebody please explain the banana anecdote? What is supposed to be so obviously wrong with this approach? I seriously did not get it.
P.S. Otherwise, great writing—but can I suggest using more transparent anecdotes for your illustrations?
They are not very explicit about it (which is a huge problem by itself), but they seem to be saying that they are only predicting the “first wave”—so they are not predicting 0 deaths after July—they just defining them to not be a part of the “first wave” anymore. So the way they present the model predictions is even more unbelievably wrong than the model itself!