It makes sense to one-box ONLY if you calculate EV by that assigns a significant probability to causality violation
It only makes sense to two-box if you believe that your decision is causally isolated from history in every way that Omega can discern. That is, that you can “just do it” without it being possible for Omega to have predicted that you will “just do it” any better than chance. Unfortunately this violates the conditions of the scenario (and everyday reality).
It seems to me that the problem in the counterlogical mugging isn’t about how much computation is required for getting the answer. It’s about whether you trust Omega to have not done the computation beforehand, and whether you believe they actually would have paid you, no matter how hard or easy the computation is. Next to that, all the other discussion in that section seems irrelevant.
Oh, sure. I was wondering about the reverse question: is there something that doesn’t really qualify as torture where subjecting a billion people to it is worse than subjecting one person to torture.
I’m also interested in how this forms some sort of “layered” discontinuous scale. If it were continuous, then you could form a chain of relations of the form “10 people suffering A is as bad as 1 person suffering B”, “10 people suffering B is as bad as 1 person suffering C”, and so on to span the entire spectrum.
Then it would take some additional justification for saying that 100 people suffering A is not as bad as 1 person suffering C, 1000 A vs 1 D, and so on.
Is there some level of discomfort short of extreme torture for a billion to suffer where the balance shifts?
It makes sense to very legibly one-box even if Omega is a very far from perfect predictor. Make sure that Omega has lots of reliable information that predicts that you will one-box.
Then actually one-box, because you don’t know what information Omega has about you that you aren’t aware of. Successfully bamboozling Omega gets you an extra $1000, while unsuccessfully trying to bamboozle Omega loses you $999,000. If you can’t be 99.9% sure that you will succeed then it’s not worth trying.
Almost.
The argument doesn’t rule out substance dualism, in which consciousness may not be governed by physical laws, but in which it is at least causally connected to the physical processes of writing and talking and neural activity correlated with thinking about consciousness. It’s only an argument against epiphenomenalism and related hypotheses in which the behaviour or existence of consciousness has no causal influence on the physical universe.
I don’t think this was a statement about whether it’s possible in principle, but about whether it’s actually feasible in practice. I’m not aware of any conlangs, before the cutoff date or not, that have a training corpus large enough for the LLM to be trained to the same extent that major natural languages are.
Esperanto is certainly the most widespread conlang, but (1) is very strongly related to European languages, (2) is well before the cutoff date for any LLM, (3) all training corpora of which I am aware contain a great many references to other languages and their cross-translations, and (4) the largest corpora are still less than 0.1% of those available for most common natural languages.
No, Eliezer does not say that consciousness itself cannot be what causes you to think about consciousness. Eliezer says that if p-zombies can exist, then consciousness itself cannot be what causes you to think about consciousness.
If p-zombies cannot exist, then consciousness can be a cause of you thinking about consciousness.
Is conversation of expected evidence a reasonably maintainable proposition across epistemically hazardous situations such as memory wipes (or false memories, self-duplicates and so on)? Arguably, in such situation it is impossible to be perfectly rational since the thing you do your reasoning with is being externally manipulated.
I disagreed on prep time. Neither I nor anyone I know personally deliberately waits minutes between taking ice cream out of the freezer and serving it.
I could see hardness and lack of taste being an issue for commercial freezers that chill things to −25 C, but not a typical home kitchen freezer at more like −10 to −15 C.
Isn’t this basically just a negotiated trade, from a game theory point of view? The only uncommon feature is that A intrinsically values s at zero and B knows this (and that A is the only supplier of s). This doesn’t greatly affect the analysis though, since most of the meat of the problem is what division of gains of trade may be acceptable to both.
Unfortunately, I think all three of those listed points of view poorly encapsulate anything related to moral worth, and hence evaluating unaligned AIs from them is mostly irrelevant.
They do all capture some fragment of moral worth, and under ordinary circumstances are moderately well correlated with it, but the correlation falls apart out of the distribution of ordinary experience. Unaligned AGI expanding to fill the accessible universe is just about as far out of distribution as it is possible to get.
Good point! Lewis’ notation P_+(HEADS) does indeed refer to the conditional credence upon learning that it’s Monday, and he sets it to 2⁄3 by reasoning backward from P(HEADS) = 1⁄2 and using my (1).
So yes, there are indeed people who believe that if Beauty is told that it’s Monday, then she should update to believing that the coin was more likely heads than not. Which seems weird to me—I have a great deal more suspicion that (1) is unjustifiable than that (2) is.
What is a “2nd CWT” as referenced in the title? The term doesn’t appear anywhere in the post.
They … what? I’ve never read anything suggesting that. Do you have any links or even a memory of an argument that you may have seen from such a person?
Edit: Just to clarify, conditional credence P(X|Y) is of the form “if I knew Y held, then my credence for X would be …”. Are you saying that lots of people believe that if they knew it was Monday, then they would hold something other than equal credence for heads and tails?
Generally I think the 1⁄3 argument is more appealing, just based on two principles:
Credences should follow the axioms of a probability space (including conditionality);
The conditional credence for heads given that it is Monday, is 1⁄2.
That immediately gives P(Heads & Monday) = P(Tails & Monday). The only way this can be compatible with P(Heads) = 1⁄2 is if P(Tails & Tuesday) = 0, and I don’t think anybody supports that!
P(Tails & Tuesday) = P(Tails & Monday) isn’t strictly required by these principles, but it certainly seems a highly reasonable assumption and yields P(Heads) = 1⁄3.
I don’t think anybody disagrees with principle (2).
Principle (1) is somewhat more dubious though. Since all credences are conditional on epistemic state, and this experiment directly manipulates epistemic state (via amnesia), it is arguable that “rational” conditional credences might not necessarily obey probability space rules.
Of course you’re predicting something different. In all cases you’re making a conditional prediction of a state of the world given your epistemic state at the time. Your epistemic state on Wednesday is different from that on Monday or Tuesday. On Tuesday you have a 50% chance of not being asked anything at all due to being asleep, which breaks the symmetry between heads and tails.
By Wednesday the symmetry may have been restored due to the amnesia drug—you may not know whether the awakening you remember was Monday (which would imply heads) or Tuesday (which would imply tails). However, there may be other clues such as feeling extra hungry due to sleeping 30+ hours without eating.
The “Origins of Happiness” excerpt had rather a lot of ill-founded statements and obviously invalid (and/or very grossly misstated) arguments for them. I didn’t read much further into the next excerpt before downvoting.
I certainly didn’t click through to the site as I don’t expect the author chose the worst sections to quote, so I expect the rest is at least as bad.
Ordinarily I would just downvote silently and move on, since there was just too much to unpack for a constructive discussion on improvements. However, in this case the author specifically noted the downvotes and asked for comments. I certainly won’t give a complete coverage of what I found objectionable, but here’s a very brief list of just the first few that occurred to me while reading:
How does evolution go backward in time and downward in scope from fitness of current societies, to evolution of happiness in humans at least tens of thousands of years ago (and almost certainly millions of years ago in pre-humans)?
The argument jumps between happiness and kindness with just a passing mention of “correlation”. If kindness is the relevant factor for evolutionary fitness (disregarding the first objection), why should it be correlated at all with happiness? If you’re trying to blame happiness on evolution, it’s not enough to say that there is a correlation, you have to provide a mechanism by which evolution caused the correlation.
Where is the evidence that people with less kindness necessarily form less effective societies that fail to survive?
Large states extinguishing both kindness and happiness is just stated as a fact. Where is the evidence?
… and so on. On reading more deeply and carefully, I found only more problems and none of the original issues I had were resolved.
Granting that LLMs in inference mode experience qualia, and even granting that they correspond to human qualia in any meaningful way:
I find both arguments invalid. Either conclusion could be correct, or neither, or the question might not even be well formed. At the very least, the situation is a great deal more complicated than just having two arguments to decide between!
For example in scenario (A), what does it mean for an LLM to answer a question “eagerly”? My first impression is that it’s presupposing the answer to the question, since the main meaning of “eagerly” is approximately “in the manner of having interest, desire, and/or enjoyment”. That sounds a great deal like positive qualia to me!
Maybe it just means the lesser sense of apparently showing such emotions, in which case it may mean no more than an author writing such expressions for a character. The author may actually be feeling frustration that the scene isn’t flowing as well as they would like and they’re not sure that the character’s behaviour is really in keeping with their emotions from recent in-story events. Nonetheless, the words written are apparently showing eagerness.
The “training loss” argument seems totally ill-founded regardless. That doesn’t mean that its conclusion in this hypothetical instance is false, just that the reasoning provided is not sufficient justification for believing it.
So in the end, I don’t see this as a dilemma at all. It’s just two possible bad arguments out of an enormously vast space of bad arguments.
GDP is a rather poor measure of wealth, and was never intended to be a measure of wealth but of something related to productivity. Since its inception it has never been a stable metric, as standards on how the measure is defined have changed radically over time in response to obvious flaws for any of its many applications. There is widespread and substantial disagreement on what it should measure and for which purposes it is a suitable metric.
It is empirically moderately well correlated with some sort of aggregate economic power of a state, and (when divided by population) some sort of standard of living of its population. As per Goodhart’s Law, both correlations weakened when the metric became a target. So the question is on shaky foundation right from the beginning.
In terms of more definite questions such as price of food and agricultural production, that doesn’t really have anything to do with GDP or virtual reality economy at all. Rather a large fraction of final food price goes to processing, logistics, finance, and other services, not the primary agriculture production. The fraction of price paid by food consumers going to agricultural producers is often less than 20%.