I mean efficiently in terms of number of bits, not computation time. Which contributes to posterior probability.
Yes, I agree. “Reference class” is a property of some models, not all models.
At this point it seems simplest to construct your reference class so as to only contain agents that can be found using the same procedure as yourself. Since you have to be decidable for the hypothesis to predict your observations, all others in your reference class are also decidable.
If there’s a constant-length function mapping the universe description to the number of agents in that universe, doesn’t that mean K(n) can’t be more than the Kolmogorov complexity of the universe by more than that constant length?
If it isn’t constant-length, then it seems strange to assume Solomonoff induction would posit a large objective universe, given that such positing wouldn’t help it predict its inputs efficiently (since such prediction requires locating agents).
This still leads to the behavior I’m talking about in the limit; the sum of 1/2^K(n) over all n can be at most 1 so the probabilities on any particular n have to go arbitrarily small in the limit.
My understanding is that Solomonoff induction leads to more SSA-like behavior than SIA-like, at least in the limit, so will reject the presumptuous philosopher’s argument.
Asserting that there are n people takes at least K(n) bits, so large universe sizes have to get less likely at some point.
Active nihilism described in the paragraph definitely includes, but is not limited to, the negation of values. The active nihilists of a moral parliament may paralyze the parliament as a means to an end; perhaps, to cause systems other than the moral parliament to be the primary determinants of action, rather than the moral parliament.
What you are describing is a passive sort of nihilism. Active nihilism, on the other hand, would actively try to negate the other values. Imagine a parliament where whenever a non-nihilist votes in favor of X, a nihilist votes against X, such that these votes exactly cancel out. Now, if (active) nihilists are a majority, they will ensure that the parliament as a whole has no aggregate preferences.
CDT and EDT have known problems on 5 and 10. TDT/UDT are insufficiently formalized, and seem like they might rely on known-to-be-unfomalizable logical counterfactuals.
So 5 and 10 isn’t trivial even without spurious counterfactuals.
What does this add over modal UDT?
No requirement to do infinite proof search
More elegant handling of multi-step decision problems
Also works on problems where the agent doesn’t know its source code (of course, this prevents logical dependencies due to source code from being taken into account)
Philosophically, it works as a nice derivation of similar conclusions to modal UDT. The modal UDT algorithm doesn’t by itself seem entirely
well-motivated; why would material implication be what to search for? On the other hand, every step in the linear logic derivation is quite natural, building action into the logic, and encoding facts about what the agent can be assured of upon taking different actions. This makes it easier to think clearly about what the solution says about counterfactuals, e.g. in a section of this post.
Why lie on the d100 coming up 1 instead of “can neither confirm nor deny”?
Note: the provided utility function is incredibly insecure; even a not-very-powerful individual can manipulate the AI by writing down that hash code under certain conditions.
Also, the best way to minimize V + W is to minimize both V and W (i.e. write the hash code and create hell). If we replace this with min(V, W) then the AI becomes nihilistic if someone writes down the hash code, also a significant security vulnerability.
Reals are still defined as sets of (a, b) rational intervals. The locale contains countable unions of these, but all these are determined by which (a, b) intervals contain the real number.
Good point; I’ve changed the wording to make it clear that the rational-delimited open intervals are the basis, not all the locale elements. Luckily, points can be defined as sets of basis elements containing them, since all other properties follow. (Making the locale itself countable requires weakening the definition by making the sets to form unions over countable, e.g. by requiring them to be recursively enumerable)
I’ve also been thinking about the application of agency abstractions to decision theory, from a somewhat different angle.
It seems like what you’re doing is considering relations between high-level third-person abstractions and low-level third-person abstractions. In contrast, I’m primarily considering relations between high-level first-person abstractions and low-level first-person abstractions.
The VNM abstraction itself assumes that “you” are deciding between different options, each of which has different (stochastic) consequences; thus, it is inherently first-personal. (Applying it to some other agent requires conjecturing things about that agent’s first-person perspective: the consequences it expects from different actions)
In general, conditions of rationality are first-personal, in the sense that they tell a given perspective what they must believe in order to be consistent.
The determinism vs. free will paradox comes about when trying to determine when a VNM-like choice abstraction is valid of a third-personal physical world.
My present view of physics is that it is also first-personal, in the sense that:
If physical entities are considered perceptible, then there is an assumed relation between them and first-personal observations.
If physical entities are causal in a Pearlian sense, then there is an assumed relation between them and metaphysically-real interventions, which are produced through first-personal actions.
Decision theory problems, considered linguistically, are also first-personal. In the five and ten problem, things are said about “you” being in a given room, choosing between two items on “the” table, presumably the one in front of “you”. If the ability to choose different dollar bills is, linguistically, considered a part of the decision problem, then the decision problem already contains in it a first-personal VNM-like choice abstraction.
The naturalization problem is to show how such high-level, first-personal decision theory problems could be compatible with physics. Such naturalization is hard, perhaps impossible, if physics is assumed to be third-personal, but may be possible if physics is assumed to be first-personal.
Looking back on this, it does seem quite similar to EDT. I’m actually, at this point, not clear on how EDT and TDT differ, except in that EDT has potential problems in cases where it’s sure about its own action. I’ll change the text so it notes the similarity to EDT.
On XOR blackmail, SIDT will indeed pay up.
Yes, it’s about no backwards assumption. Linear has lots of meanings, I’m not concerned about this getting confused with linear algebra, but you can suggest a better term if you have one.
Epistemic Status: Something Is Wrong On The Internet.
Epistemic Status: Something Is Wrong On The Internet.
If you think this applies, it would seem that “The Internet” is being construed so broadly that it includes the mainstream media, policymaking, and a substantial fraction of people, such that the “Something Is Wrong On The Internet” heuristic points against correction of public disinformation in general.
This is a post that is especially informative, aligned with justice, and likely to save lives, and so it would be a shame if this heuristic were to dissuade you from writing it.
The presumption with conspiracies is that they are engaged in for some local benefit by the conspiracy at the detriment of the broader society. Hence, the “unilateralist’s curse” is a blessing in this case, as the overestimation by one member of a conspiracy of their own utility in having the secret exposed, brings their estimation more in line with the estimation of the broader society, whose interests differ from those of the conspirators.
If differences between the interests of different groups were not a problem, then there would be no motive to form a conspiracy.
In general, I am quite annoyed at the idea of the unilateralist’s curse being used as a general argument against the revelation of the truth, without careful checking of the correspondence between the decision theoretic model of the unilateralist’s curse and the actual situation, which includes crime and conflict.
A major problem with physicalist dismissal of experiential evidence (as I’ve discussed previously) is that the conventional case for believing in physics is that it explains experiential evidence, e.g. experimental results. Solomonoff induction, among the best formalizations of Occam’s razor, believes in “my observations”.
If basic facts like “I have observations” are being doubted, then any case for belief in physics has to go through something independent of its explanations of experiential evidence. This looks to be a difficult problem.
You could potentially resolve the problem by saying that only some observations, such as those of mechanical measuring devices, count; however, this still leads to an analogous problem to the hard problem of consciousness, namely, what is the mapping between physics and the outputs of the mechanical measuring devices that are being explained by theories? (The same problem comes up of “what data is the theorizing trying to explain” whether the theorizing happens in a single brain or in a distributed intelligence, e.g. a collection of people using the scientific method)