You are faster than me, I took a few years before realizing I should probably be doing something different with my life than just proving math theorems. I guess talking with my advisors was easier because they don’t have any domain knowledge in AI so they didn’t have any real opinion about whether AI safety is important. Anyway, I am doing a summer internship at MIRI, so maybe I will see you around.
Dacyn(David Simmons)
Karma: 512
a random letter contains about 7.8 (bits of information)
This is wrong, a random letter contains log(26)/log(2) = 4.7 bits of information.
-”If all that The Rock is cooking is setting the probability of every possible change to epsilon, then when the first of those events happens his Briar score is suddenly going to explode and he is going to lose all his Bayes points.”
I think you are thinking of logarithmic scoring. With the Brier score, a wrong 100% prediction is scored the same as just four (wrong or right) 50% predictions, hardly an “explo[sion]”.
Once you introduce any meaningful uncertainty into a non-Archimedean utility framework, it collapses into an Archimedean one. This is because even a very small difference in the probabilities of some highly positive or negative outcome outweighs a certainty of a lesser outcome that is not Archimedean-comparable. And if the probabilities are exactly aligned, it is more worth your time to do more research so that they will be less aligned, than to act on the basis of a hierarchically less important outcome.
For example, if we cared infinitely more about not dying in a car crash than about reaching our destination, we would never drive, because there is a small but positive probability of crashing (and the same goes for any degree of horribleness you want to add to the crash, up to and including torture—it seems reasonable to suppose that leaving your house at all very slightly increases your probability of being tortured for 50 years).
For the record, EY’s position (and mine) is that torture is obviously preferable. It’s true that there will be a boundary of uncertainty regardless of which answer you give, but the two types of boundaries differ radically in how plausible they are:
if SPECKS is preferable to TORTURE, then for some N and some level of torture X, you must prefer 10N people to be tortured at level X than N to be tortured at a slightly higher level X’. This is unreasonable, since X is only slightly higher than X’, while you are forcing 10 times as many people to suffer the torture.
On the other hand, if TORTURE is preferable to SPECKS, then there must exist some number of specks N such that N-1 specks is preferable to torture, but torture is preferable to N+1 specks. But this is not very counterintuitive, since the fact that torture costs around N specks means that N-1 specks is not much better than torture, and torture is not much better than N+1 specks. So knowing exactly where the boundary is isn’t necessary to get approximately correct answers.
Moral realism plus moral internalism does not imply heterogonality. Just because there is an objectively correct morality, does not mean that any sufficiently powerful optimization process would believe that that morality is correct.
Calvinists believe in predestination, not Protestants in general.
1) The notion of a “perfectly selfish rational agent” presupposes the concept of a utility function. So does the idea that agent A’s strategy must depend on agent B’s which must depend on agent A’s. It doesn’t need to depend, you can literally just do something. And that is what people do in real life. And it seems silly to call it “irrational” when the “rational” action is a computation that doesn’t converge.
2) I think humanity as a whole can be thought of as a single agent. Sure maybe you can have a person who is “approximately that selfish”, but if they are playing a game against human values, there is nothing symmetrical about that. Even if you have two selfish people playing against each other, it is in the context of a world infused by human values, and this context necessarily informs their interactions.
I realize that simple games are only a proxy for complicated games. I am attacking the idea of simple games as a proxy for attacking the idea of complicated games.
3) Eliezer definitely says that when your decision is “logically correlated” with your opponent’s decision then you should cooperate regardless of whether or not there is anything causal about the correlation. This is the essential idea of TDT/UDT. Although I think UDT does have some valuable insights, I think there is also an element of motivated reasoning in the form of “it would be nice if rational agents played (C,C) against each other in certain circumstances rather than (D,D), how can we argue that this is the case”.
Regarding silence after the last pixel of sun, “no pre-planning” is not exactly right, there were some people passing around the message that that was what we were supposed to do. It was a little ad-hoc though.
Update: I moved to Berkeley last week and noticed a huge difference in how the rationalist/EA community deals with these sorts of conversations and how the rest of the world does. Yesterday I was talking to someone I had barely met and they asked “how are you doing?” I said “you just opened a whole can of worms” and we ended up having an interesting discussion, including about how the conversational norms are different here from elsewhere. In general, I think people in this community are both more likely to give an honest answer to such questions, and less likely to ask them if they aren’t interested in an honest answer.
I’d thought the Hilbert space was uncountably dimensional because the number of functions of a real line is uncountable. But in QM it’s countable… because everything comes in multiples of Planck’s constant, perhaps? Though I haven’t seen the actual reason stated, and perhaps it’s something beyond my current grasp.
Ahh… here’s something I can help with. To see why Hilbert space has a countable basis, let’s first define Hilbert space. So let
= the set of all functions such that the integral of is finite, and let
= the set of all functions such that the integral of is zero. This includes for example the Dirichlet function which is one on rational numbers but zero on irrational numbers. So it’s actually a pretty big space.
Hilbert space is defined to be the quotient space . To see that it has a countable basis, it suffices to show that it contains a countable dense set. Then the Gram-Schmidt orthogonalization process can turn that set into a basis. What does it mean to say that a set is dense? Well, the metric on Hilbert space is given by the formula
%20=%20\sqrt{\int%20|f%20-%20g|%5E2}),so a sequence is dense if for every element of Hilbert space, you can find a sequence such that
%20\to%200). Now we can see why we needed to mod out by -- any two points of are considered to have distance zero from each other!So what’s a countable dense sequence? One sequence that works is the sequence of all piecewise-linear continuous functions with finitely many pieces whose vertices are rational numbers. This class includes for example the function defined by the following equations:
%20=%200) for all%20=%20(1/2%20+%20x)/3) for all%20=%20(1/2%20-%20x)/3) for all%20=%200) for allNote that I don’t need to specify what does if I plug in a number in the finite set
, since any function which is zero outside of that set is an element of , so would represent the same element of Hilbert space as .So to summarize:
The uncountable set that you would intuitively think is a basis for Hilbert space, namely the set of functions which are zero except at a single value where they are one, is in fact not even a sequence of distinct elements of Hilbert space, since all these functions are elements of , and are therefore considered to be equivalent to the zero function.
The actual countable basis for Hilbert space will look much different, and the Gram-Schmidt process I alluded to above doesn’t really let you say exactly what the basis looks like. For Hilbert space over the unit interval, there is a convenient way to get around this, namely Parseval’s theorem, which states that the sequences
%20=%20\cos(2\pi%20nx)) and%20=%20\sin(2\pi%20nx)) form a basis for Hilbert space. For Hilbert space over the entire real line, there are some known bases but they aren’t as elegant, and in practice we rarely need an explicit countable basis.Finally, the philosophical aspect: Having a countable basis means that elements of Hilbert space can be approximated arbitrarily well by elements which take only a finite amount of information to describe*, much like real numbers can be approximated by rational numbers. This means that an infinite set atheist should be much more comfortable with countable-basis Hilbert space than with uncountable-basis Hilbert space, where such approximation is impossible.
* The general rule is:
Elements of a finite set require a finite and bounded amount of information to describe.
Elements of a countable set require a finite but unbounded amount of information to describe.
Elements of an uncountable set (of the cardinality of the continuum) require a countable amount of information to describe.
To repeat what was said in the CFAR mailing list here: This “bet” isn’t really a bet, since there is no upside for the other party; they are worse off than when they started in every possible scenario.
In constructivist logic, proof by contradiction must construct an example of the mathematical object which contradicts the negated theorem.
This isn’t true. In constructivist logic, if you are trying to disprove a statement of the form “for all x, P(x)”, you do not actually have to find an x such that P(x) is false—it is enough to assume that P(x) holds for various values of x and then derive a contradiction. By contrast, if you are trying to prove a statement of the form “there exists x such that P(x) holds”, then you do actually need to construct an example of x such that P(x) holds (in constructivist logic at least).
It can delete its timeline if others exist.
Sounds like you are assuming some quantum suicide framework. This goes beyond normal quantum mechanics.
If it attempts to act on anything but the memory
I think the issue is supposed to be that it is not clear what this phrase means. Any change in memory will have many physical consequences on the outside world; how can we know which of these are important?
I don’t really see what the point of this security measure is. If you want to develop an AI whose only purpose is to find a proof of some already formulated theorem, well, it is not clear why such an AI would need to be general-purpose, and it is only with general-purpose AIs that you have the problem that they might try to do things other than just “acting on the memory”.
Finally, it is not clear how plausible it is that there could be a “ZF-formulable safety guarantee”—this seems to imply that we have some argument that the AI is safe if a certain mathematical condition holds, but not otherwise, such that the mathematical condition is complicated enough that it is not clear whether it holds. I guess I am not aware of any deep purely mathematical questions coming from AI safety research, most of the questions are about a mix of math and philosophy.
Couldn’t you equally argue that they will do their best not to be smallest by not putting any money in all their opponent’s boxes? After all, “second-fullest” is the same as “third-emptiest”.
The question “how would the coin have landed if I had guessed tails?” seems to me like a reasonably well-defined physical question about how accurately you can flip a coin without having the result be affected by random noise such as someone saying “heads” or “tails” (as well as quantum fluctuations). It’s not clear to me what the answer to this question is, though I would guess that the coin’s counterfactual probability of landing heads is somewhere strictly between 0% and 50%.
There can be amounts of things other than suffering, though. Caring about the “number of chickens that lead meaningful lives” doesn’t mean that one isn’t a utilitarian. (For the record, I agree with the OP that the notion of “leading meaningful lives” isn’t so important for animals, but I think it’s possible to disagree with this and still be advocating an EA intervention.)
If neither player folds – either both players check, or there is a bet and a call – then both cards are revealed and the player with the higher card takes all 4 chips.
Presumably you mean to say that if both players check, the player with the higher card takes the 2 chip pot, not 4 chips?
Anyway here is the unique (up to choice of x) Nash equilibrium, if my calculations are correct:
1 tbrf svefg: org jvgu cebonovyvgl k, arire pnyy
2 tbrf svefg: arire org, pnyy jvgu cebonovyvgl k+1/3
3 tbrf svefg: org jvgu cebonovyvgl 3k, nyjnlf pnyy
1 tbrf frpbaq: org jvgu cebonovyvgl 1⁄3, arire pnyy
2 tbrf frpbaq: arire org, pnyy jvgu cebonovyvgl 1⁄3
3 tbrf frpbaq: nyjnlf org, nyjnlf pnyy
The whole calculation is based on the premise that Neal’s concept of “full non-indexical conditioning” is a reasonable way to do probability theory. Usually you do probability theory on what you are calling “centered propositions”, and you interpret each data point you receive as the proposition “I have received this data”. Not as “There exists a version of me which has received this data as well as all of the prior data I have received”. It seems really odd to do the latter, and I think more motivation is needed for it. (To be fair, I don’t have a better alternative in mind.)
I think Robin uses the word “want” pretty differently from most people (IIRC he’s said explicitly it doesn’t necessarily have anything to do with conscious thought processes). You just have to get used to his way of talking. (He also appears to have some strange beliefs about how it is a good thing to give people the things that they “want” in his sense, but I think that argument would have to be had on its own terms.)
On the Chatham House website I see
which seems reasonable. The comment about not circulating the attendee list beyond the participants is a response to the question “Can a list of attendees at the meeting be published?”, and my impression is that it is only meant as an answer to this question: i.e. such a list should not be published outside of the meeting, but it is OK if some people happen to come across it randomly. So I think you are just taking the Chatham Rule much more literally than it is intended.