Benya 9 Sep 2012 22:32 UTC
30 points
on: Eliezer Yudkowsky Facts
When Eliezer Yudkowsky once woke up as Britney Spears, he recorded the world’s most-reviewed song about leveling up as a rationalist.

Eliezer Yudkowsky got Clippy to hold off on reprocessing the solar system by getting it hooked on HP:MoR, and is now writing more slowly in order to have more time to create FAI.

If you need to save the world, you don’t give yourself a handicap; you use every tool at your disposal, and you make your job as easy as you possibly can. That said, it is true that Eliezer Yudkowsky once saved the world using nothing but modal logic and a bag of suggestively-named Lisp tokens.

Eliezer Yudkowsky once attended a conference organized by some above-average Powers from the Transcend that were clueful enough to think “Let’s invite Eliezer Yudkowsky”; but after a while he gave up and left before the conference was over, because he kept thinking “What am I even doing here?”

Eliezer Yudkowsky has invested specific effort into the awful possibility that one day, he might create an Artificial Intelligence so much smarter than him that after he tells it the basics, it will blaze right past him, solve the problems that have weighed on him for years, and zip off to see humanity safely through the Singularity. It might happen, it might not. But he consoles himself with the fact that it hasn’t happened yet.

Eliezer Yudkowsky once wrote a piece of rationalist Harry Potter fanfiction so amazing that it got multiple people to actually change their lives in an effort at being more rational. (...hm’kay, perhaps that’s not quite awesome enough to be on this list… but you’ve got to admit that it’s in the neighbourhood.)

Benya 19 Sep 2012 16:23 UTC
29 points
in reply to: jimrandomh’s comment on: Less Wrong Polls in Comments
Current results: Red: 0%; Green, 33%; Blue: 67%; Other: 0%.

I’m gladdened to see that even though we don’t discuss politics on LW, the green scum are in the minority here!

SUDT: A toy decision theory for updateless anthropics

Benya23 Feb 2014 23:50 UTC

27 points

14 comments8 min readLW link

Benya 28 Jun 2013 16:53 UTC
26 points
on: Book: AKA Shakespeare (an extended Bayesian investigation)

Beatrice and Claudia end up agreeing that the leading candidate is de Vere, with Ignotus second and Stratford a very distant third. Beatrice’s entries lead to a final probability of 10−13 (one chance in ten million million) that Shakespeare was the gentleman from Stratford-upon-Avon. Claudia’s entries lead to an even smaller probability.

I don’t think I want to use this book to try to learn how to produce to well-calibrated probability estimates.

Self-modification is the correct justification for updateless decision theory

Benya11 Apr 2010 16:39 UTC

25 points

34 comments2 min readLW link

Benya 30 Jun 2013 9:43 UTC
24 points
in reply to: Qiaochu_Yuan’s comment on: Harry Potter and the Methods of Rationality discussion thread, part 19, chapter 88-89
But for your reasoning upthread to work, Harry has to be so sure that the outpouring of magic carried all the information about Hermione that it’s not worth it to him to try and protect her brain, and with (a) his protestations that brain damage means that the information must be in the brain and (b) there not being a shred of evidence that I can remember off the top of my head that Muggles require any magic to run (in which case witches/wizards’ brains/souls would presumably have to work completely differently from Muggles), I don’t think Harry is at a point where he can conclude that the chance of reviving her by saving the body is so low that he should concentrate his efforts on chasing her souly-looking emanation of magic.

Benya 17 Dec 2012 22:27 UTC
24 points
on: More Cryonics Probability Estimates
I think Robin’s reply to that comment (which he left there last week) got to the heart of the issue:

No doubt you can identify particular local info that is causally effective in changing local states, and that is lost or destroyed in cryonics. The key question is the redundancy of this info with other info elsewhere. If there is lots of redundancy, then we only need one place where it is held to be preserved. Your comments here have not spoken to this key issue.

It may be that what the brain uses to store some vital information is utterly destroyed by cryonics, but there is some other feature of the arrangement of atoms in the brain, possibly some side effect that has no function in the living brain, that is sufficiently correlated with the information we care about that we can reverse-engineer what we need from it. This is the “hard drive” argument for cryonics (I got it from the Sequences, but I would suspect it didn’t originate there): it’s not that hard (I think, though I do not know much about this topic) to erase data from a hard drive so that the normal functionality of the hard drive can’t bring it back, but it’s rather difficult to erase it in a way that someone sufficiently motivated with enough funding can’t get it back.

However, kalla724 did say

Distortion of the membranes and replacement of solvent irretrievably destroys information that I believe to be essential to the structure of the mind. I don’t think that would ever be readable into anything but a pale copy of the original person, no matter what kind of technological advance occurs (information simply isn’t there to be read, regardless of how advanced the reader may be).

This is a clear assertion that there aren’t even any correlates of that information preserved, if kalla724 has already thought the correlates argument through. It’s not clear to me whether or not they have.
What links here?
- [deleted]'s comment on More Cryonics Probability Estimates by jefftk (18 Dec 2012 13:53 UTC; 3 points)

Benya 28 Jun 2013 17:53 UTC
23 points
in reply to: Dentin’s comment on: Book: AKA Shakespeare (an extended Bayesian investigation)
(Okay, I’ll unpack the implication:) Assigning a probability of 10^(-3) would mean being really, really, really, really sure that the hypothesis is wrong. To be well-calibrated, you would have to be able to make ten thousand similar judgments with similar strengths of evidence and only be wrong about ten times, and if you can do that, you’re very good at this sort of thing.

Assigning 10^(-13) -- i.e., suggesting that you’re so good that you can do this and only be wrong one in ten million million times—is just obviously wrong.

So I was implying that the fact that the book suggests that this kind of number can be a plausible outcome means that it isn’t a very good place to learn the art of making Bayesian probability estimates. To learn to make well-calibrated estimates, I should try to learn from people who stand a snowball’s chance in hell of making such estimates themselves.

For an example from someone who has a claim to actually being good at this sort of thing, see Gwern’s Who wrote the Death Note script?.

Benya 24 Mar 2013 11:43 UTC
23 points
on: Reflection in Probabilistic Logic
Some technical remarks on topological aspects of the paper:

The notion of a “coherent probability distribution” and the use of the product topology on the set of all such distributions are a bit ideosyncratic. Here’s a way to relate them to more standard notions: Consider a countable first-order language L and the Stone space of complete theories over it. Equip this with the Borel-σ-algebra. (1.) A coherent probability distribution is (equivalent to) a probability distribution on the resulting measure space. (2.) The (AFAICT) most commonly used topology on the space of all such distributions is the one of weak convergence of probability measures. It turns out that this is equivalent to the product topology on the coherent probability distributions.

(1.) The main unusual feature of coherent probability distributions is that they’re only defined on the sentences of L, which don’t form a σ-algebra. Their equivalence classes under logical equivalence form a Boolean algebra, though, which is in particular a ring of sets (in the measure-theoretic sense) of complete theories (we identify each equivalence class with the set of complete theories containing the sentences in the equivalence class). Furthermore, this ring generates our σ-algebra: by the definition of “Stone space”, the sets in the ring form a base of the Stone space, and since this base is countable, every open set is a countable union of base sets (meaning that the smallest σ-algebra containing the open sets is also the smallest σ-algebra containing the base).

A coherent probability distribution, by the alternative characterization from the paper, is a finitely additive probability measure. But a finitely additive measure on a ring is already a premeasure (i.e., σ-additive on the ring) if for every descending sequence $A_nsupseteqA_{n 1}$ of elements of the ring,
%20%3E%200) implies $t e x t s t y l e b i g c a p_n A_n n e q e m p t y s e t$ , and a premeasure on a ring extends uniquely to a measure on the generated σ-algebra. Now, by the assumption, we have
%20%3E%200) and therefore $A_n n e q e m p t y s e t$ for all $n i n m a t h b b {N}$ ; since $textstyleA_n=bigcap_{i=1}nA_i$ , this means that the family
) has the finite intersection property, and so since Stone spaces are compact and each $A_n$ is clopen, the intersection of all $A_n$ is non-empty as desired.

(2.) The Stone space of a Boolean algebra is metrizable if and only if the Boolean algebra is countable, so since we’re interested in countable languages, the notion of weak convergence of probability measures on our space is well-defined. One of the equivalent definitions is that $mathbb{P}_ntomathbb{P}$ if
\ge\mathbb{P}(B)) for all open sets $B$ . We want to show that this is equivalent to convergence in the product topology on coherent probability distributions, which amounts to pointwise convergence of
) to
) for all $A$ in the base (i.e., for $A = {X, :, v a r p h i i n X}$ for some sentence $v a r p h i$ ).

Suppose first that
\to\mathbb{P}(A)) for all base sets $A$ and let $B$ be an arbitrary open set. $B$ can be written as a countable union $t e x t s t y l e b i g c u p_m A_m$ of base sets; since the base is closed under Boolean operations, it follows that it can be written as a countable disjoint union (let
). For any $e p s i l o n > 0$ , there is an $m i n m a t h b b {N}$ such that
\le\mathbb{P}(\bigcup_{i=1}%5Em%20A_i)+\epsilon/2). By pointwise convergence, for sufficiently large $n$ we have
\le\mathbb{P}_n(A_i)%20+%20\epsilon/2m) for all $i l e m$ . Therefore,

\;\le\;\sum_{i=1}%5Em\mathbb{P}(A_i)+\epsilon/2\;\le\;\sum_{i=1}%5Em\mathbb{P}_n(A_i)%20+%20\epsilon\;\le\;\mathbb{P}_n(B)+\epsilon.)

Since this holds for all $e p s i l o n$ , the desired inequality follows.

Suppose now that
\ge\mathbb{P}(B)) for all open sets $B$ . We must show that for all base sets $A$ ,
\to\mathbb{P}(A)). But base sets are clopen, so we have both
\ge\mathbb{P}(A)) and

%20\;=\;%201-\liminf\mathbb{P}_n(A%5Ec)%20\;\le\;%201%20-%20\mathbb{P}(A%5Ec)%20\;=\;%20\mathbb{P}(A),)

implying
%20=%20\mathbb{P}(A)).

How to cheat Löb’s Theorem: my second try

Benya22 Aug 2012 18:21 UTC

21 points

26 comments10 min readLW link

Benya 28 Apr 2013 5:09 UTC
20 points
on: Privileging the Question

The questions above are probably not the most important questions we could be answering right now, even in politics (I’d guess that the economy is more important). Outside of politics, many LWers probably think “what can we do about existential risks?” is one of the most important questions to answer, or possibly “how do we optimize charity?”

(This is not meant as a criticism of the post:) I hope I’m not the only one who went “gaaah” here about the latter two not being questions we could be answering right now in politics :-) Not that I have much hope that this is doable, but still.

(I started out writing “not that I have any hope”, but then remembered that GiveWell didn’t manage to find good opportunities for funding immunizations or micronutrient supplementation—the first of which they “consider to have the strongest evidence base of any intervention we know of”—with a major reason being that government and multilateral funders are already taking the best opportunities. See also Eliezer’s comment and Holden’s reply, suggesting the reason is that there are some people in government to whom measurable, quantifiable, tangible benefits make a very attractive pitch.)