Noah Topper

Why you should learn sign language

• Noah Topper 2 Apr 2022 23:01 UTC

  24 points

  on: MIRI an­nounces new “Death With Dig­nity” strategy

  I’m curious about what continued role you do expect yourself to have. I think you could still have a lot of value in helping train up new researchers at MIRI. I’ve read you saying you’ve developed a lot of sophisticated ideas about cognition that are hard to communicate, but I imagine could be transmitted easier within MIRI. If we need a continuing group of sane people to be on the lookout for positive miracles, would you still take a relatively active role in passing on your wisdom to new MIRI researchers? I would genuinely imagine that being in more direct mind-to-mind contact with you would be useful, so I hope you don’t become a hermit.

• Noah Topper 26 May 2021 17:41 UTC

  19 points

  on: MIRI lo­ca­tion op­ti­miza­tion (and re­lated top­ics) discussion

  I’d like to note that Texas is passing strong restrictions on abortion. They’ve passed a “heartbeat bill” banning abortions after six weeks, and it seems likely that they’ll pass a trigger bill outlawing abortion almost entirely, contingent on the Supreme Court overturning Roe v Wade.

  I’m not a Supreme Court expert, but I know people who are sincerely worried about Roe v Wade being undone. This would be a pretty big deal breaker for my fiancée (and by extension myself). From what I read, the Supreme Court will make a Roe v Wade ruling in the middle of 2022.

  Does this factor into your considerations? I feel like this would be a pretty big deal for the rationalist community at large.

What makes a prob­a­bil­ity ques­tion “well-defined”? (Part I)

• Noah Topper 2 Mar 2022 4:40 UTC

  14 points

  on: Late 2021 MIRI Con­ver­sa­tions: AMA /​ Discussion

  Eliezer, do you have any advice for someone wanting to enter this research space at (from your perspective) the eleventh hour? I’ve just finished a BS in math and am starting a PhD in CS, but I still don’t feel like I have the technical skills to grapple with these issues, and probably won’t for a few years. What are the most plausible routes for someone like me to make a difference in alignment, if any?

• Noah Topper 24 Nov 2021 19:12 UTC

  12 points

  in reply to: Richard_Ngo’s comment on: Ngo and Yud­kowsky on AI ca­pa­bil­ity gains

  I think I might actually be happy to take e.g. the Bellman equation, a fundamental equation in RL, as a basic expression of consistent utilities and thereby claim value iteration, Q-learning, and deep Q-learning all as predictions/​applications of utility theory. Certainly this seems fair if you claim applications of the central limit theorem for probability theory.

  To expand a bit, the Bellman equation only expresses a certain consistency condition among utilities. The expected utility of this state must equal its immediate utility plus the best expected utility among each possible next state I may choose. Start with some random utilities assigned to states, gradually update them to be consistent, and you get optimal behavior. Huge parts of RL are centered around this equation, including e.g. DeepMind using DQNs to crack Atari games.

  I understand Eliezer’s frustration in answering this question. The response to “What predictions/​applications does utility theory have?” in regards to intelligent behavior is, essentially, “Everything and nothing.”

• Noah Topper 30 May 2019 0:54 UTC

  9 points

  on: Micro feed­back loops and learning

  I don’t have a wonderful example of this insta-feedback (which definitely sounds ideal for learning), but I’ve gotten annoyed lately with any math book that doesn’t have exercises. Some of the books on MIRI’s Research Guide list are like this, and it really boggles my mind how anyone could learn math from a book when they don’t have anything to practice with. So I’m getting more selective.

  Even some books with exercises are super hard, and really don’t have any kind of walkthrough process. AI: A Modern Approach is a critically-acclaimed textbook, but has little to no build up in the difficulty of its exercises, and little to help you if you get lost. Right now I’m reading How to Prove It, which is *super* good. The whole book is one big walkthrough on mathematical proof, down to how to organize your scratch work. It has tons of exercises with varying difficulty, with some answers and hints. It’s much better feedback, and is helping me a lot more, although the material is comparatively simple.

What makes a prob­a­bil­ity ques­tion “well-defined”? (Part II: Ber­trand’s Para­dox)

• Noah Topper 2 Apr 2022 23:28 UTC

  5 points

  in reply to: Rob Bensinger’s comment on: MIRI an­nounces new “Death With Dig­nity” strategy

  Hm, I worry I might be a confused LWer. I definitely agree that “having a utility function” and “being a utilitarian” are not identical concepts, but they’re highly related, no? Would you agree that, to a first-approximation, being a utilitarian means having a utility function with the evolutionary godshatter as terminal values? Even this is not identical to the original philosophical meaning I suppose, but it seems highly similar, and it is what I thought people around here meant.

• Noah Topper 10 Aug 2018 21:34 UTC

  5 points

  in reply to: sone3d’s comment on: A Ra­tion­al­ist’s Guide to...

  Well, I tend to throw them onto my general to-read list, so I’m not entirely sure. A few I remember are Godel, Escher, Bach, Judgment Under Uncertainty: Heuristics and Biases, Influence: Science and Practice, The End of Time, QED: The Strange Theory of Light and Matter, The Feynman Lectures, Surely You’re Joking Mr. Feynman, Probability Theory: The Logic of Science, Probabilistic Inference in Intelligent Systems, and Player of Games. There’s a longer list here, but it’s marked as outdated.

• Noah Topper 19 Jan 2023 2:19 UTC

  4 points

  0

  in reply to: Raemon’s comment on: Why you should learn sign language

  It’s entirely just a neat thing. I think most people should consider learning to sign, and the idea of it becoming a rationalist “thing” just sounded fun to me. I did try to make that clear, but apologies if it wasn’t. And as I said, sorry this is kind of off topic, it’s just been a thing bouncing around in my head.

• Noah Topper 3 Apr 2022 4:41 UTC

  4 points

  in reply to: Said Achmiz’s comment on: MIRI an­nounces new “Death With Dig­nity” strategy

  I meant to convey a utility function with certain human values as terminal values, such as pleasure, freedom, beauty, etc.; godshatter was a stand-in.

  If the idea of a utility function has literally nothing to do with moral utilitarianism, even around here, I would question why in the above when Eliezer is discussing moral questions he references expected utility calculations? I would also point to “intuitions behind utilitarianism“ as pointing at connections between the two? Or “shut up and multiply”? Need I go on?

  I know classical utilitarianism is not exactly the same, but even in what you linked, it talks about maximizing the total sum of human happiness and sacrificing some goods for others, measured under a single metric “utility”. That sounds an awful lot like a utility function trading off human terminal values? I don’t see how what I’m pointing at isn’t just a straightforward idealization of classical utilitarianism.

• Noah Topper 3 Jun 2019 21:31 UTC

  4 points

  on: TAISU—Tech­ni­cal AI Safety Unconference

  I think you need to fix the days listed on the application form, they say August 17th − 20th.

• Noah Topper 24 Apr 2022 18:52 UTC

  3 points

  in reply to: Davidmanheim’s comment on: Hiring a math­e­mat­i­cian to work on the learn­ing-the­o­retic AI al­ign­ment agenda

  Cool, makes sense. I was planning on making various inquiries along these lines starting in a few weeks, so I may reach out to you then. Would there be a best way to do that?

• Noah Topper 3 Apr 2022 15:00 UTC

  3 points

  in reply to: Said Achmiz’s comment on: MIRI an­nounces new “Death With Dig­nity” strategy

  None of what you have linked so far has particularly conveyed any new information to me, so I think I just flatly disagree with you. As that link says, the “utility” in utilitarianism just means some metric or metrics of “good”. People disagree about what exactly should go into “good” here, but godshatter refers to all the terminal values humans have, so that seems like a perfectly fine candidate for what the “utility” in utilitarianism ought to be. The classic “higher pleasures” in utilitarianism lends credence toward this fitting into the classical framework; it is not a new idea that utilitarianism can include multiple terminal values with relative weighting.

  Under utilitarianism, we are then supposed to maximize this utility. That is, maximize the satisfaction of the various terminal goals we are taking as good, aggregated into a single metric. And separately, there happens to be this elegant idea called “utility theory”, which tells us that if you have various preferences you are trying to maximize, there is a uniquely rational way to do that, which involves giving them relative weights and aggregating into a single metric… You seriously think there’s no connection here? I honestly thought all this was obvious.

  In that last link, they say “Now, it is sometimes claimed that one may use decision-theoretic utility as one possible implementation of the utilitarian’s ‘utility’” then go on to say why this is wrong, but I don’t find it to be a knockdown argument; that is basically what I believe and I think I stand by it. Like, if you plug “aggregate human well-being along all relevant dimensions” into the utility of utility theory, I don’t see how you don’t get exactly utilitarianism out of that, or at least one version of it?

  EDIT: Please also see in the above post under “You should never try to reason using expected utilities again. It is an art not meant for you. Stick to intuitive feelings henceforth.” It seems to me that Eliezer goes on to consistently use the “expected utilities” of utility theory to be synonymous to the “utilities” of utilitarianism and the “consequences” of consequentialism. Do you agree that he’s doing this? If so, I assume you think he’s wrong for doing it? Eliezer tends to call himself a utilitarian. Do you agree that he is one, or is he something else? What would you call “using expected utility theory to make moral decisions, taking the terminal value to be human well-being”?

ACX/​LW Meetup

• Noah Topper 18 Apr 2023 15:53 UTC

  2 points

  0

  in reply to: rain8dome9’s comment on: ACX/​LW Meetup

  Thanks for coming. :)

• Noah Topper 18 Jan 2023 19:46 UTC

  2 points

  0

  in reply to: Yoav Ravid’s comment on: Why you should learn sign language

  Honestly I found ASL easier to learn than, say, the limited Spanish I tried to learn in high school. Maybe because it doesn’t conflict with the current way you communicate. Just from watching the ASL 1 − 4 lectures I linked to, I was surprisingly able to manage once dropped in a one-on-one conversation with a deaf person.

  It would definitely be good to learn with a buddy. My wife hasn’t explicitly learned it yet, but she’s picked up some from me. Israel is a tough choice, I’m not sure what the learning resources are like for it.

• Noah Topper 19 Apr 2022 14:53 UTC

  2 points

  in reply to: Davidmanheim’s comment on: Hiring a math­e­mat­i­cian to work on the learn­ing-the­o­retic AI al­ign­ment agenda

  But even in the case of still being in school, one would require the background of having proved non-trivial original theorems? All this sounds exactly like the research agenda I’m interested in. I have a BS in math and am working on an MS in computer science. I have a good math background, but not at that level yet. Should I consider applying or no?

• Noah Topper 3 Apr 2022 15:04 UTC

  2 points

  in reply to: TAG’s comment on: MIRI an­nounces new “Death With Dig­nity” strategy

  I said nothing about an arbitrary utility function (nor proof for that matter). I was saying that applying utility theory to a specific set of terminal values seems to basically get you an idealized version of utilitarianism, which is what I thought the standard moral theory was around here.