Adam Shai

• Adam Shai 17 Mar 2023 16:03 UTC

  1 point

  0

  in reply to: Cleo Nardo’s comment on: The al­gorithm isn’t do­ing X, it’s just do­ing Y.

  I think the issue is that what people often mean by. “computing matrix multiplication” is something like what youve described here, but when (at least sometimes, as you’ve so elegantly talked about in other posts, vibes and context really matter!) talk about “recognizing dogs” they are referring not only to the input output transformation of the task (or even the physical transformation of world states) but also the process by which the dog is recognized, which includes lots of internal human abstractions moving about in a particular way in the brains of people, which may or may not be recapitulated in an artificial classification system.

  To some degree it’s a semantic issue. I will grant you that there is a way of talking about “recognizing dogs” that reduces it to the input/​output mapping, but there is another way in which this doesn’t work. The reason it makes sense for human beings to have these two different notions of performing a task is because we really care about theory of mind, and social settings, and figuring out what other people are thinking (and not just the state of their muscles or whatever dictates their output).

  Although for precisions sake, maybe they should really have different words associated with them, though I’m not sure what the words should be exactly. Maybe something like “solving a task” vs. “understanding a task” though I don’t really like that.

  Actually my thinking can go the other way to. I think there actually is a sense in which the computer is not doing matrix multiplication, and its really only the system of computer+human that is able to do it, and the human is doing A LOT of work here. I recognize this is not the sense people usually mean when they talk about computers doing matrix multiplication, but again, I think there are two senses of performing a computation even though people use the same words.

• Adam Shai 17 Mar 2023 15:52 UTC

  1 point

  0

  in reply to: Cleo Nardo’s comment on: The al­gorithm isn’t do­ing X, it’s just do­ing Y.

  I think I am the one that is misunderstanding. Why doesn’t your definitions work?

  For every Rilke that that can turn 0 pages into 1 page, there exists another machine B s.t.

  (1) B can turn 1 page into 1 page, while interacting with Rilke. (I can copy a poem from a rilke book while rilke writes another poem next to me, or while Rilke reads the poem to me, or while Rilke looks at the first wood of the poem and then creates the poem next to me, etc.)

  (2) the combined Rilke and B doesnt expend much more physical resource to turn 1 page into 1 page as Rilke expends writing a page of poetry.

  I have a feeling I am misentrepreting one or both of the conditions.

• Adam Shai 17 Mar 2023 14:00 UTC

  2 points

  0

  in reply to: Cleo Nardo’s comment on: The al­gorithm isn’t do­ing X, it’s just do­ing Y.

  Instead of responding philosophically I think it would be instructive to go through an example, and hear your thoughts about it. I will take your definition of physical reduction (focusing on 4.) and assign tasks and machines to the variables:

  Here’s your defintion:

  A task $X$ reduces to task $Y$ if and only if...
  
  For every machine $A$ that solves task $Y$, there exists another machine $B$ such that...
  
  (1) $B$ solves task $X$ by interacting with $A$.
  (2) The combined machine $(A\otimes B)$ doesn’t expend much more physical resources to solve $X$ as $A$ expends to solve $Y$.

  Now I want X to be the task of copying a Rilke poem onto a blank piece of paper, and Y to be the task of Rilke writing a poem onto a blank piece of paper.

  so let’s call X = COPY_POEM, Y = WRITE_POEM, and let’s call A = Rilke. So plugging into your definition:

  A task COPY_POEM reduces to task WRITE_POEM if and only if...
  
  For every Rilke that solves task WRITE_POEM, there exists another machine $B$ such that...
  
  (1) $B$ solves task COPY_POEM by interacting with Rilke.
  (2) The combined machine $(Rilke\otimes B)$ doesn’t expend much more physical resources to solve COPY_POEM as Rilke expends to solve WRITE_POEM.

  This seems to work. If I let Rilke write the poem, and I just copy his work, the the poem will be written on the piece of paper., and Rilke has done much of the physical labor. The issue is that when people say something like “writing a poem is more than just copying a poem,” that seems meaningful to me (this is why teachers are generally unhappy when you are assigned to write a poem and they find out you copied one from a book), and to dismiss the difference as not useful seems to be missing something important about what it means to write a poem. How do you feel about this example?

  Just for context, I do strongly agree with many of your other examples, I just think this doesn’t work in general. And basing all of your intuitions about intelligence on this will leave you missing something fundamental about intelligence (of the type that exists in humans, at least).

• Adam Shai 17 Mar 2023 0:18 UTC

  3 points

  −2

  on: The al­gorithm isn’t do­ing X, it’s just do­ing Y.

  Like, seriously? What do you mean when you say Google Maps “finds the shortest route from your house to the pub”? Your phone is just displaying certain pixels, it doesn’t output an actual physical road! So what do you mean? What you mean is that, by using Google Maps as an oracle with very little overhead, you can find the shortest route from your house to the pub.

  This is getting at a deep and important point, but I think this sidesteps an important difference between “writing poetry” (like when a human does it) and “computing addition” (like when a calculator does it). You get really close to it here.

  The problem is that when the task is writing poetry (as a human does it), what entity is the “you” who is making use of the physical machinations that is producing the poetry “with very little overhead”? There is something different about a writing poetry and doing addition with a calculator. The task of writing poetry (as a human) is not just about transforming inputs to outputs, it matters what the internal states are. Unlike in the case where “you” make sense of the dynamics of the calculator in order get the work of addition done, in the case of writing poetry, you are the one who is making sense of your own dynamics.

  I’m not saying there’s anything metaphysical going on, but I would argue your definition of task is not a good abstraction for humans writing poetry, it’s not even a good abstraction for humans performing mathematics (at least when they aren’t doing rote symbol manipulation using pencil and paper).

  Maybe this will jog your intuitions in my direction: one can think of the task of recognizing a dog, and think about how a human vs. a convnet does that task.

  I wrote about these issues here a little bit. But I have been meaning to write something more formalized. https://​​www.lesswrong.com/​​posts/​​f6nDFvzvFsYKHCESb/​​pondering-computation-in-the-real-world

• Adam Shai 6 Mar 2023 19:52 UTC

  3 points

  2

  on: Ad­den­dum: ba­sic facts about lan­guage mod­els dur­ing training

  I really appreciate this work and hope you and others continue to do more like it. So I really do mean this criticism with a lot of goodwill. I think even a small amount of making the figures/​animations look nicer would go a long way to making this be more digestible (and look more professional). Things like keeping constant the axes through the animations, and using matplotlib or seaborn styles. https://​​seaborn.pydata.org/​​generated/​​seaborn.set_context.html and https://​​seaborn.pydata.org/​​generated/​​seaborn.set_style.html

  apologies if you already know this and were just saving time!

• Adam Shai 3 Mar 2023 15:17 UTC

  1 point

  0

  in reply to: Alex Hollow’s comment on: Learn the math­e­mat­i­cal struc­ture, not the con­cep­tual structure

  I thought it was a great way to put it and I appreciated it a lot! I’m not even sure the post has more value than the summary; at the very least that one sentence adds a lot of explanatory power imho.

Learn the math­e­mat­i­cal struc­ture, not the con­cep­tual structure

• Adam Shai 25 Feb 2023 23:44 UTC

  2 points

  −1

  in reply to: AnthonyRepetto’s comment on: Bayes is Out-Dated, and You’re Do­ing it Wrong

  Well I don’t know SAS at all but a quick search of the SAS documentation for dirilecht calls it a “nonparametric Bayes approach”...

  https://​​documentation.sas.com/​​doc/​​en/​​casactml/​​8.3/​​casactml_nonparametricbayes_details12.htm

• Adam Shai 25 Feb 2023 23:39 UTC

  5 points

  2

  on: Bayes is Out-Dated, and You’re Do­ing it Wrong

  I don’t know if I’m missing something, but it sounds like you are discussing for a particular method of picking a prior within a Bayesian context, but you are not arguing against Bayes itself. If anything, it seems to me this is pro-Bayes, just using DIrilecht Processes as a prior.

• Adam Shai 20 Feb 2023 20:55 UTC

  1 point

  0

  on: Are there any AI safety rele­vant fully re­mote roles suit­able for some­one with 2-3 years of ma­chine learn­ing en­g­ineer­ing in­dus­try ex­pe­rience?

  You should still apply for jobs even if they say 5+ years experience. The requirements are often more flexible than they seem, and the worst that can happen is they say no.

[Question] What is a world-model?

• Adam Shai 1 Feb 2023 19:49 UTC

  7 points

  1

  on: Schizophre­nia as a defi­ciency in long-range cor­tex-to-cor­tex communication

  Some quick thoughts, can expand later with refs:

  • there are other similar results where schizophrenics do better than neurotypical. Two I remember are (1) an experiment where the experimenter pushes on the arm (or palm of hand I dont remember) of the subject with a particular force, and then the subject is asked to recreate that force by pushing on themselves. Neurotypicals push harder on themselves than when pushed on by an external source. (2) Motion tracking of a moving ball especially when there are non-predictive jumps in the balls trajectories.

  • The theories for both of these tend to be similar to what you said, an error in the signaling having to do with predictions of upcoming sensory stimulii, usually assumed to take place via long range cortex-cortex connections (feedback).

  • For the moment I can recommend a chapter in Surfing Uncertainty, which I’m pretty sure is where I got these examples. Though there are probably predictive processing reviews that cover this.

• Adam Shai 12 Dec 2022 1:21 UTC

  3 points

  0

  in reply to: tgb’s comment on: Con­sider us­ing re­versible au­tomata for al­ign­ment research

  I share your confusions/​intution about what is meant by optimization here. But I think for the purposes of this post, optimization is defined here, which is linked to at the beginning of this post. In that link, optimization is thought of as a pattern that persists in the face of perturbations and that evolves towards a small set of states. I’m still not totally grokking it though.

• Adam Shai 11 Dec 2022 17:42 UTC

  13 points

  1

  on: Con­sider us­ing re­versible au­tomata for al­ign­ment research

  This is super interesting. I was wondering if you could give a few more thoughts/​intuitions about why you think reversibility is important. I understand that it would make the simulations more physics like, but why is being physics like important to alignment research and/​or agency research?

  I clicked on the paper by the Critter creator, which seems like it might go deeper into that issue, but don’t have the time to read through it right now. Super exciting stuff! Thanks.

• Adam Shai 6 Dec 2022 1:26 UTC

  3 points

  1

  on: Who are some promi­nent rea­son­able peo­ple who are con­fi­dent that AI won’t kill ev­ery­one?

  David Deutsch https://​​www.google.com/​​url?sa=t&source=web&rct=j&url=https://​​www.daviddeutsch.org.uk/​​wp-content/​​uploads/​​2019/​​07/​​PossibleMinds_Deutsch.pdf&ved=2ahUKEwj16YjV6OP7AhXxL0QIHXU4DdkQFnoECDoQAQ&usg=AOvVaw0giHdn4BKOci3swaQ1bqlN

• Adam Shai 1 Dec 2022 20:31 UTC

  3 points

  in reply to: DirectedEvolution’s comment on: Call For Distillers

  Do you have a particular story that shows the types of negative outcomes that could happen? While it’s not impossible for me to imagine an overly sensitive academic getting angry or annoyed unreasonably, at a distillation, it hardly seems to me like it would be at all likely. I have fairly high confidence in my understanding of academic mindsets, and a single sentence at the top “this is a summary of XYZ’s work on whatever” with a link would in almost all cases be enough. You could even add in another flattering sentence, “I’m very excited about this work because… I find it super exciting so here’s my notes/​attempt at understanding it more”

  Generally, academics like it when people try to understand their work.

Men­tal Abstractions

Some thoughts about nat­u­ral com­pu­ta­tion and interactions

En­tropy Scal­ing And In­trin­sic Me­mory

• Adam Shai 29 Oct 2022 17:27 UTC

  1 point

  on: Is RL in­volved in sen­sory pro­cess­ing?

  Another paper you might be interested in that shows reinforcement learning effects even after training has reached asymptotic performance in a perceptual task: https://​​elifesciences.org/​​articles/​​49834