Logan Zoellner

Anti MMAcevedo Protocol

• Logan Zoellner 30 Mar 2024 20:21 UTC

  2 points

  0

  in reply to: ReaderM’s comment on: Modern Trans­form­ers are AGI, and Hu­man-Level

  then that’s just irrelevant. You don’t need to evaluate millions of positions to backtrack (unless you think humans don’t backtrack) or play chess.

  Humans are not transformers. The “context window” for a human is literally their entire life.

[Question] Is there a “crit­i­cal thresh­old” for LLM scal­ing laws?

• Logan Zoellner 30 Mar 2024 12:05 UTC

  2 points

  0

  in reply to: ReaderM’s comment on: Modern Trans­form­ers are AGI, and Hu­man-Level

  Setting up the architecture that would allow a pretrained LLM to trial and error whatever you want is relatively trivial.

  I agree. Or at least, I don’t see any reason why not.
  
  My point was not that “a relatively simple architecture that contains a Transformer as the core” cannot solve problems via trial and error (in fact I think it’s likely such an architecture exists). My point was that transformers alone cannot do so.
  
  You can call it a “gut claim” if that makes you feel better. But the actual reason is I did some very simple math (about the window size required and given quadratic scaling for transformers) and concluded that practically speaking it was impossible.

  Also, importantly, we don’t know what that “relatively simple” architecture looks like. If you look at the various efforts to “extend” transformers to general learning machines, there are a bunch of different approaches: alpha-geometry, diffusion transformers, baby-agi, voyager, dreamer, chain-of-thought, RAG, continuous fine-tuning, V-JEPA. Practically speaking, we have no idea which of these techniques is the “correct” one (if any of them are).

  In my opinion saying “Transformers are AGI” is a bit like saying “Deep learning is AGI”. While it is extremely possible that an architecture that heavily relies on Transformers and is AGI exists, we don’t actually know what that architecture is.

  Personally, my bet is either on a sort of generalized alpha-geometry approach (where the transformer generates hypothesis and then GOFAI is used to evaluate them) or Diffusion Transformers (where we iteratively de-noise a solution to a problem). But I wouldn’t be at all surprised if a few years from now it is universally agreed that some key insight we’re currently missing marks the dividing line between Transformers and AGI.

• Logan Zoellner 30 Mar 2024 1:10 UTC

  2 points

  0

  in reply to: ReaderM’s comment on: Modern Trans­form­ers are AGI, and Hu­man-Level

  Ok? That’s how you teach anybody anything.

  Have you never figured out something by yourself? The way I learned to do Sudoku was: I was given a book of Sudoku puzzles and told “have fun”.

  you said it would be impossible to train a chess playing model this century.

  I didn’t say it was impossible to train an LLM to play Chess. I said it was impossible for an LLM to teach itself to play a game of similar difficulty to chess if that game is not in it’s training data.

  These are two wildly different things.

  Obviously LLMs can learn things that are in their training data. That’s what they do. Obviously if you give LLMs detailed step-by-step instructions for a procedure that is small enough to fit in its attention window, LLMs can follow that procedure. Again, that is what LLMs do.

  What they do not do is teach themselves things that aren’t in their training data via trial-and-error. Which is the primary way humans learn things.

• Logan Zoellner 29 Mar 2024 20:44 UTC

  2 points

  0

  in reply to: ReaderM’s comment on: Modern Trans­form­ers are AGI, and Hu­man-Level

  sure. 4000 words (~8000 tokens) to do a 9-state 9-turn game with the entire strategy written out by a human. Now extrapolate that to chess, go, or any serious game.
  
  And this doesn’t address at all my actual point, which is that Transformers cannot teach themselves to play a game.

• Logan Zoellner 28 Mar 2024 14:22 UTC

  2 points

  −2

  AF

  in reply to: AnthonyC’s comment on: Modern Trans­form­ers are AGI, and Hu­man-Level

  Absolutely. I don’t think it’s impossible to build such a system. In fact, I think a transformer is probably about 90% there. Need to add trial and error, some kind of long-term memory/​fine-tuning and a handful of default heuristics. Scale will help too, but no amount of scale alone will get us there.

• Logan Zoellner 27 Mar 2024 20:45 UTC

  LW: 2 AF: 1

  −2

  AF

  in reply to: Matt Goldenberg’s comment on: Modern Trans­form­ers are AGI, and Hu­man-Level

  It certainly wouldn’t generalize to e.g Hidouku

• Logan Zoellner 27 Mar 2024 17:51 UTC

  LW: 4 AF: 3

  0

  AF

  in reply to: Matt Goldenberg’s comment on: Modern Trans­form­ers are AGI, and Hu­man-Level

  In the technical sense that you can implement arbitrary programs by prompting an LLM (they are turning complete), sure.

  In a practical sense, no.

  GPT-4 can’t even play tic-tac-toe. Manifold spent a year getting GPT-4 to implement (much less discover) the algorithm for Sudoku and failed.
  
  Now imagine trying to implement a serious backtracking algorithm. Stockfish checks millions of positions per turn of play. The attention window for your “backtracking transformer” is going to have to be at lease {size of chess board state}*{number of positions evaluated}.
  
  And because of quadratic attention, training it is going to take on the order of {number or parameters}*({chess board state size}*{number of positions evaluated})^2
  
  Even with very generous assumptions for {number of parameters} and {chess board state}, there’s simply no way we could train such a model this century (and that’s assuming Moore’s law somehow continues that long).

• Logan Zoellner 27 Mar 2024 13:21 UTC

  LW: -2 AF: -2

  0

  AF

  on: Modern Trans­form­ers are AGI, and Hu­man-Level

  Obvious bait is obvious bait, but here goes.

  Transformers are not AGI because they will never be able to “figure something out” the way humans can.

  If a human is given the rules for Sudoku, they first try filling in the square randomly. After a while, they notice that certain things work and certain things don’t work. They begin to define heuristics for things that work (for example, if all but one number appears in the same row or column as a box, that number goes in the box). Eventually they work out a complete algorithm for solving Sudoku.

  A transformer will never do this (pretending Sudoku wasn’t in its training data). Because they are next-token predictors, they are fundamentally incapable of reasoning about things not in their training set. They are incapable of “noticing when they made a mistake” and then backtracking they way a human would.

  Now it’s entirely possible that a very small wrapper around a Transformer could solve Sudoku. You could have the transformer suggest moves and then add a reasoning/​planning layer around it to handle the back-tracking. This is effectively what Alpha-Geometry does.

  But a Transformer BY ITSELF will never be AGI.

• Logan Zoellner 25 Mar 2024 15:28 UTC

  2 points

  −4

  in reply to: Tapatakt’s comment on: All About Con­cave and Con­vex Agents

  Lots of people think that. Occasionally because they are in a situation where the assumptions of Kelly don’t hold (for example, you are not playing a repeated game), but more often because they confuse single bets (where Kelly maximizes log dollars) with repeated play (where Kelly maximizes total dollars).
  
  In the long run, Kelly is optimal so long as your utility function is monotonic in dollars since it maximizes your total dollars.

  From wikipedia
  

  

• Logan Zoellner 25 Mar 2024 13:49 UTC

  2 points

  0

  on: All About Con­cave and Con­vex Agents

  No matter what the reward function, isn’t a rational maximizer engaging in repeated play going to “end up” concave due to e.g Kelly Criterion? I would think convex agents would be sub-optimal/​quickly self-destruct if you tried to create one and nature will basically never do so.

• Logan Zoellner 25 Mar 2024 13:40 UTC

  −3 points

  −6

  on: On the Con­fu­sion be­tween In­ner and Outer Misalignment

  The confusion arises because the terms Inner/​Outer alignment were invented by people doing philosophy and not people doing actual machine learning. In practice inner misalignment is basically not a thing. See, for example, Nora Belrose’s comments about the “counting argument”.

• Logan Zoellner 25 Mar 2024 13:34 UTC

  5 points

  2

  on: “Deep Learn­ing” Is Func­tion Approximation

  but for sufficiently large function approximators, the trend reverses

  Transformers/​deep learning work because of built-in regularization methods (like dropout layers) and not because “the trend reverses”. If you did naive “best fit polynomial” with a 7 billion parameter polynomial you would not get a good result.

• Logan Zoellner 21 Mar 2024 15:24 UTC

  8 points

  5

  on: AI #56: Black­well That Ends Well

  I do not know who is right about Mamba here and it doesn’t seem easy to check?

  Nora is definitely right in this case (Mamba isn’t doing dramatically more serial operations than Transformers). If it was, it would be slower running in not-batch-mode, whereas the whole point of Mamba is it’s faster (due to sub-quadratic attention).

Are AIs con­scious? It might depend

• Logan Zoellner 7 Mar 2024 16:13 UTC

  7 points

  0

  on: AI #54: Claud­ing Along

  and I am confident that no, Google did this on their own

  What makes you so confident? We know that the gov’t has its fingers in a lot of pies (remember twitter?) and this seems like exactly the sort of thing the Biden executive order wants to be about: telling industry how to safely build AI.

• Logan Zoellner 25 Feb 2024 0:06 UTC

  3 points

  0

  on: Sora What

  Same thing with image generation. When I want something specific, I expect to be frustrated and disappointed. When I want anything at all within a vibe zone, when variations are welcomed, often the results are great.

  I routinely start with a specific image in mind and use AI art to generate it. Mostly this means not just using text-to-image and instead using advanced techniques like controlnet, ipadapter, img2img, regional prompter, etc.

  Yes, this is a skill that has to be learned, but it is still 100x faster than I could achieve the same thing before AI art. When it comes to movies, the speedup will be even greater. The $500m budget marvel movies of today will be something that a team of 5-10 people like Corridor Crew can put together in six months on a budget of <$1m two years from now.

  There are also important technical limitations of existing (Clip-based) models that go away entirely when we switch to a transformer architecture. This image would be basically impossible to get (using only text-to-image) using existing models.

• Logan Zoellner 16 Feb 2024 13:45 UTC

  8 points

  11

  on: The Alt­man Technocracy

  “For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise.”

• Logan Zoellner 15 Feb 2024 10:07 UTC

  2 points

  −10

  in reply to: Steven Byrnes’s comment on: Sam Alt­man’s Chip Am­bi­tions Un­der­cut OpenAI’s Safety Strategy

  idk.
  
  Maybe I got carried away with the whole “everything overhang” idea.
  
  While I do think fast vs slow takeoff is an important variable that determines how safe a singularity is, it’s far from the only thing that matters.
  
  If you were looking at our world today and asking “what obvious inefficiencies will an AGI exploit?” there are probably a lot of lower-hanging fruits (nuclear power, genetic engineering, zoning) that you would point to before getting to “we’re not building chip fabs as fast as physically possible”.
  
  My actual views are probably closest to d/​acc which is that there are a wide variety of directions we can chose when researching new technology and we ought to focus on the ones that make the world safer.
  
  I do think that creating new obvious inefficiencies is a bad idea. For example, if we were to sustain a cap of 10**26 FLOPs on training runs for a decade or longer, that would make it really easy for a rouge actor/​ai to suddenly build a much more powerful AI than anyone else in the world has.
  
  As to the specific case of Sam/​$7T, I think that it’s largely aspiration and to the extent that it happens it was going to happen anyway. I guess if I was given a specific counterfactual, like: TSMC is going to build 100 new fabs in the next 10 years, is it better that they be built in the USA or Taiwan? I would prefer they be built in the USA. If, on the other hand, the counterfactual was: the USA is going to invest $7T in AI in the next 10 years, would you prefer it be all spent on semiconductor fabs or half on semiconductor fabs and half on researching controllable AI algorithms, I would prefer the latter.
  
  Basically, my views are “don’t be an idiot”, but it’s possible to be an idiot both by arbitrarily banning things and by focusing on a single line of research to the exclusion of all others.