So, like, I remain pretty strongly pro Hanson on this point:
I think LLaMA 7b is very cool, but it’s really stretching it to call it a state-of-the-art language model. It’s much worse than LLaMA 65b, which much worse than GPT-4, which most people think is > 100b as far as I know. I’m using a 12b model right now while working on an interpretability project… and it is just much, much dumber than these big ones.
Not being able to train isn’t a small deal, I think. Learning in a long-term way is a big part of intelligence.
Overall, and not to be too glib, I don’t see why fitting a static and subhuman mind into consumer hardware from 2023 means that Yudkowsky doesn’t lose points for saying you can fit a learning (implied) and human-level mind into consumer hardware from 2008.
I don’t see why fitting a static and subhuman mind into consumer hardware from 2023 means that Yudkowsky doesn’t lose points for saying you can fit a learning (implied) and human-level mind into consumer hardware from 2008.
Because one has nothing to do with the other. LLMs are getting bigger and bigger, but that says nothing about whether a mind designed algorithmically could fit on consumer hardware.
So, like, I remain pretty strongly pro Hanson on this point:
I think LLaMA 7b is very cool, but it’s really stretching it to call it a state-of-the-art language model. It’s much worse than LLaMA 65b, which much worse than GPT-4, which most people think is > 100b as far as I know. I’m using a 12b model right now while working on an interpretability project… and it is just much, much dumber than these big ones.
Not being able to train isn’t a small deal, I think. Learning in a long-term way is a big part of intelligence.
Overall, and not to be too glib, I don’t see why fitting a static and subhuman mind into consumer hardware from 2023 means that Yudkowsky doesn’t lose points for saying you can fit a learning (implied) and human-level mind into consumer hardware from 2008.
Because one has nothing to do with the other. LLMs are getting bigger and bigger, but that says nothing about whether a mind designed algorithmically could fit on consumer hardware.