I think alignment is easier than I used to, since we can kind-of look into LLM’s and find the concepts, which might let us figure out the knobs we need to turn even though we don’t know what they are right now (i.e. weirdly enough, there might be a “lie to humans” button and we can just prevent the AI from pushing it). I still think it’s unclear if we’ll actually do the necessary research fast enough though. Alignment-by-default also seems more likely than I would have expected, although it does seem to be getting worse as we make LLM’s larger. I’m not really sure how this has changed within the community since people who don’t think AI is a problem don’t really post about it.
I think older posts about were mostly arguments about whether things could happen (could you make an oracle that’s not an agent, could you keep the AI in a box, is AI even possible, etc.) and now that the AI doomers conclusively won all of those arguments, the discussions are more concrete (discussion of actually-existing AI features).
It depends on what you mean by easier, but my timelines are shorter than they used to be, and I think most people’s are.
I’m definitely surprised that glorified decompression engines might be sufficient for AGI. The remaining problems don’t really surprise me on top of knowing how they’re trained[1]. I’m guessing the evolutionary AI people are feeling very vindicated though.
There’s lots of coding training data and not very much training data for creating documents of a specific length. I think if we added a bunch of “Write ### words about X” training data the LLM’s would suddenly be good at it.
There’s lots of coding training data and not very much training data for creating documents of a specific length. I think if we added a bunch of “Write ### words about X” training data the LLM’s would suddenly be good at it.
My point was that it’s surprising that AI is so bad at generalizing to tasks that it hasn’t been trained on. I would’ve predicted that generalization would be much better (I also added a link to a post with more examples). This is also why I think creating AGI will be very hard, unless there will be a massive paradigm shift (some new NN architecture or a new way to train NNs).
EDIT: It’s not “Gemini can’t count how many words it has in its output” that surprises me, it’s “Gemini can’t count how many words it has in its output, given that it can code in Python and in a dozen other languages and can also do calculus”.
I think alignment is easier than I used to, since we can kind-of look into LLM’s and find the concepts, which might let us figure out the knobs we need to turn even though we don’t know what they are right now (i.e. weirdly enough, there might be a “lie to humans” button and we can just prevent the AI from pushing it). I still think it’s unclear if we’ll actually do the necessary research fast enough though. Alignment-by-default also seems more likely than I would have expected, although it does seem to be getting worse as we make LLM’s larger. I’m not really sure how this has changed within the community since people who don’t think AI is a problem don’t really post about it.
I think older posts about were mostly arguments about whether things could happen (could you make an oracle that’s not an agent, could you keep the AI in a box, is AI even possible, etc.) and now that the AI doomers conclusively won all of those arguments, the discussions are more concrete (discussion of actually-existing AI features).
It depends on what you mean by easier, but my timelines are shorter than they used to be, and I think most people’s are.
I’m definitely surprised that glorified decompression engines might be sufficient for AGI. The remaining problems don’t really surprise me on top of knowing how they’re trained[1]. I’m guessing the evolutionary AI people are feeling very vindicated though.
There’s lots of coding training data and not very much training data for creating documents of a specific length. I think if we added a bunch of “Write ### words about X” training data the LLM’s would suddenly be good at it.
My point was that it’s surprising that AI is so bad at generalizing to tasks that it hasn’t been trained on. I would’ve predicted that generalization would be much better (I also added a link to a post with more examples). This is also why I think creating AGI will be very hard, unless there will be a massive paradigm shift (some new NN architecture or a new way to train NNs).
EDIT: It’s not “Gemini can’t count how many words it has in its output” that surprises me, it’s “Gemini can’t count how many words it has in its output, given that it can code in Python and in a dozen other languages and can also do calculus”.
The AIs that are good at art and suck at math were probably a surprise for everyone.