Yeah I don’t understand where Eliezer is coming from in that tweet (and he’s written similar things elsewhere).
I don’t know as much about algorithms as you; my argument centers instead on the complexity of the world.
Like, here a question: is “tires are usually black”, and stuff like that, encoded directly in the source code?
If yes, then (1) that’s unrealistic, and (2) even if it happened, the source code would wind up horrifically complicated and inscrutable, because it’s a complicated world.
If no, then the source code must be defining a learning algorithm of some sort, which in turn will figure out for itself that tires are usually black. Might this learning algorithm be simple and legible? Yes! But that was also true for GPT-3 too, which Eliezer has always put in the inscrutable category.
So what is he taking about? He must have some alternative in mind, but I’m unsure what.
I suppose the learning process could work in a more legible way—for instance, it’s not clear why neural networks generalize successfully. But this seems to be more closely related to the theoretical understanding of learning algorithms than their knowledge representation.
Yeah I don’t understand where Eliezer is coming from in that tweet (and he’s written similar things elsewhere).
I don’t know as much about algorithms as you; my argument centers instead on the complexity of the world.
Like, here a question: is “tires are usually black”, and stuff like that, encoded directly in the source code?
If yes, then (1) that’s unrealistic, and (2) even if it happened, the source code would wind up horrifically complicated and inscrutable, because it’s a complicated world.
If no, then the source code must be defining a learning algorithm of some sort, which in turn will figure out for itself that tires are usually black. Might this learning algorithm be simple and legible? Yes! But that was also true for GPT-3 too, which Eliezer has always put in the inscrutable category.
So what is he taking about? He must have some alternative in mind, but I’m unsure what.
I suppose the learning process could work in a more legible way—for instance, it’s not clear why neural networks generalize successfully. But this seems to be more closely related to the theoretical understanding of learning algorithms than their knowledge representation.