I asked about the memory and generative models because I feel uncertain about the differences, if any, between storing information in memory versus storing in generative models. Example of storing in memory would be something like a knowledge graph. Example of retrieving info from a generative model would be something like inputing a vector into deconvolutional NN so that it outputs an image (models have capacity making them function like memory). One question on my mind is, are there things that are better suited (or “more naturally”) stored in a generative model versus in memory.
Thanks for explaining where you’re coming from. Hmm. For the brain, I disagree that there’s a distinction—the generative models, semantic memory, and a knowledge graph are three different descriptions of the same thing. Like, say you know that “if you push the button on the toy, it goes Beep”. You could call that part of a knowledge graph—some relationship between a certain toy, its affordance of pressing the button, and the beep sound—but you could also call it a generative model—a little kinda movie in your head, in which you picture the button being pressed, and then you hear the sound Beep. Right?
For images, it gets a bit trickier to visualize what’s going on, but I think the Dileep George vision model is probably a better starting point than a deconvolutional NN if you’re thinking about the brain. You don’t normally think of your visual knowledge as organized into a “knowledge graph”, but I do think that there is in fact a giant repository of known, um, snippets of aspects of images, with known relations between them—like how known contours fit together into known shapes, and how known subcomponents fit together into known assemblies, etc.—and in this sense your visual memory can in fact be treated as a knowledge graph, and formalized as some elaborate variant of a probabilistic graphical model. By contrast, I don’t think I would describe a deconvolutional NN as implicitly encoding a “knowledge graph”. I mean, it’s a different type of generative model, it’s not structured that way, it doesn’t seem very knowledge-graph-ish to me...
Yes, I also think that memory and generative models could be “different forms” of the same thing. A generative model seems like compressed memory. Perhaps memory to a biological organism could be like short-term memory (representations being focused on (attention) and recent history. Contents readily retrievable). And generative models to a biological organism could be like long-term memory (effort needed in retrieving compressed contents). However, a machine with large memory capacities might have less need of generative models solely for the sake of memory compression. Engineers might still elect generative models to be implemented for the sake of creating new memories (e.g. GANs to merge concepts together to form a new memory). This might help in creativity — obtaining a new memory from mashing together two or more previously acquired memories.
Yes, I too, don’t think it’s natural to describe a deconv NN as implicitly encoding a knowledge graph. Mentioned knowledge graph as an example.
Having brought up Dileep George’s and Randall O Reilly’s work in your posts, would you happen to have had the time to try out the code (Reilly’s work is on github) and have comments/feedback?
At work I encounter deconv NNs more frequently even for topics like predictive coding, curiosity, etc.. Would you happen to have encountered alternative models (other than Dileep George’s) that are amenable to someone wanting to take baby steps into the AGI community?
For the interested reader, I came across a video of Dileep George talking about RCN:
Engineers might still elect generative models to be implemented for the sake of creating new memories (e.g. GANs to merge concepts together to form a new memory). This might help in creativity — obtaining a new memory from mashing together two or more previously acquired memories.
Oh yeah, definitely, and also planning, reasoning, and so on. It’s all about mashing together compositional generative models. Like: “I need to put this book into my bag. Will it fit?” Well, you have a generative model of all the ways that the book can be oriented, and you have a generative model of all the ways that the bag can be reshaped and that its current contents can be shuffled around, and you try to mix and match all those models until you fit them together into a plausible composite model wherein the book slides easily into the bag. Then you reshape the bag, shuffle the contents, and orient the book, and it slides in, just like you imagined!
would you happen to have had the time to try out the code (Reilly’s work is on github) and have comments/feedback?
No, I haven’t had time. Also, I think that safely using AGI systems remains an unsolved problem. If we had a complete neocortex simulator right now, I think we would be able to quickly (years not decades) turn it into an extremely powerful system with superhuman cross-domain reasoning and common sense and a drive to accomplish goals. But we would have only sketchy and unreliable methods to “steer it” towards trying to do what we want it to try to do, or to even know what it’s trying to do at any given time. So, such a system would be a very dangerous thing, and it would get rapidly and unpredictably more dangerous as we optimized the hyperparameters and scaled the algorithms etc. And I don’t think that having these systems right in front of us would make it much easier to figure out how to reliably control them. (It would make it easier to find unreliable control methods, which work for a while then suddenly fail.)
So that’s why I’m not inclined to be part of the project to reverse-engineer the neocortex—not until we have a better plan for “what if we succeed”. I feel like I understand the neocortex about as well as I need to in order to think about how and whether a future neocortex-like AGI can be controlled, or more generally how to make sure it’s really a step towards the awesome post-AGI utopia we’re all hoping for. :-) Well, I mean, I still read papers speculating about how the neocortex works, for example because I want to continue to build confidence that I’m not wildly misunderstanding it or missing anything important. But that’s not my main interest right now.
Instead, I’ve been been spending my limited free time trying to get a better understanding and intuitions for what the reward signals are, how they’re calculated, how they work in humans and might work in AGIs, what the amygdala and tectum and cerebellum and whatnot are calculating and how, whether neocortex-like AGIs would be conscious or suffering, how they would be similar and different from humans and what would be the political and societal consequences of that …. Lots to do, I’ve got my work cut out! See also my most recent posts here. :-)
For the interested reader, I came across a video of Dileep George talking about RCN:
Yes I liked that talk too. Dileep also has done podcasts more recently with Lex Fridman and TWIMLAI.
It’s all about mashing together compositional generative models. Like: “I need to put this book into my bag. Will it fit?” Well, you have a generative model of all the ways that the book can be oriented, and you have a generative model of all the ways that the bag can be reshaped and that its current contents can be shuffled around, and you try to mix and match all those models until you fit them together into a plausible composite model wherein the book slides easily into the bag. Then you reshape the bag, shuffle the contents, and orient the book, and it slides in, just like you imagined!
This reminds me of works like Capsule Network and Reinforcement Learning works that use imagination (e.g. learning how to drive a car in a game by imagining how upcoming roads curve, learning to dodge fireballs from enemies by imagining enemies shooting fireballs).
So that’s why I’m not inclined to be part of the project to reverse-engineer the neocortex—not until we have a better plan for “what if we succeed
Regarding the threat of AGI — one perspective is that people accidentally stumble upon AGI architecture (perhaps a simple one, but nonetheless one), don’t recognize it because it’s capabilities are evaluated on narrow tasks (making it seem similar to traditional, narrow AI), is popularized (e.g. blogposts, academic papers) and distributed (e.g. github), and eager, well-meaning folks try it and its improved variants in increasingly realistic environments (access to websites, social media, embodied in a robot waiter), and suddenly realize… hey… this thing is learning things we did not quite expect. I mean, we did expect it to learn, especially to mimic material it’s exposed to, but not these action sequences that seem superfluous at first but yield surprisingly meaningful outcomes. Real-world example: see footnote. Generally speaking, reward-driven agents that have to figure out what actions to take have this potential.
One way to prevent the above scenario from accidentally happening is to map models, prioritizing the popular and proven models, to known cognitive functions. Such knowledge lets us estimate the scope of cognitive functions mirrored by a given model pipeline. An informative side-effect of this is that we might come to realize that not all features of our own (human) cognition are necessary for AGI — as an example, the absence of the pre-, sub-, & fully- conscious distinction has [EDIT: might have] trivial effects on AGI.
It’d be helpful in the near future for there to be voices that can warn when models come dangerously close to forming a set of cognitive functions minimally required for a basic AGI. Admittedly, historically, AGI predictions aren’t exactly known for being prescient, but communities at least get an informed warning.
Footnote:
Go experts were impressed by the program’s performance and its nonhuman play style; Ke Jie stated that “After humanity spent thousands of years improving our tactics, computers tell us that humans are completely wrong… I would go as far as to say not a single human has touched the edge of the truth of Go.”
Its strategy of maximising its probability of winning is distinct from what human players tend to do which is to maximise territorial gains, and explains some of its odd-looking moves. It makes a lot of opening moves that have never or seldom been made by humans, while avoiding many second-line opening moves that human players like to make. It likes to use shoulder hits, especially if the opponent is over concentrated.
My comment: A concrete example is “move 37” by alphago, a move typically eschewed by human players due to intuition passed on through the ages.
The South Korean Go champion Lee Se-dol has retired from professional play… “With the debut of AI in Go games, I’ve realized that I’m not at the top even if I become the number one through frantic efforts,” Lee told Yonhap. “Even if I become the number one, there is an entity that cannot be defeated.” Lee lost 4-1 to DeepMind’s AlphaGo in 2016.
For years, Go was considered beyond the reach of even the most sophisticated computer programs. The ancient board game is famously complex, with more possible configurations for pieces than atoms in the observable universe. … “Even with a two-stone advantage, I feel like I will lose the first game to HanDol [a Korean AI program],” Lee told Yonhap. “These days, I don’t follow Go news …”
not all features of our own (human) cognition are necessary for AGI — as an example, the absence of the pre-, sub-, & fully- conscious distinction has trivial effects on AGI.
Well, I think it’s premature to say what is or isn’t important for an AGI until we build one.
This reminds me of works like Capsule Network
Yeah I agree that capsule networks capture part of that, even if it’s not the whole story.
Regarding the threat of AGI — one perspective is that people accidentally stumble upon AGI architecture (perhaps a simple one, but nonetheless one), don’t recognize it …
Sure, I was talking about digging into the gory details of the neocortical algorithm. What do the layer 2 neurons calculate and how? That kind of thing. Plenty of people are doing that all the time, of course, and making rapid progress in my opinion. I find that fact a bit nerve-wracking, but oh well, what can you do? Hope for the best, I suppose, and meanwhile work as fast as possible on the “what if we succeed” question. I mean, I do actually have idiosyncratic opinions about what layer 2 neurons are calculating and how, or whatever, but wouldn’t think to blog about them, on the off chance that I’m actually right. :-P
Bigger-picture thinking, like you’re talking about, is more likely to be a good thing, I figure, although the details matter. Like, creating common knowledge that a certain path will imminently lead to AGI could lead to a frantic race between teams around the world where safety gets thrown out the window. But some big-picture knowledge is necessary for “what if we succeed”. Of course I blog on big-picture stuff myself. I think I’m pushing things in the right direction, but who knows :-/
I’m going slightly off-topic but couldn’t help but notice that your website says that you’re doing this in your spare time. I’m surprised that you’ve covered so much ground. If you don’t mind me the question—how do you keep abreast of the AI field with so many papers published every year? Like do you attend periodic meet-ups in your circle of friends/colleagues to discuss such matters? do you opt to read summaries of papers instead of the long paper?
Oh, there are infinity papers on AI per month. I couldn’t dream of reading them all. Does anyone?
I think I’m at least vaguely aware of what the ML people are all talking about and excited about, through twitter.
Mainstream ML papers tend to be pretty low on my priority list compared to computational neuroscience, and neuroscience in general, and of course AGI safety and strategy.
Learning is easier when you have specific questions you’re desperately trying to answer :-)
Beyond that, I dunno. I don’t watch TV, and I quit my other hobbies to clear more time for this one. It is a bit exhausting sometimes. Maybe I should take a break. Oh but I do really want to write up that next blog post! And the one after that… :-)
Thanks for writing back.
I asked about the memory and generative models because I feel uncertain about the differences, if any, between storing information in memory versus storing in generative models. Example of storing in memory would be something like a knowledge graph. Example of retrieving info from a generative model would be something like inputing a vector into deconvolutional NN so that it outputs an image (models have capacity making them function like memory). One question on my mind is, are there things that are better suited (or “more naturally”) stored in a generative model versus in memory.
Thanks for explaining where you’re coming from. Hmm. For the brain, I disagree that there’s a distinction—the generative models, semantic memory, and a knowledge graph are three different descriptions of the same thing. Like, say you know that “if you push the button on the toy, it goes Beep”. You could call that part of a knowledge graph—some relationship between a certain toy, its affordance of pressing the button, and the beep sound—but you could also call it a generative model—a little kinda movie in your head, in which you picture the button being pressed, and then you hear the sound Beep. Right?
For images, it gets a bit trickier to visualize what’s going on, but I think the Dileep George vision model is probably a better starting point than a deconvolutional NN if you’re thinking about the brain. You don’t normally think of your visual knowledge as organized into a “knowledge graph”, but I do think that there is in fact a giant repository of known, um, snippets of aspects of images, with known relations between them—like how known contours fit together into known shapes, and how known subcomponents fit together into known assemblies, etc.—and in this sense your visual memory can in fact be treated as a knowledge graph, and formalized as some elaborate variant of a probabilistic graphical model. By contrast, I don’t think I would describe a deconvolutional NN as implicitly encoding a “knowledge graph”. I mean, it’s a different type of generative model, it’s not structured that way, it doesn’t seem very knowledge-graph-ish to me...
Yes, I also think that memory and generative models could be “different forms” of the same thing. A generative model seems like compressed memory. Perhaps memory to a biological organism could be like short-term memory (representations being focused on (attention) and recent history. Contents readily retrievable). And generative models to a biological organism could be like long-term memory (effort needed in retrieving compressed contents). However, a machine with large memory capacities might have less need of generative models solely for the sake of memory compression. Engineers might still elect generative models to be implemented for the sake of creating new memories (e.g. GANs to merge concepts together to form a new memory). This might help in creativity — obtaining a new memory from mashing together two or more previously acquired memories.
Yes, I too, don’t think it’s natural to describe a deconv NN as implicitly encoding a knowledge graph. Mentioned knowledge graph as an example.
Having brought up Dileep George’s and Randall O Reilly’s work in your posts, would you happen to have had the time to try out the code (Reilly’s work is on github) and have comments/feedback?
At work I encounter deconv NNs more frequently even for topics like predictive coding, curiosity, etc.. Would you happen to have encountered alternative models (other than Dileep George’s) that are amenable to someone wanting to take baby steps into the AGI community?
For the interested reader, I came across a video of Dileep George talking about RCN:
Oh yeah, definitely, and also planning, reasoning, and so on. It’s all about mashing together compositional generative models. Like: “I need to put this book into my bag. Will it fit?” Well, you have a generative model of all the ways that the book can be oriented, and you have a generative model of all the ways that the bag can be reshaped and that its current contents can be shuffled around, and you try to mix and match all those models until you fit them together into a plausible composite model wherein the book slides easily into the bag. Then you reshape the bag, shuffle the contents, and orient the book, and it slides in, just like you imagined!
No, I haven’t had time. Also, I think that safely using AGI systems remains an unsolved problem. If we had a complete neocortex simulator right now, I think we would be able to quickly (years not decades) turn it into an extremely powerful system with superhuman cross-domain reasoning and common sense and a drive to accomplish goals. But we would have only sketchy and unreliable methods to “steer it” towards trying to do what we want it to try to do, or to even know what it’s trying to do at any given time. So, such a system would be a very dangerous thing, and it would get rapidly and unpredictably more dangerous as we optimized the hyperparameters and scaled the algorithms etc. And I don’t think that having these systems right in front of us would make it much easier to figure out how to reliably control them. (It would make it easier to find unreliable control methods, which work for a while then suddenly fail.)
So that’s why I’m not inclined to be part of the project to reverse-engineer the neocortex—not until we have a better plan for “what if we succeed”. I feel like I understand the neocortex about as well as I need to in order to think about how and whether a future neocortex-like AGI can be controlled, or more generally how to make sure it’s really a step towards the awesome post-AGI utopia we’re all hoping for. :-) Well, I mean, I still read papers speculating about how the neocortex works, for example because I want to continue to build confidence that I’m not wildly misunderstanding it or missing anything important. But that’s not my main interest right now.
Instead, I’ve been been spending my limited free time trying to get a better understanding and intuitions for what the reward signals are, how they’re calculated, how they work in humans and might work in AGIs, what the amygdala and tectum and cerebellum and whatnot are calculating and how, whether neocortex-like AGIs would be conscious or suffering, how they would be similar and different from humans and what would be the political and societal consequences of that …. Lots to do, I’ve got my work cut out! See also my most recent posts here. :-)
Yes I liked that talk too. Dileep also has done podcasts more recently with Lex Fridman and TWIMLAI.
This reminds me of works like Capsule Network and Reinforcement Learning works that use imagination (e.g. learning how to drive a car in a game by imagining how upcoming roads curve, learning to dodge fireballs from enemies by imagining enemies shooting fireballs).
Regarding the threat of AGI — one perspective is that people accidentally stumble upon AGI architecture (perhaps a simple one, but nonetheless one), don’t recognize it because it’s capabilities are evaluated on narrow tasks (making it seem similar to traditional, narrow AI), is popularized (e.g. blogposts, academic papers) and distributed (e.g. github), and eager, well-meaning folks try it and its improved variants in increasingly realistic environments (access to websites, social media, embodied in a robot waiter), and suddenly realize… hey… this thing is learning things we did not quite expect. I mean, we did expect it to learn, especially to mimic material it’s exposed to, but not these action sequences that seem superfluous at first but yield surprisingly meaningful outcomes. Real-world example: see footnote. Generally speaking, reward-driven agents that have to figure out what actions to take have this potential.
One way to prevent the above scenario from accidentally happening is to map models, prioritizing the popular and proven models, to known cognitive functions. Such knowledge lets us estimate the scope of cognitive functions mirrored by a given model pipeline. An informative side-effect of this is that we might come to realize that not all features of our own (human) cognition are necessary for AGI — as an example, the absence of the pre-, sub-, & fully- conscious distinction has [EDIT: might have] trivial effects on AGI.
It’d be helpful in the near future for there to be voices that can warn when models come dangerously close to forming a set of cognitive functions minimally required for a basic AGI. Admittedly, historically, AGI predictions aren’t exactly known for being prescient, but communities at least get an informed warning.
Footnote:
- source https://en.wikipedia.org/wiki/AlphaGo
My comment: A concrete example is “move 37” by alphago, a move typically eschewed by human players due to intuition passed on through the ages.
- source https://www.theverge.com/2019/11/27/20985260/ai-go-alphago-lee-se-dol-retired-deepmind-defeat
Well, I think it’s premature to say what is or isn’t important for an AGI until we build one.
Yeah I agree that capsule networks capture part of that, even if it’s not the whole story.
Sure, I was talking about digging into the gory details of the neocortical algorithm. What do the layer 2 neurons calculate and how? That kind of thing. Plenty of people are doing that all the time, of course, and making rapid progress in my opinion. I find that fact a bit nerve-wracking, but oh well, what can you do? Hope for the best, I suppose, and meanwhile work as fast as possible on the “what if we succeed” question. I mean, I do actually have idiosyncratic opinions about what layer 2 neurons are calculating and how, or whatever, but wouldn’t think to blog about them, on the off chance that I’m actually right. :-P
Bigger-picture thinking, like you’re talking about, is more likely to be a good thing, I figure, although the details matter. Like, creating common knowledge that a certain path will imminently lead to AGI could lead to a frantic race between teams around the world where safety gets thrown out the window. But some big-picture knowledge is necessary for “what if we succeed”. Of course I blog on big-picture stuff myself. I think I’m pushing things in the right direction, but who knows :-/
I’m going slightly off-topic but couldn’t help but notice that your website says that you’re doing this in your spare time. I’m surprised that you’ve covered so much ground. If you don’t mind me the question—how do you keep abreast of the AI field with so many papers published every year? Like do you attend periodic meet-ups in your circle of friends/colleagues to discuss such matters? do you opt to read summaries of papers instead of the long paper?
Oh, there are infinity papers on AI per month. I couldn’t dream of reading them all. Does anyone?
I think I’m at least vaguely aware of what the ML people are all talking about and excited about, through twitter.
Mainstream ML papers tend to be pretty low on my priority list compared to computational neuroscience, and neuroscience in general, and of course AGI safety and strategy.
Learning is easier when you have specific questions you’re desperately trying to answer :-)
Beyond that, I dunno. I don’t watch TV, and I quit my other hobbies to clear more time for this one. It is a bit exhausting sometimes. Maybe I should take a break. Oh but I do really want to write up that next blog post! And the one after that… :-)