When you say “the dog metaphor” do you mean the original one with the biscuit, or the later one with the killing and breeding?
London L.
Karma: 44
It is! You (and others who agree with this) might be interested in this competition (https://futureoflife.org/project/worldbuilding-competition/) which aims to create more positive stories of AI, which may help shift pop culture in a positive direction.
I had a friend in a class today where you need to know the programming language C in order to do the class. But now with ChatGPT available, I told them it probably wasn’t that big of an issue, as you could probably have ChatGPT teach you C as you go through the class. I probably would have told them they should drop the class just one semester ago (before ChatGPT).
My personal analogy has been that these chat bots are like a structural speed up for humans in a similar way that Google Docs and Drive were for working on documents and files with people—it’s a free service that everyone just has access to now to talk through ideas or learn things. It’s ethical to use, and if you don’t use it, you’ll probably not be as capable as those who do.
Small typo in point (-2): “Less than fifty percent change” --> “Less than 50 percent chance”
I say this because I think it relates to / could be within your same world:
One of my favorite current story ideas is to write a story where there’s a girl who lives in a world where each society is running an experiment about the best way to live life. In school, she finds out that while most societies are aware of the experiment that’s being run, her society is one of the few where the experiment is kept secret from the people in it.
As she grows up, she realizes how poor her society is, and how little technology they have. It improves very quickly over the years, but when she was younger they didn’t even have washing machines or dishwashers.
So, she starts digging to try to figure out why it is that all their technology was so bad, and why her society started off so poor, and begins advocating for the idea that whatever experiment is being run, it should be shut down, because their quality of life is so much lesser than other societies.Eventually, the movement she starts begins gaining momentum, and the entire society begins to revolt against being in the experiment. When she’s finally able to bring her case to the main governing decision body (the council), she finds out what the experiment was:
Despite their wealth, many of the people in the other societies are depressed and feel like they have very little to live for. Is it better to live wealthy and securely without worries, or to struggle but have a purpose of overcoming those challenges?
The later chapters cover the (now older) woman fighting with her emotions. She is proud that she was able to fight for her society to raise its standards, and she feels like she’s lived a good life. However, not having the wealth at first was really tough, and she’s worried the council will make more societies like hers and subject more people to those tribulations she experienced when she was younger. But, because she and others in her society ended up greatly enjoying their lives despite (and because of, as the experiment showed) the hardships, the experiment was considered a success. If she likes her life overall, should she really oppose the creation of more societies like hers?
I have not read through this in its entirety, but it strikes me that an article I wrote about how the mathematical definition of infinity doesn’t match human intuitions might be useful for people to read who are also interested in this material. I’m also fairly new here, so if cross posting this isn’t okay, please let me know.
https://london-lowmanstone.medium.com/comparing-infinities-e4a3d66c2b07
Is it possible to hide the values of other predictors? I’m worried that seeing the values that others predict might influence future predictions in a way that’s not helpful for feedback or accurate group predictions. (Especially since names are shown and humans tend to respect others’ opinions.)
Yeah, that sounds right! I think this video supports that idea as well:
For example, when asked to think about something I would like more deeper, masterful knowledge about, I replied “artificial neural networks”.
The closest thing I could think to potentially experiencing them and interacting is either 1. Through interactive demos or 2. Through a suit like this. I’m unsure if that is what is meant by interaction, though it does seem closer.
This is my first comment on this site, so if I’m missing particular norms, please let me know.
I understand why this was done, but it is amusing to me that in order to describe what “contact with the territory” looks like, you must use map-like terms such as “arm”, “finger”, and “hand”.
I was going to recommend that you could qualify all of these terms, but I realized that this would likely not be needed for most readers; hopefully most people will understand, after having reading the previous essays, that by “hand”, you’re taking advantage of our preconceptions and built-in maps in order to describe your thought.
(I also realize that this has been done elsewhere out of necessity in the essays as well—it just particularly stood out to me in this one section.)
I was talking through this with an AGI Safety group today, and while I think the selection lens is helpful and helps illustrate your point, I don’t think the analogy quoted above is accurate in the way it should be.
The analogy you give is very similar to genetic algorithms, where models that get high reward are blended together with the other models that also get high reward and then mutated randomly. The only process that pushes performance to be better is that blending and mutating, which doesn’t require any knowledge of how to improve the models’ performance.
In other words, carrying out the breeding process requires no knowledge about what will make the dog more likely to sit. You just breed the ones that do sit. In essence, not even the breeder “gets the connection between the dogs’ genetics and sitting”.
However, at the start of your article, you describe gradient descent, which requires knowledge about how to get the models to perform better at the task. You need the gradient (the relationship between model parameters and reward at the level of tiny shifts to the model parameters) in order to perform gradient descent.
In gradient descent, just like the genetic algorithms, the model doesn’t get access to the gradient or information about the reward. But you still need to know the relationship between behavior and reward in order to do the update. In essence, the model trainer “gets the connection between the models’ parameters and performance”.
So I think a revised version of the dog analogy that takes this connection information into account might be something more like:
1. Asking a single dog to sit, and monitoring its brain activity during and afterwards.
2. Search for dogs that have brain structures that you think would lead to them sitting, based on what you observed in the first dog’s brain.
3. Ask one of those dogs to sit and monitor its brain activity.
4. Based on what you find, search for a better brain structure for sitting, etc.
At the end, you’ll have likely found a dog that sits when you ask it to.
A more violent version would be:
1. Ask a single dog to sit and monitor brain activity
2. Kill the dog and modify its brain in a way you think would make it more likely to sit. Also remove all memories of its past life. (This is equivalent to trying to find a dog with a brain that’s more conducive to sitting.)
3. Reinsert the new brain into the dog, ask it to sit again, and monitor its brain activity.
4. Repeat the process until the dog sits consistently.
To reiterate, with your breeding analogy, the person doing the breeding doesn’t need to know anything about how the dogs’ brains relate to sitting. They just breed them and hope for the best, just like in genetic algorithms. However, with this brain modification analogy, you do need to know how the dog’s brain relates to sitting. You modify the brain in a way that you think will make it better, just like in gradient descent.
I’m not 100% sure why I think that this is an important distinction, but I do, so I figured I’d share it, in hopes of making the analogy less wrong.