I don’t mean a formal scaling law, just an intuitive “if we look at how much difference a 10x increase has made in the past to general cognitive ability, it seems extremely unlikely that this 10x increase will lead to an agent that is capable of taking over the world”.
I don’t expect that I would make this sort of argument against deception, just against existential catastrophe.
Oh OK. I didn’t mean for this to be merely a 10x increase; I said it was the size of a human brain which I believe makes it a 1000x increase in parameter count and (if we follow the scaling laws) something like a 500x increase in training data or something? idk.
If you had been imagining that the AI I was talking about used only 10x more compute than GPT-3, then I’d be more inclined to take your side rather than MIRI’s in this hypothetical debate.
I meant that it would be a ~10x increase from what at the time was the previously largest system, not a 10x increase from GPT-3. I’m talking about the arguments I’d use given the evidence we’d have at that time, not the evidence we have now.
If you’re arguing that a tech company would do this now before making systems in between GPT-3 and a human brain, I can’t see how the path you outline is even remotely feasible—you’re positing a 500,000x increase in compute costs, which I think brings compute cost of the final training run alone to high hundreds of billions or low trillions of dollars, which is laughably far beyond OpenAI and DeepMind’s budgets, and seems out of reach even for Google or other big tech companies.
Ah. Well, it sounds like you were thinking that in the scenario I outlined, the previous largest system, 10x smaller, wasn’t making much money? I didn’t mean to indicate that; feel free to suppose that this predecessor system also clearly has massive economic implications, significantly less massive than the new one though…
I wasn’t arguing that we’d do 500,000x in one go. (Though it’s entirely possible that we’d do 100x in one go—we almost did, with GPT-3)
Am I right in thinking that your general policy is something like “Progress will be continuous; therefore we’ll get warning shots; therefore if MIRI argues that a certain alignment problem may be present in a particular AI system, but thus far there hasn’t been a warning shot for that problem, then MIRI is wrong.”
Well, it sounds like you were thinking that in the scenario I outlined, the previous largest system, 10x smaller, wasn’t making much money?
No, I wasn’t assuming that? I’m not sure why you think I was.
Tbc, given that you aren’t arguing that we’d do 500,000x in one go, the second paragraph of my previous comment is moot.
Progress will be continuous; therefore we’ll get warning shots; therefore if MIRI argues that a certain alignment problem may be present in a particular AI system, but thus far there hasn’t been a warning shot for that problem, then MIRI is wrong.
Yes, as a prior. Obviously you’d want to look at the actual arguments they give and take that into account as well.
I don’t mean a formal scaling law, just an intuitive “if we look at how much difference a 10x increase has made in the past to general cognitive ability, it seems extremely unlikely that this 10x increase will lead to an agent that is capable of taking over the world”.
I don’t expect that I would make this sort of argument against deception, just against existential catastrophe.
Oh OK. I didn’t mean for this to be merely a 10x increase; I said it was the size of a human brain which I believe makes it a 1000x increase in parameter count and (if we follow the scaling laws) something like a 500x increase in training data or something? idk.
If you had been imagining that the AI I was talking about used only 10x more compute than GPT-3, then I’d be more inclined to take your side rather than MIRI’s in this hypothetical debate.
I meant that it would be a ~10x increase from what at the time was the previously largest system, not a 10x increase from GPT-3. I’m talking about the arguments I’d use given the evidence we’d have at that time, not the evidence we have now.
If you’re arguing that a tech company would do this now before making systems in between GPT-3 and a human brain, I can’t see how the path you outline is even remotely feasible—you’re positing a 500,000x increase in compute costs, which I think brings compute cost of the final training run alone to high hundreds of billions or low trillions of dollars, which is laughably far beyond OpenAI and DeepMind’s budgets, and seems out of reach even for Google or other big tech companies.
Ah. Well, it sounds like you were thinking that in the scenario I outlined, the previous largest system, 10x smaller, wasn’t making much money? I didn’t mean to indicate that; feel free to suppose that this predecessor system also clearly has massive economic implications, significantly less massive than the new one though…
I wasn’t arguing that we’d do 500,000x in one go. (Though it’s entirely possible that we’d do 100x in one go—we almost did, with GPT-3)
Am I right in thinking that your general policy is something like “Progress will be continuous; therefore we’ll get warning shots; therefore if MIRI argues that a certain alignment problem may be present in a particular AI system, but thus far there hasn’t been a warning shot for that problem, then MIRI is wrong.”
No, I wasn’t assuming that? I’m not sure why you think I was.
Tbc, given that you aren’t arguing that we’d do 500,000x in one go, the second paragraph of my previous comment is moot.
Yes, as a prior. Obviously you’d want to look at the actual arguments they give and take that into account as well.
OK. I can explain why I thought you thought that if you like, but I suspect it’s not important to either of us.
I think I have enough understanding of your view now that I can collect my thoughts and decide what I disagree with and why.