She embraces the mysticism and proposes that holistic, non-reductionist model free systems are undeniably effective.
> “The biggest problem, as I see it, is that you haven’t come to a thorough understanding of what you mean”
That’s another one, that Monica writes a lot about: Understanding.
What does is mean to understanding something? And what is the meaning of meaning?
Yes, they sounds like metaphysical, mystical ideas, and they might be fundamentally unsolvable (See: Hard problem of consciousness or explanatory gap).
But we already see that it must be possible for systems to exist in this universe, that are aligned within a group. Many groups of humans use their brains to figure out how to coexist peacefully.
So unless humans posses a mystical ingredient, it must be possible to recreate this sort of understanding in machines.
But we don’t currently know how to teach this to this machines, in part because we lack a good dataset, and we don’t understand it, ourselves.
Do you think that alignment is fundamentally an engineering problem, or is it one of the humanities and one of philosophy?
It’s an engineering problem. If I’m honest, I see essentially zero room for the humanities in AI alignment. The level of fuzzy-thinking and lack of rigor that characterizes the humanities is a hindrance, where alignment is concerned.
In other words, we can discuss the philosophical implications of having machines that “understand”, after we’ve implemented the guardrails that prevent those machines from ending intelligent life.
EDIT: I read the first two articles on the blog that you linked, and I found it to be a classic example of what “Mysterious Answers To Mysterious Questions” is warning about. “Understanding” is used the same way that ancient natural philosophers used “phlogiston”, or “elan vital” or “luminiferous aether”.
Fair point! But how do you know that this ungrounded mysticism doesn’t apply to current debate about the potential capabilities of AI systems?
Why is an AI suddenly able to figure out how to break the laws of physics and be super intelligent about how to end intelligent life, but somehow incapable of comprehending the human laws of ethics and morality, and valuing life as we know it?
What makes the laws of physics easier to understand and easier to circumvent than the human laws of ethics and morality? (And also, navigating the human laws of ethics and morality must be required for ending all life. Unless software suddenly has the same energy as enriched plutonium or something like that, and one wrong bit flip causes an explosive chain reaction)
What makes it so much more difficult to understand critical thinking and “how to store human values in a computer”, and in contrast what makes “accidentally ending all intelligent life” so easy, by comparison?
It seems to me that “ASI on mission to destroy the humans” is the same thing as “luminiferous aether”.
We taught AI English and how to draw pictures and create art. Both pretty “fuzzy” things.
How hard can it be to train AI on a dataset of 90% of known human values and 90% of known problems and solutions with respect to those values for a neural net to have an “above average human”-grasp on the idea that “ending all intelligent life” computes as “that’s a problem and it’s immoral” ?
Beyond that, alignment is unsolvable anyways for AGI systems that perform at above human intelligence. Can’t predict the future with a software, because there could always be software that uses the future predicting software and negates the output—aka the Halting Problem. Can’t do anything about that.
Why is an AI suddenly able to figure out how to break the laws of physics and be super intelligent about how to end intelligent life, but somehow incapable of comprehending the human laws of ethics and morality, and valuing life as we know it?
Why do you think an AI would need to break the laws of physics in order to become superintelligent? As Eliezer and gwern have pointed out, the laws of physics are no bar to a machine achieving power beyond our capability to stop.
What makes it so much more difficult to understand critical thinking and “how to store human values in a computer”, and in contrast what makes “accidentally ending all intelligent life” so easy, by comparison?
“Accidentally ending all intelligent life” is the default outcome. It’s what happens when you program a self-optimizing maximizing process and unleash it. As Eliezer said, once, “The AI does not hate you. The AI does not fear you. The AI merely sees that you are composed of atoms that it could use for its own purposes.”
Furthermore, why do you think the comprehension is the problem? A superintelligence may fully comprehend human values, but it might be programmed in a way where it just doesn’t care. A superintelligent AI tasked with maximizing the number of paperclips in the universe will of course be capable of comprehending human morality and ethics. It might even say that it agrees. But its utility function is fixed. Its goal is to maximize paperclips. It will do whatever it can to maximize the number of paperclips and if that happens to go against what it knows of human morality, well, so much the worse for human morality, then.
How hard can it be to train AI on a dataset of 90% of known human values and 90% of known problems and solutions with respect to those values for a neural net to have an “above average human”-grasp on the idea that “ending all intelligent life” computes as “that’s a problem and it’s immoral” ?
I look forward to you producing such a database.
Beyond that, alignment is unsolvable anyways for AGI systems that perform at above human intelligence. Can’t predict the future with a software, because there could always be software that uses the future predicting software and negates the output—aka the Halting Problem. Can’t do anything about that.
That is a misunderstanding of the Halting Problem.
Interesting article! It reminds me of Monica Anderson’s blog: https://experimental-epistemology.ai/
She embraces the mysticism and proposes that holistic, non-reductionist model free systems are undeniably effective.
> “The biggest problem, as I see it, is that you haven’t come to a thorough understanding of what you mean”
That’s another one, that Monica writes a lot about: Understanding.
What does is mean to understanding something? And what is the meaning of meaning?
Yes, they sounds like metaphysical, mystical ideas, and they might be fundamentally unsolvable (See: Hard problem of consciousness or explanatory gap).
But we already see that it must be possible for systems to exist in this universe, that are aligned within a group. Many groups of humans use their brains to figure out how to coexist peacefully.
So unless humans posses a mystical ingredient, it must be possible to recreate this sort of understanding in machines.
But we don’t currently know how to teach this to this machines, in part because we lack a good dataset, and we don’t understand it, ourselves.
Do you think that alignment is fundamentally an engineering problem, or is it one of the humanities and one of philosophy?
It’s an engineering problem. If I’m honest, I see essentially zero room for the humanities in AI alignment. The level of fuzzy-thinking and lack of rigor that characterizes the humanities is a hindrance, where alignment is concerned.
In other words, we can discuss the philosophical implications of having machines that “understand”, after we’ve implemented the guardrails that prevent those machines from ending intelligent life.
EDIT: I read the first two articles on the blog that you linked, and I found it to be a classic example of what “Mysterious Answers To Mysterious Questions” is warning about. “Understanding” is used the same way that ancient natural philosophers used “phlogiston”, or “elan vital” or “luminiferous aether”.
Fair point! But how do you know that this ungrounded mysticism doesn’t apply to current debate about the potential capabilities of AI systems?
Why is an AI suddenly able to figure out how to break the laws of physics and be super intelligent about how to end intelligent life, but somehow incapable of comprehending the human laws of ethics and morality, and valuing life as we know it?
What makes the laws of physics easier to understand and easier to circumvent than the human laws of ethics and morality? (And also, navigating the human laws of ethics and morality must be required for ending all life. Unless software suddenly has the same energy as enriched plutonium or something like that, and one wrong bit flip causes an explosive chain reaction)
What makes it so much more difficult to understand critical thinking and “how to store human values in a computer”, and in contrast what makes “accidentally ending all intelligent life” so easy, by comparison?
It seems to me that “ASI on mission to destroy the humans” is the same thing as “luminiferous aether”.
We taught AI English and how to draw pictures and create art. Both pretty “fuzzy” things.
How hard can it be to train AI on a dataset of 90% of known human values and 90% of known problems and solutions with respect to those values for a neural net to have an “above average human”-grasp on the idea that “ending all intelligent life” computes as “that’s a problem and it’s immoral” ?
Beyond that, alignment is unsolvable anyways for AGI systems that perform at above human intelligence. Can’t predict the future with a software, because there could always be software that uses the future predicting software and negates the output—aka the Halting Problem. Can’t do anything about that.
Why do you think an AI would need to break the laws of physics in order to become superintelligent? As Eliezer and gwern have pointed out, the laws of physics are no bar to a machine achieving power beyond our capability to stop.
“Accidentally ending all intelligent life” is the default outcome. It’s what happens when you program a self-optimizing maximizing process and unleash it. As Eliezer said, once, “The AI does not hate you. The AI does not fear you. The AI merely sees that you are composed of atoms that it could use for its own purposes.”
Furthermore, why do you think the comprehension is the problem? A superintelligence may fully comprehend human values, but it might be programmed in a way where it just doesn’t care. A superintelligent AI tasked with maximizing the number of paperclips in the universe will of course be capable of comprehending human morality and ethics. It might even say that it agrees. But its utility function is fixed. Its goal is to maximize paperclips. It will do whatever it can to maximize the number of paperclips and if that happens to go against what it knows of human morality, well, so much the worse for human morality, then.
I look forward to you producing such a database.
That is a misunderstanding of the Halting Problem.