I’ve ruminated about this for several days. As an outsider to the field of artificial intelligence (coming from a IT technical space, with an emphasis on telecom and large call centers which are complex systems where interpretability has long held significant value for the business org) I have my own perspective on this particular (for the sake of brevity) “problem.”
What triggered my desire to respond
For my part, I wrote a similarly sized article not for the purposes of posting, but to organize my thoughts. And then I let that sit. (I will not be posting that 2084 word response. Consider this my imitation of Pascal: I dedicated time to making a long response shorter.) However, this is one of the excerpts that I would like to extract from that my longer response:
The arbital pages for Orthogonality and Instrumental Convergence are horrifically long.
This stood out to me, so I went to assess:
This article (at the time I counted it) ranked at 2398 words total.
Arbital Orthogonality article ranked at 2246 words total (less than this article.)
Arbital Instrumental Convergence article ranked at 3225 words total (more than this article.)
A random arxiv article I recently read for anecdotal comparison, ranked in at 9534 words (far more than this article.)
Likewise, the authors response to Eliezer’s short response stood out to me:
This raises red flags from a man who has written millions of words on the subject, and in the same breath asks why Quintin responded to a shorter-form version of his argument.
These elements provoke me to ask questions like:
Why does a request for brevity from Eliezer provoke concern?
Why does the author not apply their own evaluations on brevity to their article?
Can the authors point be made more succinctly?
These are rhetorical and are not intended to imply an answer, but it might give some sense of why I felt a need to write my own 2k words on the topic in order to organize my thoughts.
Observations
I observe that
Jargon, while potentially exclusive, can also serve as shorthand for brevity.
Presentation improvement seems to be the author’s suggestion to combat confirmation bias, belief perseverance and cognitive dissonance. I think the author is talking about boundaries. In Youtube: Machine Learning Street Talk: Robert Miles—“There is a good chance this kills everyone” offers what I think is a fantastic analogy for this problem—Someone asks an expert to provide an example of the kind of risk we’re talking about, but the risk example requires numerous assumptions be made for the example to have meaning, then, because the student does not already buy into the assumptions, they straw man the example by coming up with a “solution” to that problem and ask “Why is it harder than that?”—Robert gives a good analogy by saying this is like asking Robert what chess moves would defeat Magnus, but, in order for the answer to be meaningful, Robert requires more expertise at chess than Magnus. And when Robert comes up with a move that is not good, even a novice at chess might see a way to counter Robert’s move. These are not good engagements in the domain, because they rely upon assumptions that have not been agreed to, so there can be no short hand.
p(doom) is subjective and lacks systemization/formalization. I intuit that Availability heuristics plays a role. An analogy might be that if someone hears Eliezer express something that sounds like hyperbole, then they assess their p(doom) must be lower than his. This seems as if this is the application of confirmation bias to what appears to be a failed appeal to emotion. (i.e., you seem to have appealed to my emotion, but I didn’t feel the way you intended for me to feel, therefore I assume that I don’t believe the way you believe, therefore I believe your beliefs must be wrong.) I would caution that critics of Eliezer have a tendency to quote his more sensational statements out of context. Like quoting him about his “kinetic strikes on data centers” comment, without quoting the full context of the argument. You can find related twitter exchange and admissions that his proposal is an extraordinary one.
There may be still other attributes that I did not enumerate (I am trying to stay below 1k words.)[1]
Axis of compression potential
Which brings me to the idea that the following attributes are at the core of what the author is talking about:
Principal of Economy of Thought—The idea that truth can be expressed succinctly. This argument might also be related to Occam’s Razor. There are multiple examples of complex systems that can be described simply, but inaccurately, and accurately but not simply. Take the human organism, or the atom. And yet, there is a (I think) valid argument for rendering complex things down to simple, if inaccurate, forms so that they can be more accessible to students of the topic. Regardless of complexity required, trying to express something in the smallest form has utility. This is a principal I play with, literally daily, at work. However, when I offer an educational analogy, I often feel compelled to qualify that “All analogies have flaws.”
An improved sensitivity to boundaries in the less educated seems like a reasonable ask. While I think it is important to recognize that presentation alone may not change the mind of the student, it can still be useful to shape ones presentation to be less objectionable to the boundaries of the student. However, I think it important to remember that shaping an argument to an individuals boundaries is a more time consuming process and there is an implied impossibility of shaping every argument to the lowest common denominator. More complex arguments and conversation is required to solve the alignment problem.
Conclusion
I would like to close with, for the reasons the author uttered
I don’t see how we avoid a catastrophe here …
I concur with this, and this alone puts my personal p(doom) at over 90%.
Do I think there is a solution? Absolutely.
Do I think we’re allocating enough effort and resources to finding it? Absolutely not.
Do I think we will find the solution in time? Given the propensity towards apathy, as discussed in the bystander effect I doubt it.
Discussion (alone) is not problem solving.[2] It is communication. And while communication is necessary in parallel with solution finding, it is not a replacement therefore.
So in conclusion, I generally support finding economic approaches to communication/education that avoid barrier issues, and I generally support promoting tailored communication approaches (which imply and require a large number of non-experts working collaboratively with experts to spread the message that risks exist with AI, and there are steps we can take to avoid risks, and that it is better to take steps before we do something irrevocable.)
But I also generally think that communication alone does not solve the problem. (Hopefully it can influence an investment in other necessary effort domains.)
Discussion is a likely requirement of problem solving, but I meant “non-problem solving” discussion. I am not intentionally equivocating here. (Lots of little edits for typographical errors, and mistakes with markdown.)
Preamble
I’ve ruminated about this for several days. As an outsider to the field of artificial intelligence (coming from a IT technical space, with an emphasis on telecom and large call centers which are complex systems where interpretability has long held significant value for the business org) I have my own perspective on this particular (for the sake of brevity) “problem.”
What triggered my desire to respond
For my part, I wrote a similarly sized article not for the purposes of posting, but to organize my thoughts. And then I let that sit. (I will not be posting that 2084 word response. Consider this my imitation of Pascal: I dedicated time to making a long response shorter.) However, this is one of the excerpts that I would like to extract from that my longer response:
This stood out to me, so I went to assess:
This article (at the time I counted it) ranked at 2398 words total.
Arbital Orthogonality article ranked at 2246 words total (less than this article.)
Arbital Instrumental Convergence article ranked at 3225 words total (more than this article.)
A random arxiv article I recently read for anecdotal comparison, ranked in at 9534 words (far more than this article.)
Likewise, the authors response to Eliezer’s short response stood out to me:
These elements provoke me to ask questions like:
Why does a request for brevity from Eliezer provoke concern?
Why does the author not apply their own evaluations on brevity to their article?
Can the authors point be made more succinctly?
These are rhetorical and are not intended to imply an answer, but it might give some sense of why I felt a need to write my own 2k words on the topic in order to organize my thoughts.
Observations
I observe that
Jargon, while potentially exclusive, can also serve as shorthand for brevity.
Presentation improvement seems to be the author’s suggestion to combat confirmation bias, belief perseverance and cognitive dissonance. I think the author is talking about boundaries. In Youtube: Machine Learning Street Talk: Robert Miles—“There is a good chance this kills everyone” offers what I think is a fantastic analogy for this problem—Someone asks an expert to provide an example of the kind of risk we’re talking about, but the risk example requires numerous assumptions be made for the example to have meaning, then, because the student does not already buy into the assumptions, they straw man the example by coming up with a “solution” to that problem and ask “Why is it harder than that?”—Robert gives a good analogy by saying this is like asking Robert what chess moves would defeat Magnus, but, in order for the answer to be meaningful, Robert requires more expertise at chess than Magnus. And when Robert comes up with a move that is not good, even a novice at chess might see a way to counter Robert’s move. These are not good engagements in the domain, because they rely upon assumptions that have not been agreed to, so there can be no short hand.
p(doom) is subjective and lacks systemization/formalization. I intuit that Availability heuristics plays a role. An analogy might be that if someone hears Eliezer express something that sounds like hyperbole, then they assess their p(doom) must be lower than his. This seems as if this is the application of confirmation bias to what appears to be a failed appeal to emotion. (i.e., you seem to have appealed to my emotion, but I didn’t feel the way you intended for me to feel, therefore I assume that I don’t believe the way you believe, therefore I believe your beliefs must be wrong.) I would caution that critics of Eliezer have a tendency to quote his more sensational statements out of context. Like quoting him about his “kinetic strikes on data centers” comment, without quoting the full context of the argument. You can find related twitter exchange and admissions that his proposal is an extraordinary one.
There may be still other attributes that I did not enumerate (I am trying to stay below 1k words.)[1]
Axis of compression potential
Which brings me to the idea that the following attributes are at the core of what the author is talking about:
Principal of Economy of Thought—The idea that truth can be expressed succinctly. This argument might also be related to Occam’s Razor. There are multiple examples of complex systems that can be described simply, but inaccurately, and accurately but not simply. Take the human organism, or the atom. And yet, there is a (I think) valid argument for rendering complex things down to simple, if inaccurate, forms so that they can be more accessible to students of the topic. Regardless of complexity required, trying to express something in the smallest form has utility. This is a principal I play with, literally daily, at work. However, when I offer an educational analogy, I often feel compelled to qualify that “All analogies have flaws.”
An improved sensitivity to boundaries in the less educated seems like a reasonable ask. While I think it is important to recognize that presentation alone may not change the mind of the student, it can still be useful to shape ones presentation to be less objectionable to the boundaries of the student. However, I think it important to remember that shaping an argument to an individuals boundaries is a more time consuming process and there is an implied impossibility of shaping every argument to the lowest common denominator. More complex arguments and conversation is required to solve the alignment problem.
Conclusion
I would like to close with, for the reasons the author uttered
I concur with this, and this alone puts my personal p(doom) at over 90%.
Do I think there is a solution? Absolutely.
Do I think we’re allocating enough effort and resources to finding it? Absolutely not.
Do I think we will find the solution in time? Given the propensity towards apathy, as discussed in the bystander effect I doubt it.
Discussion (alone) is not problem solving.[2] It is communication. And while communication is necessary in parallel with solution finding, it is not a replacement therefore.
So in conclusion, I generally support finding economic approaches to communication/education that avoid barrier issues, and I generally support promoting tailored communication approaches (which imply and require a large number of non-experts working collaboratively with experts to spread the message that risks exist with AI, and there are steps we can take to avoid risks, and that it is better to take steps before we do something irrevocable.)
But I also generally think that communication alone does not solve the problem. (Hopefully it can influence an investment in other necessary effort domains.)
I failed. This ranks in at 1240 words, including markdown.
Discussion is a likely requirement of problem solving, but I meant “non-problem solving” discussion. I am not intentionally equivocating here. (Lots of little edits for typographical errors, and mistakes with markdown.)