My top interest is AI safety, followed by reinforcement learning. My professional background is in software engineering, computer science, machine learning. I have degrees in electrical engineering, liberal arts, and public policy. I currently live in the Washington, DC metro area; before that, I lived in Berkeley for about five years.
David James
Karma: 4
One failure mode could be a perception that the USG’s support of evals is “enough” for now. Under such a perception, some leaders might relax their efforts in promoting all approaches towards AI safety.
Let’s step back. This thread of the conversation is rooted in this claim: “Let’s be honest: all fiction is a form of escapism.”. Are we snared in the Disputing Definitions trap? To quote from that LW article:
if the issue arises, both sides should switch to describing the event in unambiguous lower-level constituents, like acoustic vibrations or auditory experiences. Or each side could designate a new word, like ‘alberzle’ and ‘bargulum’, to use for what they respectively used to call ‘sound’; and then both sides could use the new words consistently. That way neither side has to back down or lose face, but they can still communicate. And of course you should try to keep track, at all times, of some testable proposition that the argument is actually about.
I propose that we recognize several lower-level testable claims, framed as questions. How many people read fiction to …
entertain?
distract from an unpleasant reality?
understand the human condition (including society)?
think through alternative scenarios?
Now I will connect the conversation to these four points:
-
Luke_A_Somers wrote “Why would I ever want to escape from my wonderful life to go THERE?” which relates to #2.
-
thomblake mentions the The Philosophy of Horror. Consider this quote from the publisher’s summary: ”… horror not only arouses the senses but also raises profound questions about fear, safety, justice, and suffering. … horror’s ability to thrill has made it an integral part of modern entertainment.” which suggests #1 and #3.
-
JonInstall pulls out the dictionary in the hopes of “settling” the debate. He’s talking about #1.
-
Speaking for myself, when reading e.g. the embedded story The Tale of the Omegas in Life 3.0, my biggest takeaway was #4.
Does this sound about right?
If we know a meteor is about to hit earth, having only D days to prepare, what is rational for person P? Depending on P and D, any of the following might be rational: throw an end of the world party, prep to live underground, shoot ICBMs at the meteor, etc.
I listened to part of “Processor clock speeds are not how fast AIs think”, but I was disappointed by the lack of a human narrator. I am not interested in machine readings; I would prefer to go read the article.
For Hopfield networks in general, convergence is not guaranteed. See [1] for convergence properties.
[1] J. Bruck, “On the convergence properties of the Hopfield model,” Proc. IEEE, vol. 78, no. 10, pp. 1579–1585, Oct. 1990, doi: 10.1109/5.58341.
The audio reading of this post [1] mistakenly uses the word hexagon instead of pentagon; e.g. “Network 1 is a hexagon. Enclosed in the hexagon is a five-pointed star”.
[1] [RSS feed](https://intelligence.org/podcasts/raz); various podcast sources and audiobooks can be found [here](https://intelligence.org/rationality-ai-zombies/)
I’m not so sure.
I would expect that a qualified, well-regarded leader is necessary, but I’m not confident it is sufficient. Other factors might dominate, such as: budget, sustained attention from higher-level political leaders, quality and quantity of supporting staff, project scoping, and exogenous factors (e.g. AI progress moving in a way that shifts how NIST wants to address the issue).
What are the most reliable signals for NIST producing useful work, particularly in a relatively new field? What does history show us? What kind of patterns do we find when NIST engages with: (a) academia; (b) industry; (c) the executive branch?
perhaps I should apply Cantor’s Diagonal Argument to my clever construction, and of course it found a counterexample—the binary number (. . . 1111), which does not correspond to any finite whole number.
I’m not following despite having recently reviewed Cantor’s Diagonal Argument. I can imagine constructing a matrix such that the diagonal is all ones… but I don’t see how this connects up to the counterexample claim above.
Also, why worry that an infinite binary representation (of any kind) doesn’t correspond to a finite whole number? I suspect I’m missing something here. A little help please to help close this inferential distance?
Regarding the cost of a making an incorrect probability estimate, “Overconfidence is just as bad as underconfidence.” is not generally true. In binary classification contexts, one leads to more false positives and another to more false negatives. The costs of each are not equal in general for real world situations.
The author may simply mean that both are incorrect; this I accept.
My point is more than pedantic; there are too many examples of machine learning systems failing to recognize different misclassification costs.
How to interpret the comment above? Is it suggesting that EY’s behavior was pompous? (As of this writing, the commenter only made one comment, this one, and does not seem to be around LessWrong at this time.) My take: >60% likely. Going “one level up”, I would expect a majority of readers would at least wonder.
EY views other people’s irrationality as his problem, and it seems to me this discussion demonstrates a sincere effort to engage with someone he perceived as irrational. The conversation was respectful; as it progressed, each person clarified what they meant, and they ended with a handshake. If I were there at the outset of the conversation, I would not have expected this good of an outcome. (Updated on 2024-May-1)
Another failure mode—perhaps the elephant in the room from a governance perspective—is national interests conflicting with humanity’s interests. For example, actions done in the national interest of the US may ratchet up international competition (instead of collaboration).
Even if one puts aside short-term political disagreements, what passes for serious analysis around US national security seems rather limited in terms of (a) time horizon and (b) risk mitigation. Examples abound: e.g. support of one dictator until he becomes problematic, then switching support and/or spending massively to deal with the aftermath.
Even with sincere actors pursuing smart goals (such as long-term global stability), how can a nation with significant leadership shifts every 4 to 8 years hope to ensure a consistent long-term strategy? This question suggests that an instrumental goal for AI safety involves promoting institutions and mechanisms that promote long-term governance.