I work on applied mathematics and AI at the Johns Hopkins University Applied Physics Laboratory. I also do AI safety related work for the Johns Hopkins Institute for Assured Autonomy. I am currently doing a CS PhD focused on AI safety at the University of Maryland, Baltimore County.
Aryeh Englander
Karma: 624
[Link] New Stanford Encyclopedia of Philosophy article, “Normative Theories of Rational Choice: Rivals to Expected Utility”
A friend pointed out on Facebook that Gato uses TPU-v3′s. Not sure why—I thought Google already had v4′s available for internal use a while ago? In any case, the TPU-v4 might potentially help a lot for the latency issue.
“More specifically, says my Inner Eliezer, it is less helpful to reason from or about one’s priors about really smart, careful-thinking people making or not making mistakes, and much more helpful to think directly about the object-level arguments, and whether they seem true.”
When you say it’s much more helpful, do you mean it’s helpful for (a) forming accurate credences about which side is in fact correct, or do you just mean it’s helpful for (b) getting a much deeper understanding of the issues? If (b) then I totally agree. If (a) though, why would I expect myself to achieve a more accurate credence about the true state of affairs than any of the people in this argument? If it’s because they’ve stated their arguments for all the world to see so now anybody can go assess those arguments—why should I think I can better assess those arguments than Eliezer and his interlocutors? They clearly still disagree with each other despite reading all the same things I’m reading (and much more, actually). And add to that the fact that Eliezer is essentially saying in these dialogues that he has private reasoning and arguments that he cannot properly express and nobody seems to understand, in which case we have no choice but to do a secondary assessment of how likely he is to have good arguments of that type, or else to form our credences while completely ignoring the possible existence of a very critical argument in one direction.
Sometimes assessments of the argument maker’s cognitive abilities and access to relevant knowledge / expertise is in fact the best way to get the most accurate credence you can, even if it’s not ideal.
(This is all just repeating standard arguments in favor of modest epistemology, but still.)
Heh, no problem. At least I changed my LessWrong username from Iarwain to my real name a while back.
Darn, there goes my ability to use Iarwain as a really unusual pseudonym. I’ve used it off and on for almost 20 years, ever since my brother made me a new email address right after having read the LOTR appendixes.
Slides: Potential Risks From Advanced AI
Thanks, looks useful!
Yes please!
Thanks!
[Question] What to include in a guest lecture on existential risks from AI?
How about, “the words “hello world!” written on a piece of paper”? Or you could substitute “on a compute screen” instead of a piece of paper, or you could just leave out the writing medium entirely. I’m curious if it can handle simple words if asked specifically for them.
Yes, I’m aware of that. But that’s a yearly list, and I’m asking for all-time favorites.
[Question] What are the top 1-10 posts / sequences / articles / etc. that you’ve found most useful for yourself for becoming “less wrong”?
I keep having kind of off-the-cuff questions I would love to ask the community, but I don’t know where the right place is to post those questions. I don’t usually have the time to go polish up the questions so that they are high quality, cite appropriate sources and previous discussions, etc., but I would still like them answered! Typically these are the types of questions I might post on Facebook, but I think I would get higher quality answers here.
Do questions of this sort belong as question posts, shortform posts, or comments on the monthly open threads? Or do they in fact belong on Facebook and not here since they are not at all polished or well researched beyond some quick Google searches? And if I ask as a short form post or as a comment on the open thread, will that get only a small fraction of the attention (and therefore the responses) as if I would have posted as a separate question post?
On presenting the case for AI risk
My general impression based on numerous interactions is that many EA orgs are specifically looking to hire and work with other EAs, many longtermist orgs are looking to specifically work with longtermists, and many AI safety orgs are specifically looking to hire people who are passionate about existential risks from AI. I get this to a certain extent, but I strongly suspect that ultimately this may be very counterproductive if we are really truly playing to win.
And it’s not just in terms of who gets hired. Maybe I’m wrong about this, but my impression is that many EA funding orgs are primarily looking to fund other EA orgs. I suspect that a new and inexperienced EA org may have an easier time getting funded to work on a given project than if a highly experienced non-EA org would apply for funding to pursue the same idea. (Again, entirely possible I’m wrong about that, and apologies to EA funding orgs if I am mis-characterizing how things work. On the other hand, if I am wrong about this then that is an indication that EA orgs might need to do a better job communicating how their funding decisions are made, because I am virtually positive that this is the impression that many other people have gotten as well.)
One reason why this selectivity kind of makes sense at least for some areas like AI safety is because of infohazard concerns, where if we get people who are not focused on the long-term to be involved then they might use our money to do capability enhancement research instead of pursuing longtermist goals. Again, I get this to a certain extent, but I think that if we are really playing to win then we can probably use our collective ingenuity to find ways around this.
Right now this focus on only looking for other EAs appears (to me, at least) to be causing an enormous bottleneck for achieving the goals we are ultimately aiming for.
Also note the Percy Liang’s Stanford Center for Research on Foundation Models seems to have a strong focus on potential risks as well as potential benefits. At least that’s what it seemed to me based on their inaugural paper and from a lot of the talks at the associated workshop last year.
[Link] Eric Schmidt’s new AI2050 Fund
I think part of what I was reacting to is a kind of half-formed argument that goes something like:
My prior credence is very low that all these really smart, carefully thought-through people are making the kinds of stupid or biased mistakes they are being accused of.
In fact, my prior for the above is sufficiently low that I suspect it’s more likely that the author is the one making the mistake(s) here, at least in the sense of straw-manning his opponents.
But if that’s the case then I shouldn’t trust the other things he says as much, because it looks like he’s making reasoning mistakes himself or else he’s biased.
Therefore I shouldn’t take his arguments so seriously.
Again, this isn’t actually an argument I would make. It’s just me trying to articulate my initial negative reactions to the post.
Quick thought: What counts as a “company” and what counts as “one year of effort”? If Alphabet’s board and directors decided for some reason to divert 99% of the company’s resources towards buying up coal companies and thereby becomes a world leader in the coal industry, does that count? What if Alphabet doesn’t buy the companies outright but instead headhunts all of their employees and buys all the necessary hardware and infrastructure?
Similarly, you specified that it needs to be a “tech company”, but what exactly differentiates a tech company from a regular company? (For this at least I’m guessing there’s likely a standard definition, I just don’t know what it is.)
It seems to me that the details here can make a huge difference for predictions at least.