Re: blameless postmortems, i think the primary reason for blamelessness is because if you have blameful postmortems, they will rapidly transform (at least in perception) into punishments, and consequently will not often occur except when management is really cheesed off at someone. This was how the postmortem system ended up at Amazon while i was there.
Blameful postmortems also result in workers who are very motivated to hide issues they have caused, which is obviously unproductive.
> Give Terrence Tao 500 000$ to work on AI alignement six months a year, letting him free to research crazy Navier-Stokes/Halting problem links the rest of his time… If money really isn’t a problem, this kind of thing should be easy to do.
Literally that idea has been proposed multiple times before that I know of, and probably many more times many years ago before I was around.
What was the response? (I mean, obviously it was “not interested”, otherwise it would’ve happened by now, but why?)
This seems doubtful to me; if Yan truly believed that AI was an imminent extinction risk, or even thought it was credible, what would Yann be hoping to do or gain by ridiculing people who are similarly worried?
but it’s such a good pun!
From a rando outsider’s perspective, MIRI has not made any public indication that they are funding-constrained, particularly given that their donation page says explicitly that:
We’re not running a formal fundraiser this year but are participating in end-of-year matching events, including Giving Tuesday.
Which more or less sounds like “we don’t need any more money but if you want to give us some that’s cool”
Worth considering that humans are basically just fleshy robots, and we do our own basic maintenance and reproduction tasks just fine. If you had a sufficiently intelligent AI, it would be able to:
(1) persuade humans to make itself a general robot chassis which can do complex manipulation tasks, such as Google’s experiments with SayCan
(2) use instances of itself that control that chassis to perform its own maintenance and power generation functions
(2.1) use instances of itself to build a factory, also controlled by itself, to build further instances of the robot as necessary.
(3) kill all humans once it can do without them.
I will also point out that humans’ dependence on plants and animals has resulted in the vast majority of animals on earth being livestock, which isn’t exactly “good end”.
Interestingly i believe this is a limitation that one of the newest (as yet unreleased) diffusion models has overcome, called DeepFloyd; a number of examples have been teased already, such as the following Corgi sitting in a sushi doghouse:
https://twitter.com/EMostaque/status/1615884867304054785?t=jmvO8rvQOD1YJ56JxiWQKQ&s=19
As such the quoted paragraphs surprised me as an instance of a straightforwardly falsifiable claim in the documents.
I look forward to Thursdays specifically for these updates.
You ask elsewhere for commenters to sit down and think for 5 minutes about why an agi might fail. This seems beside the point, since averting human exctinction doesn’t require averting one possible attack from an agi. It involves averting every single one of them, because if even one succeeds everyone dies.
In this it’s similar to human security—“why might a hacker fail” is not an interesting question to system designers, because the hacker gets as many attempts as he wants. For what attempts might look like, i think other posts have provided some reasonable guesses.
I also note that there already exist (non-intelligent) distributed computer systems entirely beyond the ability of any motivated human individual, government or organization to shut down. I refer, of course, to cryptocurrencies, which have this property as an explicit goal of their design.
So. Imagine that an AGI distributes itself among human computer systems in the same way as bitcoin mining software is today. Then it starts executing on someone’s list of doomsday ideas, probably in a way secretive enough to be deniable.
Who’s gonna shut it down? And what would such an action even look like?
(A possible suggestion is “everyone realizes their best interest is in coordinating shutting down their computers so that the AGI lacks a substrate to run on”. To which i would suggest considering the last three years’ worth of response to an obvious, threatening, global enemy that’s not even sentient and will not attempt to defend itself.)
Reasonable points, all! I agree that the conflation of legality and morality has warped the discourse around this; in particular the idea of Stable Diffusion and such regurgitating copyrighted imagery strikes me as a red herring, since the ability to do this is as old as the photocopier and legally quite well-understood.
It actually does seem to me, then, that style copying is a bigger problem than straightforward regurgitation, since new images in a style are the thing that you would ordinarily need to go to an artist for; but the biggest problem of all is that fundamentally all art styles are imperfect but pretty good substitutes in the market for all other art styles.
(Most popular of all the art styles—to judge by a sampling of images online—is hyperrealism, which is obviously a style that nobody can lay either legal OR moral claim to.)
So i think that if Stability tomorrow came out with a totally unimpeachable version of SD with no copyrighted data of any kind (but with a similarly high quality of output) we would have, essentially, the same set of problems for artists.
One possible options play is puts on shutterstock, since as of about 2 weeks ago midjourney got up to a level where you can for a pittance replicate the most common and popular stock image varieties at an extremely high level of quality. (E.g. girl holding a credit card and smiling).
I think the most likely way this shakes out is adobe integrates image generation with figma and its other products, leaving “buying a stock image” as an increasingly niche and limited option for people who want an image to decorate a thing where they aren’t all that particular about what the image is.
Primary question to me is on what time scale the SSTK business model dissolves in, since these changes take time.
Yup! It’s branded as “Topamax”, but I’ve heard that some users refer to it as “Stupamax” because of the brain fog effect. It doesn’t sound awesome.
Also, it sounded like it increases probability of getting a kidney stone by a lot, though I’d need to track down the reference. All told, feels like one of the worse options out there.
As far as I understand it, “combination” drugs don’t really do anything together that each component doesn’t do alone. For example, bupropion causes weight loss if you take it alone; it just causes more when you pair it with naltrexone, which also causes weight loss.
Can you give any examples of that happening, where a drug reduces lifespan but not by causing any specific fatal effect?
Hey, I really appreciated this series, particularly in that it introduced me to the fact that leveraged etfs (1) exist and (2) can function well as a fixed proportion of overall holdings over long periods.
Is the lesswrong investing seminar still around/open to new participants, by any chance? I’ve been doing lots of research on this topic (though more for long-term than short-term strategies) and am curious about how deep the unconventional investing rabbit hole goes.
“If you think this is a simplistic or distorted version of what EY is saying, you are not paying attention. If you think that EY is merely saying that an AGI can kill a big fraction of humans in accident and so on but there will be survivors, you are not paying attention.”
Not sure why this functions as a rebuttal to anything i’m saying.
I think you’re totally right that to the extent that the stock market is a zero-sum game retail traders will lose almost every time, since the big players on the other end will always have more information and power to leverage that information than retail.
I think a lot of the relevance of this comment depends on your view of stock-market-as-casino vs stock-market-as-generator-of-wealth-at-several-steps-removed. I take the view that it’s mostly the latter; widget maker IPOs, accepts money from big institutional IPO investor and buys capital with it in exchange for proceeds, IPO investor (effectively after several intermediate trades) sells that share of proceeds-generated-from-capital to retail trader. The capital is still doing stuff for people! It’s just exactly what it’s doing is totally opaque to almost everyone.
I’m not sure whether the unspoken context of this comment is “We tried to hire Terry Tao and he declined, citing lack of interest in AI alignment” vs “we assume, based on not having been contacted by Terry Tao, that he is not interested in AI alignment.”
If the latter: the implicit assumption seems to be that if Terry Tao would find AI alignment to be an interesting project, we should strongly expect him to both know about it and have approached MIRI regarding it, neither which seems particularly likely given the low public profile of both AI alignment in general and MIRI in particular.
If the former: bummer.