I’m Jérémy Perret. Based in France. PhD in AI (NLP). AI Safety & EA meetup organizer. Information sponge. Mostly lurking since 2014. Seeking more experience, and eventually a position, in AI safety/governance.
Extremely annoyed by the lack of an explorable framework for AI risk/benefits. Working on that.
Ah, I remembered that there were weekly Open Threads back in the day, and Stupid Questions, and others… so I went ahead and tagged as many as I could. There are now 369 tagged posts and I’m too tired to continue digging for lonely OT posted by users that didn’t post regularly.
I have taken the survey. Cannot remember my old public key, shrug
This is one of the rare times I can in good faith use the prefix “as a parent...”, so thank you for the opportunity.
So, as a parent, lots of good ideas here. Some I couldn’t implement in time, some that are very dependent on living conditions (finding space for the trampoline is a bit difficult at the moment), some that are nice reminders (swamp water, bad indeed), some that are too early (because they can’t read yet)...
… but most importantly, some that genuinely blindsided me, because I found myself agreeing with them, and they were outside my thought process! The one-Brilliant-problem a day one, the let-them-eat-more-cookies, mainly.
I appreciate, in particular, the breadth of the ideas. Thanks for sharing, even if you don’t practice what you preach, you’ll be able to get feedback.
I had the dubious honor to tag it! Thanks, everyone, for your work!
I finished reading all the conversations a few hours ago. I have no follow-up questions (except maybe “now what?”), I’m still updating from all those words.
One except in particular, from the latest post, jumped at me (from Eliezer Yudkowsky, emphasis mine):
This is not aimed particularly at you, but I hope the reader may understand something of why Eliezer Yudkowsky goes about sounding so gloomy all the time about other people’s prospects for noticing what will kill them, by themselves, without Eliezer constantly hovering over their shoulder every minute prompting them with almost all of the answer.
The past years or reading about alignment have left me with an intense initial distrust of any alignment research agenda. Maybe it’s ordinary paranoia, maybe something more. I’ve not come up with any new ideas myself, and I’m not particularly confident in my ability to find flaws in someone else’s proposal (what if I’m not smart enough to understand them properly? What if I make things even more confused and waste everyone’s time?)
After thousands and thousands of lengthy conversations where it takes everyone ages to understand where threat models disagree, why some avenue of research is promising or not, and what is behind words (there was a whimper in my mind when the meaning/usage of corrigibility was discussed, as if this whole time experts had been talking past each other)...
… after all that, I get this strong urge to create something like Arbital to explain everything. Or maybe something simpler like Stampy. I don’t know if it would help much, the confusion is just very frustrating. When I’m facilitating discussions, trying to bring more people into the field, I insist on how not-settled many posts are, the kind of failure modes you have to watch out for.
Also this gives me an extra push to try harder, publish more things, ask more questions, because I’m getting more desperate to make progress. So, thank you for publishing this sequence.
Survey complete! I enjoyed the new questions, this should bring about some pretty graphs. Thank you for coordinating this.
Link to the podcast, with transcript.
The mention of MIRI, about (bad) AI forecasts :
Russ Roberts : [It] seems to me that there are a lot of people in AI who think [strong AI development is] only a matter of time, and that the consequences are going to be enormous. They’re not going to just be like a marginal improvement or marginal challenge. They “threaten the human race.”
Gary Marcus : Before we get to those consequences, which I actually do think are important, I’ll just say that there’s this very interesting [?] by a place called MIRI in Berkeley, MIRI (Machine Intelligence Research Institute). And what they found is that they traced people’s prediction of how far away AI is. And the first thing to know is what they found is, the central prediction, I believe it was the modal prediction, close to the median prediction, was 20 years away. But what’s really interesting is that they then went back and divided the data by year, and it turns out that people have always been saying it’s 20 years away. And they were saying it was 20 years away in 1955 and they’re saying it now. And so people always think it’s just around the corner. The joke in the field is that if you say it’s 20 years away, you can get a grant to do it. If you said it was 5 years away, you’d have to deliver it; and if 100 years, nobody’s going to talk to you.
After several nudges (which I’m grateful for, in hindsight), my votes are in.
Tagging is indeed awesome! That was a big milestone. Thanks, everyone.
I’d be interested in another Progress Bar for all posts with 30 (50?) or more comments. In the old days they may not have reached 25 karma but were very active.
Lots of things to do.
Tagging everything makes sense to me as well, and, yes, the first installments should be relevance-boosted.
I perceive the consensus to have shifted in favor of the mass-tagging, which will begin soon. I’ll report back.
Edit: all of HPMOR, 3WC, TBC have been tagged, and the tag description has been reupdated. Please boost the first chapters, and standalone pieces!
Hi Raemon! This is a topic I’m very bad ad writing structured answers about, and much better at chatting about, because there are tons of things to say and I’d need more details to know how to steer my advice.
That being said, I recommend you this repository for resources, aimed at people with a tech, but not necessarily math, background. Reading though some of the guides there should help you solve some of your last-section questions.
I’d say that staying up to date on AI developments with a goal of AI safety is entirely tractable as long as you’re not looking for the particular techniques that will lead to unsafe AI. Most AI literature is entirely disconnected from AI safety concerns, and if you dive into the field enough, you will become proficient enough to understand the papers that are relevant to safety concerns.
Cute little ML projects almost always have hidden depths, if you’re dealing with real-world data. I suggest to try them after tutorials, not as tutorials, so that you’ll be able to split whatever you’re trying to do in manageable chunks (and understand why things fail or succeed).
I wish you the best for your endeavor!
Congratulations on your launch!
As Michaël Trazzi in the other post, I’m interested in the kind of products you’ll develop, but more specifically in how the for-profit part interacts with both the conceptual research part and the incubator part. Are you expecting the latter two to yield new products as they make progress? Do these activities have different enough near-term goals that they mostly just coexist within Conjecture?
(also, looking forward to the pluralism sequence, this sounds great)
We discussed the topic of feedback with Adam. I approve of this challenge and will attempt to comment on at least half of all new posts from today to the end of November. Eventually renewing it if it works well.
I’ve been meaning to get out of mostly-lurking mode for months now, and this is as good of an opportunity as it gets.
I also want to mention the effect of “this comment could be a post”, which can help people “upgrade” from commenting, to shortform or longform, if they feel (like me), that there’s some quality bar to clear to feel comfortable posting more and get your ideas out there (hello, self-confidence issues).
You won’t get feedback if you don’t post somewhere anyways, and that could start with comments!
Thanks for the project, and the FAQ. I shall contribute.
Is there a way to retrieve the old tags from LW 1.0? I remember they were used to index Open Threads, for instance. I can’t remember the details but that could be a good way to jumpstart some tags.
I have taken the survey. Yesterday.