Early tractor models really were an imperfect substitute for horses
Can you say any more about this point?
Chris_Leong
Karma: 7,592
Reflections on AI Wisdom, plus announcing Wise AI Wednesdays
I suspect that the optimal amount of advocacy for an area depends more on lifecycle than being static. But it sounds like you’ll be discussing that in your next post.
I suspect that the talent available within the movement is an important constraint as well. Easier to find talented researchers than talented advocates.
You can natively edit somewhere in the middle
Can’t BERT do this as well?
For what it’s worth, I tried.
Whilst interesting, this post feels very assertive.
You claim that biological systems work by maintaining alignment as they scale. In what sense is this true?
You say that current methods lack a vision of a current whole. In what sense? There’s something extremely elegant about pre-training to learn a world model, doing supervised learning to select a sub-distribution and using RL to develop past the human level. In what sense does this “lack a vision”?
I’m open to the possibility that we need to align a model as we make it more intelligent to prevent the agent sabotaging the process. But it’s unclear from this article if this is why you want alignment first or for some other reason.
The three factors you identified: fast progress, vulnerabilities during times of crisis and AI progress increasing the chance of viable strategies being leveraged apply just as much, if not more, to coups, propaganda and AI lobbying.
Basically, I see two strategies that could make sense: either we attempt to tank these societal risks following the traditional alignment strategy or decide tanking is too risky and we mitigate the societal risks that are most likely to take us out (my previous comment identified some specific risks).
I see either of these strategies is defensible, but in neither does it make sense to prioritise the risks from the loss of economic power.
I think it depends on the audience. That level of collapsible sections is too much for a more “normy” audience, but there will be some folks who love it.
Sorry, I don’t have time to review the articles at the moment.
On collapsable sections vs footnotes:
• Collapsible sections work well for longer content as you’ve identified. A short collapsable section might seem weird.
• Collapsible sections are more visible than footnotes so people are more likely to click them. They also have a title attached so you know that there is something in the document addressing question/objection X even if you don’t read it. In contrast, footnotes are better for signalling that something isn’t necessary to read unless you really care about the details
• The reading flow is often nicer for collapsible sections
The BlueDot Future of AI course uses collapsible sections very well.
Holden Karnofsky often uses collapsable sections well (see example). He often recaps previous articles to fill in context without assuming someone has read the article (or to remind folks of the details).
I can also share an example of a summary I wrote. I don’t think I’m very good at this yet as I’m still learning how use collapsible sections, but I found this really helpful since it’s good for summaries to be short, but the collapsable sections allowed me to give readers a sense of all the main ideas included in the paper if that’s what they want.
Linking can help, but the reading flow isn’t as natural as with collapsable sections. On the other hand, I imagine many folk uncollapse sections by default, so it make sense to link instead if most readers wouldn’t want to follow that rabbit hole.
I recommend experimenting with collapsible sections.
People have a lot of different objections and these vary widely. If you try to answer all of them, then the article becomes too long and people won’t finish it. If you don’t answer someone’s pet objection, they won’t be persuaded.
Collapsable sections make this trade-off much less severe.
I’ve only skimmed this, but from what I’ve seen, you seem to be placing far too much emphasis on relatively weak/slow-acting economic effects.
If humanity loses control and it’s not due to misaligned AI, it’s much more likely to be due to an AI enabled coup, AI propaganda or AI enabled lobbying than humans having insufficient economic power. And the policy responses to these might look quite different.There’s a saying “when all you have is a hammer, everything looks like a nail” that I think applies here. I’m bearish on economics of transformative AI qua economics of transformative AI as opposed to multi-disciplinary approaches that don’t artificially inflate particular factors.
There’s also the AI Safety Awareness Project. They run public workshops.
I’ve linked to a short form describing why I work on wise AI advisors. I suspect that there a lot of work that could be done to figure out the best user interface for this:
https://www.lesswrong.com/posts/SbAofYCgKkaXReDy4/?commentId=Zcg9idTyY5rKMtYwo
If you’re interested, I could share some thoughts on specific things to experiment with.
I’m still quite confused about why you believe that a long-term pause is viable given the potential for actors to take unilateral action and the difficulties in verifying compliance.
Another possibility that could be included in that diagram would be the possibility of merging various national/coalitional AIs.
AI Safety & Entrepreneurship v1.0
You might find this document I created to be interesting: Proposed Alignment Solutions.
Might go away
There’s lots of things that “might” happen. When we’re talking about the future of humanity, we can’t afford to just glaze over mights.
The root cause may be that there is too much inferential distance
Perhaps, although I generally become more sympathetic to someone’s point of view the more I read from them.And it’s part of why I think it’s useful to create scenes that operate on different worldview assumptions: it’s worth working out the implications of specific beliefs without needing to justify those beliefs each time.
I used to lean more strongly towards more schools of thought being good, however I’ve updated slightly on the margin towards believing thinking some schools of thought just end up muddying the waters.
That said, Epoch has done some great research, so I’m overall happy the scene exists. And I think Matthew Barnett is extremely talented, I just think he’s unfortunately become confused.
I used to really like Matthew Barnett’s posts as providing contrarian but interesting takes.
However, over the last few years, I’ve started to few more negatively about them. I guess I feel that his posts tend to be framed in a really strange way such that, even though there’s often some really good research there, it’s more likely to confuse the average reader than anything else and even if you can untangle the frames, I usually don’t find worth it the time.
I should mention though that as I’ve started to feel more negative about them, I’ve started to read less of them and to engage less deeply with the ones I do look it, so there’s a chance my view would be different if I read more.
I’d probably feel more positive about any posts he writes that are closer to presenting data and further away from interpretation.
That said, Epoch overall has produced some really high-quality content and I’d definitely like to see more independent scenes.
There’s been a lot of discussion about how Less Wrong is mostly just AI these days.
If that’s something that folk want to address, I suspect that the best way to do this would be to run something like the Roots of Progress Blog-Building Intensive. My admittedly vague impression is that it seems to have been fairly successful.
Between Less Wrong for distribution, Lighthaven for a writing retreat and Less Online for networking, a lot of the key infrastructure is already there to run a really strong program if the Lightcone team ever decided to pursue this.
There was discussion about an FHI of the West before, but that seems hard given the current funding situation. I suspect that a program like this would be much more viable.