If this timeline is correct, what do you think people should do?
Chris_Leong
Karma: 8,278
I resonate with this.
Alignment felt hard, so I concentrated on community building, but one thing I’ve realised over time is that doing community building well doesn’t remove the need to place bets on what solving the problem is likely to look like.
If it were possible then sure.
At the moment I’m worried most wisdom capabilities might be acceleratory, but maybe RSI is happening anyway and so that doesn’t matter?
I’m very skeptical of overhang arguments. Instead of burning through the overhang, we seem to just build a new base upon which further advances are then developed.
I’m tempted to enter, but I’m also worried about capabilities externalities. I would love to hear people’s thoughts.
Sorry if this is overly harsh, but: stop relying on Holly as a crutch!
(It’s hard to criticise others for not making the hard decision with any authority of you’re not making the hard decision yourself)
The perfect is the enemy of the good. I suspect making this idea with requires compromise.
The problem with impact markets is that:
a) Most of the work funded won’t be counterfactual.
b) There isn’t much of an incentive if people aren’t certain that good work actually will be funded.Three solutions:
a) Reducing the scope of the impact market (ie. only grads of a particular fellowship)
b) Maybe a funder could just offer additional funding to top performing grantees to use however they wish.
c) For projects with high EV, allow generous retrospective funding (ie. funding for large amounts of work completed before the application)?
Have you considered any of these options?
Right now, the strongest association that comes to mind when thinking of an AI capabilities builder or researcher is something like “the elite of tech and academia”. Not like “reckless”, “unhinged”, “creepy”, “twisted”, “trading respectability for money”, or anything in the zone that I personally think is where we’d land in a rationally calibrated society. If the premise of high imminent AI x-risk is knowably true, shouldn’t it propagate into a true moral judgment about the people who advance AI capabilities?
I think there is some validity to this perspective, but insofar as this is the case:
Holly is the wrong messenger. She actual makes any future discussion harder by pursuing this so recklessly and without being properly socially calibrated. She should just let people like yourself develop this strand of thought further.
Pacing the Frontier suggests that there is significant alpha at the moment in leaning into co-operation and collaboration (much moreso than before where something like this probably wouldn’t have happened).
You might want to check out Raymond Douglas’ post: ‘AI for societal uplift’ as a path to victory.
I also wrote a post Some Preliminary Notes on the Promise of a Wisdom Explosion (half of my prize-winning submission to the AI Impacts Essay Competition on Automation of Wisdom and Philosophy).
Two potential critiques that deeply worry me:
“Great plan… for Nov 2022”
Perhaps this kind of work would also just be a key bottleneck for capabilities as well.
So I did say that I was planning to engage less here, but now I’m thinking it might still be worthwhile posting some things to short-form as the time investment is much less than writing up a proper post.
So here’s some thoughts on speed-running decision theory:
Decision theory is complex because you have to travel quite far down the stack of assumptions in order to deconfuse yourself.
Instead of trying to begin by definitively establish axioms, we need to accept that reasoning “begins in the middle”.
All arguments are “infinitely perfectible” and “the view from nowhere” doesn’t really exist.
So instead of aiming for one true perfect irrefutible argument, I suspect it’d make more sense to speedrun a whole bunch of philosophical trajectories and then prioritise further research based on cruxes.
It’s quite a different approach to how philosophy has been conducted in the past, but should we honestly be expecting philosophy to be conducted in the same way.
I’m finding myself drifting away from this community.
I don’t know, you invest so much over years and sometimes it goes the way you want, sometimes it doesn’t.
There’s so much here that’s good, but there’s also things that could be even better and I don’t know where’s the best place to pursue that, but I don’t think it’s here. Obv. different people will have different ideas about what’s important, I suppose that’s just the nature of things. I suppose I now view LW as a place to get up to speed on a lot of things and then go do things elsewhere.
I’ll always be grateful for all the things I learned and I’ll still post here occasionally, I’ll just don’t know if I’ll write much specifically for LW (just realised after posting that I still need to go through my drafts folder and see if there’s anything worth pushing out!).
re: making models better at conceptual research so they can help with alignment more, this seems like obviously a very tenuous story of impact and i wish people who are working on this rn stopped working on it. i am happy to argue with anyone who disagrees.
I’d be keen to hear your thoughts here.
Do you think it’d be a mistake to make models better at philosophy?
Reading List on Wise AI
I was just thinking a subtitle rather than a paragraph.
I don’t see how it makes LW a better place to share your own writing
Even though it is theoretically possible that authors only use subtitles because they feel forced to by other authors using it, I would suggest that our default should be to assume that subtitles developed for a reason.
Would be very curious to know his reasons.
Subtitles should be optional
I have a lot of sympathy with this perspective, but some parts feel like you’re lapsing into negativity (this doesn’t mean your overall analysis is necessity wrong).