I think future technology all has AI as a pre-requisite?
My high conviction hot take goes further: I think all positive future timelines have AI as a pre-requisite. I expect that, sans AI, our future—our immediate future: decades, not centuries—is going to be the ugliest, and last, chapter in our civilization’s history.
I have been in the position of trying to moderate a large and growing community—it was at 500k users last I checked, although I threw in the towel around 300k—and I know what a thankless, sisyphean task it is.
I know what it is to have to explain the same—perfectly reasonable—rule/norm again and again and again.
I know what it is to try to cultivate and nurture a garden while hordes of barbarians trample all over the place.
But...
If it aint broke, don’t fix it.
I would argue that the majority of the listed people penalized are net contributors to lesswrong, including some who are strongly net positive.
I’ve noticed y’all have been tinkering in this space for a while, I think you’re trying super hard to protect lesswrong from the eternal september and you actually seem to be succeeding, which is no small feat, buuut...
I do wonder if the team needs a break.
I think there’s a thing that happens to gardeners (and here I’m using that as a very broad archetype), where we become attached to and identify with the work of weeding—of maintaining, of day after day holding back entropy—and cease to take pleasure in the garden itself.
As that sets in, even new growth begins to seem like a weed.
Fine. You win. Take your upvote.
Big fan of both of your writings, this dialogue was a real treat for me.
I’ve been trying to find a satisfying answer to the seeming inverse correlation of ‘wellbeing’ and ‘agency’ (these are very loose labels).
You briefly allude to a potential mechanism for this[1]
You also briefly allude to another mechanism with explanatory power for the inverse[2] - i.e. that while it might seem an individual is highly agentic, they are in fact little more than a host for a highly agentic egregore
I’m engaged in that most quixotic endeavour of actually trying to save the world[3] [4], and thus I’m constantly playing with my world model and looking for levers to pull, dominos to push over, that might plausibly -and quickly- shift probability mass towards pleasant timelines.
I think germ theory is exactly the kind of intervention that works here—it’s a simple map that even a child can understand, yet it’s a 100x impact.
I think there’s some kind of ‘germ theory for minds’, and I think we already have all the pieces—we just need to put them together in the right way. I think it’s plausible that this is easy, rapidly scaleable and instrumentally valuable to other efforts in the ‘save the world’ space.
But… I don’t want to end up net negative on agency. In fact my primary objective is to end up strongly net positive. I need more people trying to change the world, not less.
Yet… that scale of ambition seems largely the preserve of people you’d be highly unlikey to describe as ‘enlightened’, ‘balanced’ or ‘well adjusted’; it seems to require a certain amount of delusion to even (want to) try, and benefit from unbalanced schema that are willing to sacrifice everything on the altar of success.
Most of the people who seem to succcessfully change the world are the people I least want to; whereas the people I most want to change the world seem the least likely to.
Since the schools that removed social conditioning and also empowered practitioners to upend the social order, tended to get targeted for destruction. (Or at least so I suspect and some people on Twitter said “yes this did happen” when I speculated this out loud.)
In the Buddhist model of human psychology, we are by default colonized by parasitic thought patterns, though I guess in some cases, like the aforementioned fertility increasing religious memes, they should be thought of as symbiotes with a tradeoff, such as degrading the hosts’ episteme.
I don’t expect to succeed, I don’t expect to even matter, but it’s a fun hobby.
Also the world does actually seem to be in rather urgent need of saving; short of a miracle or two it seems like I’m unlikely to live to enjoy my midlife crisis.
I don’t think there’s anything wrong with cultivating a warrior archetype; I strive to cultivate one myself.
Would love to read more on this.
+1 for the 14⁄24 club.
Hmmm, where to start. Something of a mishmash of thought here.
Actually a manager, not yet clear if I’m particularly successful at it. I certainly enjoy it and I’ve learned a lot in the past year.
Noticing Panic is a great Step 0, and I really like how you contrast it to noticing confusion.
I used to experience ‘Analysis Paralysis’ - too much planning, overthinking, and zero doing. This is a form of perfectionism, and is usually rooted in fear of failure.
I expect most academics have been taught entirely the wrong (in the sense of https://www.lesswrong.com/posts/a7n8GdKiAZRX86T5A/making-beliefs-pay-rent-in-anticipated-experiences) heuristics around failure.
My life rapidly became more agentic the more I updated toward the following beliefs:
Failure is cheap
You have an abundance of chances to get it right
Plans are maps, reality is terrain. Doing happens in reality. Thus you can offload a bunch of cognitive work to reality simply by trying stuff; failing is one of the most efficient ways of updating your map, and can sometimes reward you with unexpected success (take a look around at all the ‘stupid’ stuff that actually works/succeeded).
Thus my management strategy is something like:
Have a goal
Build a model of the factors that contribute to that goal
Determine my constraints (e.g. do I have a rigid deadline)
Notice my affordances (most people always underestimate this)
What resources do I have, e.g.
Who can I ask for help?
What work has already been done that I can use (don’t reinvent the wheel)
What actions are available to me
What is the smallest meaningful step I can take toward my goal?
What is the dumbest thing I can do that might actually work?
Prioritize my time
What needs to be done today vs this week vs this month vs actually doesn’t need to be done
So I’ve reduced a combinatorially explosive long term goal into a decent heuristic for prioritizing actions, and then I apply it to the actions I can actually take at different timescales… which is usually an easy choice between a handful of options.
Then I do stuff, and then I update/iterate based on the results.
And sometimes stuff just happens that moves me towards my goal (or my goal towards me) - life is chaotic, and if you’re rigidly following a plan then that chaos is always working against you. Whereas if you’re adaptable and opportunistic—that chaos can work for you.
I guess all of this boils down to: invest in your world model, not your plan.
Re: average age of authors/laureates and average team size
Are these data adjusted for demographic changes? i.e. Aging populations in most western countries, and general population growth.
I think this is a mistake to import “democracy” at the vision level. Vision is essentially a very high-level plan, a creative engineering task. These are not decided by averaging opinions. “If you want to kill any idea in the world, get a committee working on it.” Also, Deutsch was writing about this in “The Beginning of Infinity” in the chapter about democracy.
We should aggregate desiderata and preferences (see “Preference Aggregation as Bayesian Inference”), but not decisions (plans, engineering designs, visions). These should be created by a coherent creative entity. The same idea is evident in the design of Open Agency Architecture.
Democracy is a mistake, for all of the obvious reasons.
As is the belief amongst engineers that every problem is an engineering problem :P
We have a whole bunch of tools going mostly unused and unnoticed that could, plausibly, enable a great deal more trust and collaboration than is currently possible.
We have a whole bunch of people both thinking about and working on the polycrisis already.
My proposal is that we’re far more likely to achieve our ultimate goal—a future we’d like to live in—if we simply do our best to empower, rather than direct, others.
I expect attempts to direct, no matter how brilliant the plan or the mind(s) behind it, are likely to fail. For all the obvious reasons.
(caveat: yes AGI changes this, but it changes everything. My whole point is that we need to keep the ship from sinking long enough for AGI to take the wheel)
Joshua Williams created an initial version of a metacrisis map
It’s a good presentation, but it isn’t a map.
A literal map of the polycrisis[1] can show:
The various key facets (pollution, climate, biorisk, energy, ecology, resource constraints, globalization, economy, demography etc etc)
Relative degrees of fragility / timelines (e.g. climate change being one of the areas where we have the most slack)
Many of the significant orgs/projects working on these facets, with special emphasis placed on those that are aware of the wider polycrisis
Many of the significant communities
Many of the significant funders
Do you mean that it’s possible to earn by betting long against the current market sentiment?
I mildly prefer polycrisis because it’s less abstract. The metacrisis points toward a systems dynamic for which we have no adequate levers, whereas the polycrisis points toward the effects in the real world that we need to deal with.
I am assuming we live in a world that is going to be reshaped (or ended) by technology (probably AGI) within a few decades, and that if this fails to occur the inevitable result of the metacrisis is collapse.
I think the most impact I can have is to kick the can down the road far enough that the accelerationistas get their shot. I don’t pretend this is the world I would choose to be living in, or the horse I’d want to be betting on. It is simply my current understanding of reality.
Hence: polycrisis. Deal with the symptoms. Keep the patient alive.
The polycrisis has been my primary source of novelty/intellectual stimulation for a good long while now. Excited to see people explicitly talking about it here.
With regard to the central proposition:
I think if there were A Plan to make the world visibly less broken, made out of many components which are themselves made out of components that people could join and take responsibility for, this would increase the amount of world-fixing work being done and would meaningfully decrease the brokenness of the world. Further, I think there’s a lot of Common Cause of Many Causes stuff going on here, where people active in this project are likely to passively or actively support other parts of this project / there could be an active consulting / experience transfer / etc. scene built around it.
I think this is largely sensible and true, but consider top-down implementation of such to be a pipe dream.
Instead there is a kind of grassroots version where you do some combination of:
1.) Clearly state the problems that need to be worked on, and provide reasonable guidance as to where and how they might be worked on
2.) Notice what work is already being done on the problems, and who is doing it (avoid reinventing the wheel/not invented here syndrome; EA is especially guilty of this)
3.) Actively develop useful connections between 2.)
4.) Measure engagement (resource flows) and progress
And from that process I expect something like a plan to emerge—it won’t be the best possible plan, but it will be far from the worst plan, more adequate than not, and importantly it will survive contact with reality because reality was a key driver in the development of the plan.
The platform for generating the plan would need to be more-open-than-not, and should be fairly bleeding edge—incorporating prediction markets, consensus seeking (polis), eigenkarma etc
It should be a design goal that high value contributions should be noticed, no matter the source. An example of this actually happening is where Taiwan was able to respond rapidly to Covid thanks to a moderator noticing and doing due diligence on a post in the .g0v forums re: covid, and having a process in place where that information could be escalated to government.
It should also be subject to a serious amount of adversarial testing—such a platform, if successful, will influence $ flows, and thus will be a target for capture/gaming etc etc.
As it stands, we’re lacking all 4. We’re lacking a coherent map of the polycrisis[1], we’re lacking in useful+discoverable communication channels, we’re lacking meaningful 3rd party measurement.
As it stands, the barriers to entry for those wishing to engage in meaningful work in this space are absurd.
If you lack the credentials and/or wealth to self-fund, then you’re effectively excluded—a problem which was created by an increasingly specialized world (And the worldview, cultural dynamics and behaviours it engenders) has gatekeepers from that same world, enforcing the same bottlenecks/selective pressures of that world on those who would try to solve the problem.
The neighbourhood is on fire, and the only people allowed to join the bucket chain are those most likely to be ignoring the fire—so very catch-22.
I think there’s a ton of funding available in this space, specifically I think speculating on the markets informed by the kind of worldview that allows one to perceive the polycrisis has significant alpha. I think we can make much better predictions about the next 5-10 years than the market, and I don’t think most of the market is even trying to make good predictions on those timescales.
I’d be interested in talking/collaborating with anyone who either strongly agrees or disagrees with this logic.
On this note, if anyone wants to do and/or fund a version of aisafety.world for the polycrisis, I’m interested in contributing.
There’s a guy called Rafe Kelley on youtube who has a fairly good answer to this, which I’m going to attempt to summarize from memory because I can’t point you toward any reasonable sources (I heard him talking about it in a 1h+ conversation with everyone’s favourite boogeyman, Jordan Peterson).
His reasoning goes thus:
1.) We need play in order to develop: play teaches us how to navigate Agent—Arena relationships
This speaks to the result of playground injuries increasing despite increased supervision—kids aren’t actually getting to spend enough time playing in the physical Arena, their capability to navigate it is underdeveloped because of excess indoor time and excess supervision.
2.) We need rough play (e.g. play fighting), specifically, to teach us a whole bunch of capabilities around Agent—Agent—Arena relationships; conflict, boundaries, emotional regulation are all, Rafe argues, rooted in rough play.
Through rough and tumble play, we learn the physical boundaries between agents. We learn that it hurts them, or us, when boundaries are crossed. We learn where those boundaries are. We learn to regulate our emotions with respect to those boundaries.
These are highly transferable, core skills, without which human development is significantly stunted.
Depending on the kind of support they’re looking for https://ceealar.org could be an option. At any one time there are a handful of people staying there working independently on AI Safety stuff.
Wholely agree with the ‘if it works it works’ perspective.
Two minor niggles are worth mentioning:
As I understand it, eating any amount will signal the body to stop fasting. The overnight fast is the only one most people have and it seems to be quite important for long term metabolic health.
Your body has several inputs to its internal clock, and the two most significant ones are light and food. So there’s a pathway where this ‘solution’ might also be reinforcing the problem.
Niggles aside, if it works it works. And nothing is more important than sleep for health. If you are currently chronically sleep deprived and the usual things aren’t helping—absolutely try this.
Thanks Elizabeth for sharing another potential tool for helping people.
So I’m basically the target audience for the OP—I read a lot, of all kinds of stuff, and almost zero papers. I’m an autodidact with no academic background.
I appreciated the post. I usually need a few reminders that ‘this thing has value’ before I finally get around to exploring it :)
I would say, as the target audience, I’m probably representative when I say that a big part of the reason we don’t read papers is a lack of access, and a lack of discovery tools. I signed up for Elicit a while back, but as above—haven’t gotten around to using it yet :D
In my experience the highest epistemic standard is achieved in the context of ‘nerds arguing on the internet’. If everyone is agreeing, all you have is an echo chamber.
I would argue that good faith, high effort contributions to any debate are something we should always be grateful for if we are seeking the truth.
I think the people who would be most concerned with ‘anti-doom’ arguments are those who believe it is existentially important to ‘win the argument/support the narrative/spread the meme’ - that truthseeking isn’t as important as trying to embed a cultural norm of deep deep precaution around AI.
To those people, I would say: I appreciate you, but there are better strategies to pursue in the game you’re trying to play. Collective Intelligence has the potential to completely change the game you’re trying to play and you’re pretty well positioned to leverage that.
Re: EMH is false, long GOOG
I wish you’d picked a better example.
tl;dr LLMs make search cost more, much more, and thus significantly threaten GOOG’s bottom line.
MSFT knows this, and is explicitly using Bing Sydney as an attack on GOOG.
I’m not questioning the capabilities of GOOG’s AI department, I’m sure Deepmind have the shiniest toys.
But it’s hardly bullish for their share price if their core revenue stream is about to be decapitated or perhaps even entirely destroyed—ad based revenue has been on shaky ground for a while now, I don’t think it’s inconceivable that one day the bottom will fall out.
re: EMH in general
EMH gets weaker the less attention an asset has, the further out in time relevant information is (with significant drops around 1yr, 2yr, 5yr), and the more antimemetic that relevant information is (i.e. Sin is consistently undervalued because it makes people feel bad to think about. Most recently we saw this in coal, and I’m kicking myself for not getting in on that trade.).
Will GOOG go up? Maybe.
Is GOOG undervalued? Extremely unlikely.
Have you read Mark Solms Hidden Spring?
AI Therapy isn’t the first domino to fall, AI Customer Service is (it’s already falling).
95% of customer service humans can be replaced by a combination of Whisper+GPT; they (the humans) are already barely agentic, just following complex scripts. It’s likely that the AI customer service will provide a superior experience most of the time (less wait times, better audio quality at a minimum, often more competent and knowledgeable too, plausibly capable of supporting many languages).
Obviously huge cost savings so massive incentive for companies to replace humans (and why it’s already started with even weak chatbots).
Investing in it is tricky, same problem you mentioned at the start—picking which horse is going to win this race, most probably either don’t exist or aren’t publicly tradeable.
Zoom is a potential frontrunner, they acquired Solvvy last year which suggests some strategic awareness of this trend/potential market.
It’s an interesting framework, I can see it being useful.
I think it’s more useful when you consider both high-decoupling and low-decoupling to be failure modes, more specifically: when one is dominant and the other is neglected, you reliably end up with inacccurate beliefs.
You went over the mistakes of low-decouplers in your post, and provided a wonderful example of a high-decoupler mistake too!
Aside from https://thezvi.wordpress.com/2017/08/12/choices-are-really-bad/ there’s also the consideration of what choice I offer you, or how I frame the choice (see Kahneman’s stuff).
And that’s just considering it from the individual psychological level, but there are social/cultural levers and threads to pull here too.
I think the optimal functioning of this process is cyclical with both high decoupling phases and highly integrated phases, and the measure of balance is something like ‘this isn’t obviously wrong in either context’.