Views my own, not my employers.
cdt
Karma: 309
You’re more than welcome to disagree with the premise (“Planning law is not designed in principle to protect public goods”). I wanted to avoid opening a dispute about whether planning law is good at doing this or even can achieve it in practice—essentially the arguments from the Abundance movement—because I don’t think it’s an effective path (surfaces cruxes?) in the conversation.
It would be very easy to get in long drawn-out discussion about the ideas from Abundance movement which takes the focus away from AI 2040 and towards modern politics which there’s a bit of a taboo on LW anyway.
Yes, I think so too. The directions of land use are hard to predict from the outside—even just for agriculture there’s a lot of differing thoughts about things like animal welfare (considering most agricultural land is grazing land and intensification will make welfare worse), efficacy of vertical farming, release of demand for increased food variety (my personal pet hypothesis). A lot of these questions depend on how much time we spend around take-off without being post-singularity. I also get the impression that whether land is scarce during takeoff is not a settled question, but I don’t have a strong argument on this.
One of the many worries is that the greatly increased value AI gains from industrial activity would threaten the biosphere, and having lots of conservation land plausibly elevates the importance and urgency of that
Yes, this is true. Unfortunately these lands require active management, so one has to ask where the capital and labor is coming from. This is doubly true for heritage/cultural conservation because it exists so close to people (who are as a general rule expensive).
These lands are being set aside for a purpose but there is no discussion of what that purpose is other than a vague “good for people”. If it’s primarily tourism that’s a very different argument than ecosystem service (e.g. air quality for humans), or existence value, or option value, or whatever. So good to bring up all these topics now rather than later.
This is a good point, but there’s something a little contradictory about this. One of the methods of safeguarding human welfare during the takeoff is strong property rights but arcologies would be designed preferentially to escape planning law. Planning law is designed in principle to protect public goods[1] so this strikes me as a dangerous precedent.
Why are people moving to arcologies? Is it the power of the market, the power of preference, or what? If preserves still have significant economic activity (by being consumers), then the use of the word “preserve” is unusual. Taking the example in the story, if these preserves have a lot more tourism, that requires new infrastructure which inevitably change the character of the landscape.
So in sum (1) Incentivising avoiding the law probably bad on average and (2) does not explain the decisions of people or how these preserves can actually work
- ^
Not an invitation to argue about whether it actually does this or not
- ^
The Conservation Ethic in AI 2040
cdt’s Shortform
Re: EAs having more/less leverage in the future
Do we think that EA/Rats will have enough leverage in the future that they could repeat the success of pressuring the labs against the confidential non-disparagement-agreement scandal? (agree/disagree vote)
Thanks for this. I think that mode collapse in the case of LLM-judging-LLM is related to “LLM self-preference”. I don’t know if this paper still holds up because it’s quite old now for an LLM paper and I haven’t read any good follow-ups. Raises the question whether self-preference extends to semantic topics in output (code style/arguments/etc.).
Many people in the AI safety movement see these causes as the same thing, or the “root cause” behind all these problems. See all the talk about coordination and also Richard Ngo’s sociopolitical thinking.
Why? First, because the solutions threatened specific interests: carbon taxes and drilling restrictions would hit certain industries, and so those industries spent decades and billions deliberately coding climate as a left-wing hobbyhorse. A polarized issue is a stalled issue, after all. Second, the proposed solutions happened to rhyme with the left tribe’s preexisting preferences like regulation, international coordination, constraints on industry, so the climate change movement was nicely absorbed into the greater “left vs. right” tribal fistfight.
This is too oversimplified to be useful, really. There are many right-wing movements that have some environmentalist aims, and “regulation is a left wing idea” is so incredibly broad as to be basically useless. Take Reform UK, a right-wing party, having almost half its prospective voters being pro-climate-action, and half its prospective voters being against it (CTRL + F for “climate” in the data table). Intuitions in this space are too over-indexed on surface-level news in the last 10 years and it’s leading them astray IMO.
If you wanted to take a lesson away from environmentalism, perhaps it’s that AI safety will split along nationalist and internationalist lines. Which I suppose we’ve seen some examples of already.
I suppose operationalising and making explicit that knowledge graph is one of the remaining key problems, huh?
mode collapse: most of those agents will take very similar routes, even if they call them by different names. If they search the literature, they’ll converge on the same 3-5 papers, and discuss the same points in those papers. The severity of this varies, but I’ve tried all versions of Claude Opus and GPT over the last year (I did not get that much time with Fable). Even when prompted for diversity, e.g. with an initial observation or claim from the human researcher it is possible for the agent to revert to its median trajectory — sometimes after a cursory investigation of that initial seed. This is probably the main reason I can’t go fully hands-off right now.
Back thinking about this: do you know any public-facing examples or literature on this? I was writing about auditing model outputs when I realised that I was bringing up the same idea. This problem makes it hard for LLMs to act as autonomous judges since they will prefer the same set of concepts. When you say “mode collapse” I suppose it’s too similar in words to “model collapse” to Google which is a related but different issue.
I want to like this, your “imagine eating stopped overnight” was a nice framing device. I think you would capture more people with it being a fifth of its current length, and perhaps trimming some of the LLMisms: “the tether snaps”, “still capable of producing misery, but no longer subsidized by biology”, etc.
I am not advocating in favor of it, or against it
I think this post would have been better than your first comment, since the first comment misses the context of this dispute between you and PauseAI US. It’s very hard to interpret your first comment without this information.
It is hard to understand the population economic measures (chapter 3) if the data have not been corrected for differences between the survey and workforce occupational composition (last figure).
“Over a third expect AI to be able to do most or nearly all of their work tasks next year” but 3 in 10 of those respondents are mathematical/computational jobs. The survey doesn’t seem representative of the population at all.
Thanks for your thoughts. I’m thinking along the same lines for biological research, where the ground-truth is very far away and many analytical outcomes can seem plausible.
I am deliberately assuming humans as the final decision-makers, and asking how far we can get by augmenting human judgement
I agree with this stance and I think we should keep humans in the loop as much as possible. But I wonder what the response should be if you run several AI agents on the same topic that come to different conclusions. (Although it seems like you have the opposite problem of them all doing exactly the same thing).
are differences in productivity between academic fields
I think even looking within subfields (say, within biology), the rate of uptake of similar ideas can be very different. Some fields are just more amenable to disproving ideas more than others, considering the availability of evidence and the possible variation created by experiment. I would expect AI to make the gaps in theory smaller because synthesis across fields is easier, but I don’t think it will change the fundamental differences in experimental design.
Sign of mainstream evolutionary biology interest in ML theory here.
Many people would like to use AF to solve alignment. AF may not require empirical grounding to be sound or valid but it must describe the contemporary examples of the problem in order to form a solution to said problem.
I’m sympathetic to this viewpoint, but this particular incident is a move in the direction of nationalisation which I don’t think most have positive feelings about.
Okay, I should probably caveat this by saying that Google does have a phenotyping service in the sense of Google Health and related products, but I’m not aware of any work in this area and I’m not sure how much things like heart rate/BMI etc. are usable in genomic research. Or that they can even be released to be used in this way.
drift-diffusion threshold
I don’t think this is a typical phrase, can you cite this? Perhaps you mean the drift barrier hypothesis instead.
I have no idea what innovation looks like in PRS. ML is all good but the bottleneck is (presumably) phenotyping which none of these companies want to do.
Based on what you’re saying, I think you’d like to argue “Planning law leads to bad outcomes on average, therefore arcologies avoiding planning law is good”. I think this is a good argument.