Malthusian Competition (not as bad as it seems)


Sooner or later, someone is going to do this.

What rules govern “ungovernable” AGI?

In a state of nature, all animals exist in a Malthusian Equilibrium. That is to say, the population of a species increases until some environmental factor (food, habitat, disease) limits its growth.

population increases until carrying capacity is met

By contrast, in economics the production of goods is limited by the laws of supply and demand.

price increases or decreases until supply meets demand

While these may look like different pictures, they are actually the same. As an illustration, consider pets. Are there a commodity manufactured in order to meet demand within the market, or are they animals evolved to fit into a particular niche (human companionship)? The answer is, obviously, both.

This very good boy was created by AI to meet the demand for cute dog pics

These two laws have very different emotional valances “the invisible hand of the market” vs “nature red in tooth and claw”. But in fact the same law is at work in both places.

So when we ask “what law will govern ungovernable AGI?”, the answer is the same: the law of supply and demand which is also the law of Malthusian Competition.

A great deal of hand-wringing in the alignment community is based around the idea that AGI will “out-compete” humans economically. But taken another way, this is simply another way of stating that AGI is going to make all of us enormously rich.

Capitalism: making people richer for over 200 years

So long as property rights are respected, humans will continue to have a comparative advantage in something, and whatever that is we will be much richer in a world with hyper-competitive AGI than we are today.

What does this have to do with alignment?

Consider two AGIs, otherwise identical:

  1. The first AGI is “owned” by a human being and is programmed to fulfill the owner’s wishes as perfectly as possible

  2. The second AGI exists in perfect competition with its peers and competes to sell its services to human beings (who own all of the capital)

I postulate that there is no meaningful difference between AGI 1. and AGI 2. We should not expect “ownership” to confer any particular safety or other benefits over market competition when building AGI.

But what if the AGI owns all of the capital and we’re the ones subject to Malthusian Competition?

If you’re reading this essay, I suspect you are part of the richest 1% of people on earth. Most people do not need to ask “what would life be like if all of the wealth and power was controlled by others?”. They already know.

All I can say is that in a free market with voluntary transactions, economic competition makes people richer not poorer. So, however bad it will be, you will be better off than you are now.

None of this matters if the robots kill us all and take our stuff


I’m worried that the AGI will make a bunch of Von Neumann probes and fill the galaxy taking everything before humans can claim it

That is a very specific concern.

My specific proposal is that we declare “the future” a commons owned by all of humanity and that if an AGI wants to take an action which “consumes” some slice of that future, they must pay (with literal money) for it. This is one implementation of the “corrigible utility function” that I describe here.

If it helps, you can think of this as a form of Land Value Tax.

The part where I confess to having weird moral preferences

If someone built a self-replicating LLM and asked me what it’s “moral worth” is, I would probably say “at least as much as a fruit fly”. That is to say: not much, but as long as it’s not hurting anyone it should be allowed to continue to exist.

Suppose you brought me a jar with 100 fruit flies and said “I’m going to kill 50 of these fruit flies to make room for 50 LLMs” I would probably be okay with that.

Suppose you showed me 100 dogs, and said “I am going to kill 50 of these dogs to make room for 50 LLMs” I would probably not be okay with that.