Size circumscribes – it has no room For petty furniture – The Giant tolerates no Gnat For Ease of Gianture –
Repudiates it, all the more – Because intrinsic size Ignores the possibility Of Calumnies – or Flies.
~ Dickinson
Fergus Fettes
Karma: 108
Hey, using git branches immediately jumped out to me as a great way of doing this, is there any code you could share or could you describe some technical pros and cons that you found with the approach, in case I try it out :).
Very interesting and thoughtful post as well, reminds me of a bad trip.
Tinker Bell Theory and LLMs
“psychomotor retardation in depression” ⇐ this! sorry for lack of clarity, maybe I was using outdated terminology?
Actually you are right, it makes more sense as two independent axes. ‘Suffering’ on one axis (X) and ‘Verbal ability’ on the other (Y)-- with Tinker Bell on max(X) and high Y, animals on max(X), min(Y), humans (almost?) max(X), max(Y), LLMs on min(X), increasing Y.
It is in fact the independence of the two axes that was interesting, I botched that.
Frankly, just as its clear that Bing shows signs of intelligence (even if that intelligence is different from human), I think it is also clear that it will be able to suffer (with a kind of suffering that is different from human).
I just thought the visceral image of an animal consumed with physiological suffering was a useful image for understanding the difference.
Surely it is almost the definition of ‘embodied cognition’ that the qualia you mention are fundamentally dependent on loopback with eg. muscles, the gut.
Not only sustained in that way but most everything about their texture, their intricate characteristics.
To my mind it isn’t that chatgpt doesn’t implement an analog of the processing, but it doesn’t implement an analog of the cause. And it seems like a lot of suffering is sustained in this embodied cognition/biofeedback manner.
Not exactly. I suppose you could do so.
Do you really think it is not acceptable to assume that LLMs don’t implement any analogues for that kind of thing though?
Maybe the broader point is that there are many things an embodied organism is doing, and using language is only occasionally one of them. It seems safe to assume that an LLM that is specialized on one thing would not spontaneously implement analogues of all the other things that embodied organisms are doing.
Or do you think that is wrong? Do you, eg., think that an LLM would have to develop simulators for things like the gut in order to do its job better, is that what you are implying? Or am I totally misunderstanding you?
I think humans underestimate their own flexibility and adaptability. I’m not sure where all the anxiety disorders of our age come from, but it certainly isn’t a deep property of biology to struggle with novel circumstances, rather the opposite.
I guess I would recommend travel, moving to a different city, changing careers, all the regular ‘open minded’ things that keep people fresh. There is already an immense amount of diversity in human societies as they are, this will certainly ramp up, so makes sense to start sampling more widely now to prepare for that.
Context: “For example, it is now easy to radically modify bodies in a time-scale that is much faster than evolutionary change, to study the inherent plasticity of minds without eons of selection to shape them to fit specific body architectures. When tadpoles are created to have eyes on their tails, instead of their heads, they are still readily able to perform visual learning tasks. Planaria can readily be made with two (or more) brains in the same body, and human patients are now routinely augmented with novel inputs such as sensory substitution or novel effectors, such as instrumentized interfaces allowing thought to control engineered devices such as wheelchairs in addition to the default muscle-driven peripherals of their own bodies. The central phenomenon here is plasticity: minds are not tightly bound to one specific underlying architecture (as most of our software is today), but readily mold to changes of genomic defaults. The logical extension of this progress is a focus on self-modifying living beings and the creation of new agents in which the mind:body system is simplified by entirely replacing one side of the equation with an engineered construct. The benefit would be that at least one half of the system is now well-understood.”
Levin, 2022
Can you give me some more clues here, I want to help with this. By vectors are you talking about similarity vectors between eg. lines of text, paragraphs etc? And to optimize this you would want a vector db?
Why is sync difficult? In my experience any regular postgres db will have pretty snappy sync times? I feel like the text generation times will always be the bottleneck? Or are you more thinking for post-generation weaving?Maybe I also just don’t understand how different these types of dbs are from a regular postgres..
I also find the atoms argument very uncompelling. There is so much space and solar energy in the asteroid belt, I’m sure there is a good chance that the ASI will be chill.
However, I think Yudkowsky is shouting so loud because even if that chance of asi apocalypse is only 5%, that is 5% multiplied by all possible human goodness, which is a big deal to our species in expectation.
Personally I think the totality of the biological ecosystems on earth (including humans) will still be interesting to an ASI, so I’d hope they’d let it tick on as a museum piece.
But it’s hard for us. It would be very easy for an ASI. Even with no advancement in tech, the ASI can ride on the starlinks into space.
We are stuck here amongst the biology for very obvious reasons.
I’m not sure what you think I believe, but yeah I think we should be looking at scenarios in between the extremes.
I was giving reasons why I maintain some optimism, and maintaining optimism while reading Yudkowsky leaves me in the middle, where actions can be taken.
If there was a system which was really good at harvesting energy and it was maxxed out on intelligence, atoms might be very valuable, especially atoms close to where it is created
The number of atoms on earth is so tiny. Why not just head to the asteroid belt where you can really build?
Do you pick up every penny that you pass in the street?
The amount of energy and resources on Earth would be a rounding error in an ASI’s calculations. And it would be a rounding error that happens to be incredibly complex and possibly unique!
Maybe a more appropriate question is, do you pick every flower that you pass in the park? What if it was the only one?
If the ASI was 100% certain that there was no interesting information embedded in the Earths ecosystems that it couldn’t trivially simulate, then I would agree.
Whatever happened here is an interesting datapoint about the long-term evolution of thermodynamic systems away from equilibrium.
From the biological anchors paper:
This implies that the total amount of computation done over the course of evolution from the first animals with neurons to humans was (~1e16 seconds) * (~1e25 FLOP/s) = ~1e41 FLOP.
Note that this is just computation of neurons! So the total amount of computation done on this planet is much larger.
This is just illustrative, but the point is that what happened here is not so trivial or boring that its clear that an ASI would not have any interest in it.
I’m sure people have written more extensively about this, about an ASI freezing some selection of the human population for research purposes or whatever. I’m sure there are many ways to slice it.
I just find the idea that the ASI will want my atoms for something trivial, when there are so many other atoms in the universe that are not part of a grand exploration of the extremes of thermodynamics, unconvincing.
See my reply above for why the ASI might choose to move on before strip-mining the planet.
You might like this[0]. If I were to make a dashboard per your description my first attempt would be to replicate realtime and per capita that paper.
[0]: Chaisson, E. J. (2010). Energy rate density as a complexity metric and evolutionary driver. Complexity, 16(3), 27–40. doi:10.1002/cplx.20323