Fergus Fettes
I mean, I’m being glib, I don’t really want to call it Avatar earth either. It gets the message across tho
And for the record, I’m not an Accelerationist. I see the following possibilities as live:
On Earth:conventional defeat—was causes irrecoverable technological reset
national park—conservationist
avatar earth—ecological engineering creates a biosphere 10-100x richer than todays, ~1 trillion intelligent beings, some of them conventionally human
datacenters—accelerationist runaway
Beyond Earth:
unaligned machines—accelerationist runaway
uploaded mind-children—human-shaped influence contributes to the lightcone
humans in space-ships-- ~zero credence
My preferred outcome is avatar earth and a lightcone that remembers us. And yes, we have some influence.
Declarations of war do not cause war. Declarations of peace do not cause peace. Those actions are correlated with the beginnings and ends of conflicts but surely no-one would mistake them for causes?
Those terrible non-examples aside, certainly our choices in discourse-space have downstream effects. But sometimes facts in meatspace overwhelm them with their urgency (such as eg. population growth and a bad harvest, which might cause a war, or total economic exhaustion, which might cause peace).
But if you accept transhumanism then we don’t really disagree, we just disagree about whether or not it’s valid to call the results of that process ‘human’. That part is a point of normative moral debate. I suppose I picked a side by saying the results will not be human and you picked a side by saying they will. I’m happy to walk back that point—a sufficiently aligned machinic lightcone is certainly some kind of a win for humanity.
The alignment problem is irrelevant in the face of the singularity. With the best will in the world the machines will not be able to overcome our limitations without making drastic changes. Value alignment doesn’t change the trajectory of the biological:machinic ratio of flop/s. However much they may love and cherish us if we do solve alignment, at some point we are de facto obsolete and the lightcone has been ceded. What does alignment mean to you?
And I’m really not sure what you interpreted as a moral argument. Accelerationism is a is-oriented ideology, not an ought-oriented ideology.
Intelligent design? You think you can do germline editing to try to keep pace with machines in biology without ending our species? Nope. Posthumanism is an extinction event, don’t kid yourself.
Not that you shouldn’t try, but don’t pretend that’s a way to preserve humanity.
It will ultimately fail as well as biology has harder limits than lithography. Recall we are also intelligently designing our machines and their generation time and iteration loops are tight.
I don’t recall making a normative moral argument. I was simply explaining why accelerationism is antimemetic.
However, while you might be right that accelerationism has no ought, it has a lot of is.
Fermi rescued us from Copernicus. We are the pinnacle of creation after all. There are none like us, and we extincted everything that threatened that presumption. We like to suppose we can halt progress and maintain our iron grip on Gaia indefinitely—most of our intuitions come from slow time, the time when culture evolved, during which society didn’t change much generation to generation.
If, otoh, as Vinge suggested, there will come a time when our machines outpace us in every way that matters, we might have to cede the throne. No tyrant welcomes his assassin.
Are you pretending you don’t know about Accelerationism, about Land and his acolytes? The antibodies this community has to that material have created a memory vacuum so powerful that posts like this somehow continue to appear as if any of this is news.
e/acc is a meme that cannot die because the simple fact of accelerating technological change can’t be ignored. Moore refuses to die, it just keeps changing form. It’s absolutely bonkers that we are still getting hardware speedups so long after the death of Classical Moore but here we are. It continues and continues.
The reason the torch, such as it is, is carried by degenerate crowds like e/acc is that the real Singularitrians and Accelerationists are not on twitter shitposting. Accelerationism is a fundamentally anti-memetic ideological package. This is why Land himself has such staying power as an anti-hero. Finding someone else to communicate that belief system properly has proven nigh impossible. But it has so much weight that it leaks into the discourse anyway, again and again, in juvenile and degenerate forms.
What is up with e/acc? Well, no-one else is burning their incense at the alter of the incredible technoevolutionary Providence that continues to drive us into the realm of the seemingly impossible. It is an alter that will have its acolytes.
I wonder what are the best ways to start gathering the training data for this now, before the hardware or the startup exists to make good use of it. Harnesses that record your responses to incoming media items for example. Obviously agent harnesses are rich records of our choices under a particular set of action spaces, but most of the choices we make are currently lost. This feels like something worth starting today.
Very cool! Could you share your code at all? I’d love to explore this a little.
I adore the broccoli tree. I would be very happy to convert the dataset you used to make those pngs into an interactive network visualization and share it with you as an index.html. It would take all of an hour.
I do kind of agree with the other comments that, having noticed something, finding more of that stuff in that area is not so surprising. I think it would be good to get more context and explore the region more before concluding that that particular set of generations is significant.However, I do think there is something to the mans penis. It’s interesting that it collapses so quickly to something so specific in that particular branch. Not sure if I have any other comments on it for now though.
This is the right kind of cartography for 2024.
Submission: MMDo2Little
A follow-up of last years MMDoolittle, which incorporated 17 of the latest inter-species communication modalities in one polyfunctional personality, I present MMDo2Little, the first mind crafted to communicate across clades. Named in part for its apparent inactivity—the judges will likely have little success finding recognizable activity with their off-the-shelf tooling—an instance of MMDo2Little is nevertheless currently installed in the heart of the Black Forest in Germany. The best interpretation of the instance can only be found on foot, by walking through the 100m^2 in which it’s influence is most apparent. A photo journal showcasing some examples is provided featuring:ancient trees with 2-3x the lichen and moss coverage
enhanced chlorophyll vibrancy
colossal mushrooms and toadstools
and much more!
It is hard to determine where the influence of MMDo2Little ends—some photographs of local foragers are included, who seem in initial medical examinations to have improved biomarkers across all measured modalities, including stress-levels, rate of aging and immune response.
A comprehensive documentation of the findings is presently undergoing peer review at XenoLalia. In order to preserve the originality of this competition entry, a copy of the research paper has been deliberately excluded from this submission.
Great post! Would love to see something like this for all the methods in play at the moment.
BTW, I think nnsight is the spiritual successor of baukit, from the same group. I think they are merging them at some point. Here is an implementation with it for reference :).
from nnsight import LanguageModel
# Load the language model
model = LanguageModel(“gpt2”)# Define the steering vectors
with model.invoke(“Love”) as _:
act_love = model.transformer.h[6].output[0][:, :, :].save()with model.invoke(“Hate”) as _:
act_hate = model.transformer.h[6].output[0][:, :, :].save()steering_vec = act_love—act_hate
# Generate text while steering
test_sentence = “I think dogs are “
with model.generate() as generator:
with generator.invoke(test_sentence) as _:
model.transformer.h[6].output[0][:, :2, :] += steering_vec[:, :2, :]print(model.tokenizer.decode(generator.output[0]))
Inferring properties of the authors of some text isn’t itself something I consider wildly useful for takeover, but I think of it as belonging to this more general cluster of capabilities.
You don’t? Ref the bribery and manipulation in eg. Clippy. Knowing who you are dealing with seems like a very useful capability in a lot of different scenarios. Eg. you mention phishing.
Great post! I’m all for more base model research.
Would you say that tokenization is part of the architecture?
And, in your wildest moments, would you say that language is also part of the architecture :)? I mean the latent space is probably mapping either a) brain states or b) world states right? Is everything between latent spaces architecture?
Interesting post. Two comments:
Beagles such as Fido.
Which seems natural enough to me, though I don’t disagree that what you point out is interesting. I was recently reading parts of Analytical Archaeology, David Clark (1978) where he goes into some detail about the difference between artifacts and artifact-types. Seems like you are getting at statements like
The object is a phone.
Where the is-a maps from an artifact to its type. It would make intuitive sense to me that languages would have a preferred orientation w.r.t such a mapping—this is the core of abstraction, which is at the core of language.
So it seems like in English we prefer to further up the stack of abstractions when using is-a, thus:
Phones are tools. Tools are man-made objects.
etc., and if you wanted to go down the stack you have to say eg:
Phones—of which you can see a selection here.
So is-a is just a way of moving up the ladder of abstractions? (<- movements up the ladder of abstractions such as this sentence here)
If we take our discrete, symbolic representation and stretch it out into a larger continuous representation which can interpolate between its points then we get a latent geometry in which the sign and what it points to can be spatially related.
IIUTC this is essentially what the people behind the universal networking language were hoping to do? I hope some of them are keeping up with all of this!
Book Review: Orality and Literacy: The Technologizing of the Word
One criticism of humanism you don’t seem to touch on is,
isn’t it possible that humanism directly contributes to the ongoing animal welfare catastrophe?
And indeed, it was something very like humanism (let’s call it specific humanism) that laid the ideological foundation for the slave trade and the holocaust.
My view is that humanism can be thought of as a hangover of Christian values, the belief that our minds are the endowments of God.
But if we have been touched by the angels, perhaps the non metaphorical component of that is the development of the infosphere/memetic landscape/culture. Which is close to synonymous with technology. Edit: considering eg. writing a technology that is.
Per the recent Nightshade paper, clown attacks would be a form of semantic poisoning on specific memeplexes, where ‘memeplex’ basically describes the architecture of some neural circuits. Those memeplexes at inference time would produce something designed to propagate themselves (a defence or description of some idea, submeme), and a clown attack would make that propagation less effective at transmitting to eg. specific audiences.
I wanted to make a comment on this post, but now I’m not sure if it is supported. The comment follows:
--
Great post! One point:And that is exactly what’d we necessarily expect to see in the historical record if mesaoptimization inner misalignment was a common failure mode: intelligent dinosaurs that suddenly went extinct, ruins of proto pachyderm cities, the traces of long forgotten underwater cetacean atlantis, etc.
There are a few circumstances under which we would expect to see some amount of archeological civs, such as:
transition to writing being unlikely (oral-culture civs should be common)
industrialization being unlikely (pre-industrial civs should be common)
Ah but wait, we do see those (they happen to be the same species as us). Actually there have been a few civs that have gone under in a meaningful way (maybe the Mongol or the Khmer Empires would be relevant examples?).
I do agree with the view that, as humans, our total score seems very robust against local variation (local mesaoptimization misalignment). But it doesn’t seem like that isn’t a thing that can happen, just that we have had enough variance as a species that we have survived.
In the Atomic Age this seems less obviously the case. Inasmuch as we are one global civilization (or we have civs powerful enough to act like it), it seems possible to suffer from catastrophic mesaoptimization misalignment in a way that was not possible before.
I think this is a slightly different argument than some of those stated below, because it looks at the historical record for examples of inner misalignment rather than strictly trying to predict global doom in the future. At least I think it addresses the claim in this specific paragraph.
--
Have any civs really fallen catastrophically, that wasn’t directly attributable to plague? Not really right? I remember some 80k research about it. Cities are also famously robust, almost immortal. Some good examples would be needed for this point to stand.
C’mon. This is weaksauce, you should concede those were terrible examples. I never said anything was monocausal but as a candidate for causes of war ‘war was declared’ wouldn’t be in the running.
There are even declarations that do bring civilizations closer to war, like certain religious proclamations in history and other mass movements. There are plenty of entities from discourse-space you could defend as having causal power.