Howdy y’all, my nome de plume ’round these parts is onslaught.
I’m really into futurism, GCR mitigation, science for good, broad moral circles, and ~consequentialist utilitarianism. I think building superintelligent systems is an inherently unsafe endeavor.
Naturally, there are also a lot of other causes and interests that are dear to me. Consciousness, wild animal welfare, prisons and modern slavery, aliens, radical political change, human and animal intelligence augmentation, startups/entrepreneurship, airships, pirates, etc.
onslaught
Karma: 4
I think this puts a lot of weight on a bespoke definition of “interesting” and that kind of obscures what you’re saying. I feel similar about your use of the concept of creativity.
I think that current LLMs are extremely “creative” for many plausible definitions of that word, so I guess it doesn’t really carve things at the joints for me. Visual art, stories, plausible baby names, what sorts of recipes you can try with x ingredients. All written-ont-the-tin use cases for these things.
I do not believe that LLMs think very much in the manner that we do at all. I just don’t think I would pitch that as lacking some true spark of creativity or something. It is too opaque to me what you’re saying.
“Metalhead” from Black Mirror is a relevant contemporary art piece.
I for one find Spot spooky as hell. I would go as far as to say that I have heard others express discomfort toward Boston Dynamics demo videos.
Also, sentry guns and UAVs seem like strong examples of extant scary robots. Maybe see also autonomousweaponswatch.org .
Hey, very interesting post!
I enjoyed the “become a superintelligence” framing / thought experiment. I am not sure I fully believe the “armed balance of power” b/w Superintelligences part as stated, but I can imagine something similar I might agree with.
Ultra charged human intelligence and IA are both interesting areas to me. I think writing and database search are both interesting examples of IA information technologies to use as a foil to a world of AI servants.
I couldn’t really understand what Crystallect was doing/going for, but I liked your discussion of programming. I wish you all the best with it. I am reminded of Conjecture because of the IA and custom programming language design. It is different though; they have an attempt going to make a special programming language to harness LLM intelligence in a much more “code like” and “human accessible” way.
But who night-watches the night-watchman?
Thank you for providing a more concrete vision of a sort of “light touch good guy singleton AI” world.
I agree that “AI for peace” is broadly good. I also believe that we should collectively prioritize existential security and that this is maybe one of the most noble uses of “power” and “advanced AI”.
However, I still have some serious hangups with this plan. For one thing, while I understand the desire to kick certain questions of cosmic importance down the road to a more reflective process than oneself, I do not really think that’s what this plan buys you.
Basically, I would push back against the premise that this shouldn’t mostly be considered a form of mass disempowerment and lock-in. Cf. Joe Carlsmith:Classic problem with, “Ooh, let’s have more top-down control,” is like you love it when it’s the perfect way, but then it’s not the perfect way and then maybe it’s worse. People discover this in government.
Even with noble goals in mind, I don’t think I agree that it is necessarily noble or legitimate to opt for the “defer all power to a single machine superintelligence” strategy. It feels too totalizing and unipolar. It totally gives me “what if we had the state control all aspects of life in order to make things fair and good” vibes.
I can scarcely imagine the progress that would have to be made in the field of AI before I would be on board with governments buying into this sort of scheme on purpose and giving their “monopoly on violence” to OpenAI’s latest hyper-competent VLM-GPT Agent and zer army of slaughterbots?
I think the machine singleton overlord thing is just too scary in general. I agree that this is directionally preferable to a paper clipper or even a more explicitly unipolar jingoist AmericaBot or some kind of Super-Engaging Meta-Bot hell or something. Still, I struggle to buy the “it’s in charge of the world, but also really chill” duality. I don’t see how this could actually be light touch in the important ways in practice.
Also:Outer alignment / alignment as more than just perfect instruction-tuning
“Assuming alignment” is a big premise in the first place if we are also talking about “outer alignment” being solved too. I think I understand the ontology in which you are asking that question, but I want to flag it as an area where I maybe object to the framing.
There is a sense in which “having aligned the AI” would imply not just “ultimate intelligence power, but you control it” but also some broader sense in which you have figured out how to not just getting monkey-pawed all the time. Like, sometimes the mantle of “alignment” can mean more like “faithful instruction tuning” and sometimes it can seem more like it also has something to say about “knowing which questions to ask”.
I would point to The Hidden Complexity of Wishes as a good essay in this vein.
Parts of this are technical, like for example, maybe we can get AIs to consistently tell us important facts about what they know so that we can make informed decisions on that basis (ie. that is the safety property I associate with ELK). Parts of this also seem like they run head first into larger legal and sociological questions. I know Gradual Disempowerment is an example of a paper/framework which among other things, looks at how instituional incentives change simply as a result of several types of power being vested in machines altenatives rather than humans: “growth will be untethered from a need to ensure human flourishing”. Maybe parts of this go beyond “alignment”, but other parts of this also seem to relate to the broadly construed question of “getting the machine to do what you want”, so it is unlcear to me.
I don’t mean to get so caught up arguing over the meaning of a word. Maybe alignment really should just mean “really well instruction tuned” or something. I think I hear it used this way a lot in practice anyways. It might be a crisper interpretation to talk about the problem as just the task of “prevent scheming” and “preventing explicit Waluigi-style misbehavior” (cf. Sydney, Grok sometimes, DAN, etc. where behavior clearly goes off-spec). This is a not uncommon framing re: “aligned to who?” and value orthogonality. I guess I just really want to flag that under this usage “aligned AI” is very much not interchangeable with ~”omnibenevolent good guy AI”, but instead becomes a much more narrow safety property.
Hey, thanks for engaging.
I read what I thought were the relevant excerpts in what you linked there. I hadn’t really crossed paths with you before, but you seem to have a rich ontology and lexicon when it comes to theory of mind.
I am not sure if that pinpoints the disagreement or not. We might just be talking past each other. I’ll tell you what I think creativity is and then I’ll restate my objection to your prediction.
I do think “creativity” is a useful word, just maybe not a load bearing one in my ontology.
Like, if I really like a story and it has a lot of unexpected elements that I think it uses really well, that is what I might call creative. Or anything like that if it feels novel, exciting, clever sort of thing… Maybe if someone were giving something high praise and wanted to say it is very deep and clever they could say it is “very creative”. Especially if it was artistic or novel.
Also sometimes when it is just a lot of whacky things are together even if it’s not that clever. Like, when a kid combines a lot of elements into their pretend world or story.
Ya, I know it has something to do with a minds ability to keep learning and improving. Your “trajectory of creativity” concept is about a minds ability to continue to improve beyond the minds around it. I don’t resonate with those usages as much, but I can also kind of understand where it’s coming from and how you’re using the word.
I think my original objection / pushback was partly that it feels hard to operationalize this because what you find interesting is kind of just your thing and it doesn’t seem like a meaningful proxy for intelligence or something. I guess I would add that surely some people are already impressed and interested with some math ideas that chatbots can come up with. Also, perhaps if you could visualize extremely high dimensional spaces you would think that AlphaEvolves proofs were beautiful, elegant, and crisp. I’m not saying there’s no information/signal in what you’re saying; I just found it left a lot unclear for me when I first read it I guess.
I get that LLMs clearly aren’t as good at publishing new top tier math papers or whatever. I guess, gun to my head, I would put most of that down to, like… lacking many of the cognitive abilities needed to independently execute on large scale, messy tasks independently. Or some mix of attributes like that. I would also expect them to have really bad vibes based planning abilities… And plus by the time off the shelf AIs can write math papers of a given quality tier, the goalposts for interestingness will move accordingly… Maybe there is something to the idea that they are not generally inclined towards effing the ineffable and carving structure from reality, but also I doubt they’d have trouble with eg. neologisms.