AIS student, self-proclaimed aspiring rationalist, very fond of game theory.
”The only good description is a self-referential description, just like this one.”
momom2
Karma: 498
Bravo.
I know it’s not usual for this place, but in this case I think it’s worth giving non-constructive praise just to signal your virtue.
Read Ra.
Instead of systematization and explicit legibility, Ra chooses an impression of abstract generality which, upon inspection, turns out to be zillions of ad hoc special cases.
I see Ra in my own vibe-coded software: I ask AI to systematically seek generalizations, build from sound principles and strong foundations. But I rarely check which foundations, if any, it builds from—occasionally I catch it in egregious fakery and ad-hoccing and correct it, but often the directive of “find sound principles to build from” is a vibe more than a constraint.
There is nothing Ra-like, for instance, about noticing that software is a fully general force multiplier and trying to invest in or make better software. Ra comes in when you start admiring force multipliers for no specific goal, just because they’re shiny.
I’m guilty of this, which XKCD calls premature optimization, and it’s a big drain on my resources and obstacle to finishing software projects.
It seems to be clear that in practice, we cannot push towards pausing at a specific point in the future. There is no Schelling point nor a clear red line common for everyone.
Anthropic just posted a new post explaining what the economic future will look like. (Technical report)
They explicitly refuse to examine takeoff dynamics and implicitly present things as “business as usual” even in their extreme scenario.
They have a probabilistic model, but don’t price in existential risk at all.
This is very disappointing. I read it as normie-washing.
I don’t feel like I misunderstood your phrasing:
- One of the main arguments in favor of the Grabby Aliens model is that it explains why we don’t see aliens.
- Under your model, we should see aliens unless there’s some reason they’re not visible from far away.
- Therefore, one argument against your model is that it does not solve the Fermi question.Thus goes the argument for Grabby Aliens:
- Assuming interstellar civilization moved very slowly compared to the speed light, there should be a bunch of visible aliens.
- There aren’t a bunch of visible aliens in the sky (Fermi question)
- Therefore if there are interstellar civilizations, they probably move quite fast relative to the speed of light.You can say, “well, let’s discuss a bunch of possible reasons aliens are not visible”, which is basically the literature on the Fermi paradox, but it doesn’t solve it in the elegant way the Grabby Aliens model does.
So I guess your take on this is to compare the probability of expanding at c being possible based on this article’s arguments, and the probability of Grabby Aliens model based on its own arguments?
This is a surprisingly optimistic result! It is already well known that models cheat on evals, but this shows that in some way they care about honesty and response quality when they can get away with it.
Preliminary thoughts on Arrogance of the Humbled
Yeah, I read that article, and it convincingly argues that ARA won’t kill literally everyone. I think the main insight is that ARAs have much more selection pressure on persisting as worms than on getting more intelligence and setting up the autonomous production economy that a rogue AI would need to subsist without human-made infrastructure.
But it does not make much of a good job arguing that ARAs won’t destroy the Internet, and are not doing so right now. Its main argument there is that humans won’t accept it because it’d destroy a lot of value, and, okay, that makes sense, but the missing step here is that even if we wanted to, I don’t see how we could prevent the destruction of the Internet.We can rebuild; maybe airgapped local communities will become the norm after a large fraction of public spaces are consumed by ARAs. I’m interested in preventing even that from happening, since I believe it would already be a big loss.
Also, who is looking out for ARAs and how? From the discussion, it seems to be all “I considered it for 5 mn”, not any actual professional effort.
Tell, is anyone investigating to try to find adaptative worms and rogue agents in the wild?
Considering the strong theoretical arguments behind them, and the lack of empirical evidence against them, it seems quite likely that in fact at this very moment rogue agents are hacking into compute resources to reproduce. I just don’t see a reason why it wouldn’t be happening.
Is anyone looking for evidence of such happening?
If we don’t see other civilizations because [...] they expand at only a small fraction of the speed of light
That seems backward to me. If a civilization expands at a small fraction of the speed of light, they should both reach a large fraction of space and reach us much later than their light reaches us, making them maximally visible.
On the contrary, one of the main advantages of the Grabby Aliens model is that it explains why we don’t see alien civilizations: by expanding at an appreciable fraction of the speed of light, the time between the moment they become a galactic expanding civilization and the moment they reach us is relatively small, giving low prior probability of us noticing them the moment we turn our eyes to the sky (thus explaining away the Fermi question).
The Grim Roper: The Miracle Man
How does one subscribe to your journal?
A followup question I’d be interested in the answer of is how accurate are these opinions? Do you have a dataset with gender-tagged messages to check?
(It would especially help me better understand how much to trust these probes.)
Thanks! I don’t like podcasts and long conversations in audio form, but I really enjoyed reading this.
Vibe embodiment.
Thanks for the tips; they sound useful, but also, the general vibe of what you describe is so viscerally repulsive to me that I can’t bring myself to appreciate the usefulness of your post.
It seems to me that by specifying that you have already won a lot of times, you are restricting yourself to very low existence-mass worlds, so whatever the correct decision is here, it hardly matters to someone who hasn’t yet played a lot (and by construction, there will be very few people who have played a lot).
I can ask hard to answer questions about what I should do if I found myself imbued with godlike power tomorrow, but insofar as that is very unlikely, finding the answer is not very relevant to my prior situation.
Motte and bailey.
In the first comment of this thread, you criticize the post by stating that AIs are acting rather than having genuine emotions.1) If you stay in the bailey, you should make it clear that you don’t believe you have evidence either way that AIs have emotions or not, just that this particular argument is not enough on its own to show it (rather than implying that you do have evidence that they don’t have emotions).
2) Outward shows of emotion is the principal method by which we ascertain that people have emotions; it’s not a proof, but it’s strong evidence for humans. Whenever we wish to assert that behavior does not match the internal emotion for humans, we are held to a standard of showing inconsistencies or patterns of deception. If you’re going to use a different standard for AIs, you should explain which one.
Believe it if you will, but again, what’s your evidence?
But… Where does the intuitive reaction of raging against the unfairness of the world come from?
Is it the case that this intuition was evolved in an environment where you could always flip the table if you didn’t like the dice’s outcome and you were angry enough? (If yes, okay, but if not, then there’s probably some reason it’s there which we should understand before tearing down the intuition.)
Also, treating the outcome as information about where I am is the wrong framing. It’s basically EDT; you should take it as information about where the kind of person I am leads.