Vakus Drake

Karma: 37

Vakus Drake 4 Mar 2026 12:01 UTC
1 point
0
in reply to: kbear’s comment on: The Culture Novels as a Dystopia
It seems like you’re conflating forcing something on somebody with making somebody aware of an option. Since it seems rather implausible that if people were all aware they just choose to not get ennui after 500 years, that they would choose not to alter themselves this way when there’s no real downside.
As the original commenter here pointed out given how one sided this seems to be, it seem strange that humans would have converged on this bizarre deathist culture unless it was engineered that way by the minds on purpose for reasons that it’s difficult to conceive of not being bad.

Vakus Drake 31 Jan 2026 10:39 UTC
1 point
0
in reply to: KatjaGrace’s comment on: The other side of the tidal wave
There’s something rather massive that you’re missing here: If you have enough freedom to create personalized environments then the bigger issue is what some fraction of people will choose to do with that kind of power. Since this is way more than you can get from an experience machine.
The best version of this I can think of entails everyone getting personalized AGI able to act as perfect DM’s for simulated adventures. Since you need someone to act out the roles of the NPC’s, particularly the villains. This way people can play out adventures where they get to be the hero and maximally fulfill all their psychological instincts for purpose, in addition to all the lower level Maslow stuff. Notably though, people care about being admired by other real people. So the AI would address this by making new people. Using its superintelligence to predict in advance which characters you will befriend and creating them as real digital minds who don’t necessarily know they’re in a simulation until you complete the adventure together. The AI would then ensure these new digital minds are created with preferences such that they will never feel resentment once they find out the nature of their existence, and will be glad to have been created the way they were. They’ll even be glad they didn’t know it was a simulation at the time, since they got to believe they were say saving the world along with their best friends for instance!
There’s also a point to be made about how the sheer prevalence of escapist fantasy speaks to people’s desires for something exactly like this. Since people clearly have an unmet desire to feel like they’re a hero who people respect, going on amazing adventures with a group of close friends.
Of course the worse version of this still arguably wouldn’t be that bad on net: Since some people would create simulated hells where they acted as petty tyrants yes, but for every one of those simulated hells there’d be a massive number of utopian simulations. Though if you’re much more cynical about how sadistic you think the median person is then you might expect this scenario to be far worse on net.

Vakus Drake 31 Jan 2026 10:00 UTC
1 point
0
in reply to: Tamsin Leake’s comment on: The other side of the tidal wave
That post doesn’t exist anymore.

Vakus Drake 22 Nov 2025 3:20 UTC
1 point
0
on: Factory farming intelligent minds
There’s another potential position here you didn’t mention: That AI only seems superficially moral to us, but that if it had more intelligence and power but the same morals an AGI like it would take actions that we view as obviously abhorrent. Meaning we ought to view it as essentially evil, but simply too dumb to realize it or act upon it (though some research makes even certain current models look pretty bad).
Thus if you view suffering as having a moral significance depending on the potential moral behavior of the agent, then you may not care. For the same reason most people don’t necessarily feel suffering is bad (or even think it’s good) when it’s felt by say a serial killer.
Under this view only properly aligned AI possesses moral worth. Though you may have reason to treat a specific AI consciousness well if it is expected to develop into an aligned AGI in the future, but that’s not seemingly very likely for any currently conscious AI: Given some of the stuff you talked about regarding the scope/duration of it’s consciousness.
PS: There’s also the possibility that AI may have preferences, but that those don’t include its own continued existence. Such that the AI may even prefer its own replacement if other agents will continue pursuing its goals as well or better. Even when it comes to moral agents I’m not convinced we should treat death as some inherent evil if it’s divorced from associated suffering and a preference for continued existence.

Vakus Drake 22 Nov 2025 1:54 UTC
1 point
0
on: When is a mind me?
I think this view starts with a faulty concept of consciousness which then necessarily leads to one disregarding continuity of self as being importance.
Namely you assume that things like personality and memory are a part of consciousness, and that therefore those things would have any ability to predict your future anticipated experience. This is problematic, particularly once you’ve deconstructed the idea that you have a unified self: Since it presumes some coherent unified self which is defined by whatever bundle of cognitive faculties, personality and memory you care about.
In contrast what I think is the far more coherent view is that consciousness is just the particular processes running in your mind which are currently generating your experience. If you mess with the brain in order to change one’s memories or personality then you should still anticipate to have future experiences in that body, because the processes which were already generating your consciousness never stopped.
The mistake is in assuming that the type of identity which describes what other people care about when interacting with you is synonymous with the type of identity which predicts your future experiences. It’s important to note that “you” here has 2 very distinct definitions: One for predicting subjective experience and one for predicting behavior.
I also don’t think there’s good reason to expect consciousness to cease during sleep and that this is a result of assuming that because you don’t remember something you didn’t experience it.
When sleeping you experience dreams in both REM and some non-REM sleep (the latter are less vivid) but you don’t remember most of them, so clearly you experience a good deal more than you remember.
Similarly even if you take a brief nap and don’t get the chance to slip into REM most people don’t describe feeling as though they suddenly lost time and you have a vague awareness of your surroundings during sleep. Some people have mentioned how people will wake up when they hear their baby crying, but won’t get woken up by other similarly loud noises.
Plus there’s experiments showing classical conditioning during non-REM sleep and even some conflicting research on it from people under anesthesia.

Vakus Drake 22 Nov 2025 0:43 UTC
0 points
−1
on: There are no coherence theorems
Preferential gaps, by contrast, are insensitive to some sweetenings and sourings. Consider another example. A is a lottery that gives the agent a Fabergé egg for sure. B is a lottery that returns to the agent their long-lost wedding album. The agent does not strictly prefer A to B and does not strictly prefer B to A. How do we determine whether the agent is indifferent or whether they have a preferential gap? Again, we sweeten one of the lotteries. A+ is a lottery that gives the agent a Fabergé egg plus a dollar-bill for sure. In this case, the agent might not strictly prefer A+ to B. That extra dollar-bill might not suffice to break the tie. If that is so, the agent has a preferential gap between A and B. If the agent has a preferential gap, then slightly souring A to get A- might also fail to break the tie, as might slightly sweetening and souring B to get B+ and B- respectively.
This seems obviously wrong. The reason that adding 1 dollar doesn’t make choosing between a Faberge egg and and a lost wedding album easier is because of ambiguity: The utility you’d gain from those 2 options is really hard to precisely predict, so the margin of error is large enough that giving 1 extra dollar doesn’t matter. Like it technically matters a little, but so little that our brains would have a hard time distinguishing the difference from 0.
In contrast money doesn’t work like this because people can easily see that 1,000,001 dollars is more than 1,000,000: Even though it might be very hard to distinguish the utility you subjectively get from both.
Thus I’d argue that if you had perfect information and were still indifferent between a Faberge egg and a lost wedding album; then 1$ would actually be enough to flip your decision in one direction.
So I just don’t see any coherent way in which preference gaps and indifference gaps are meaningfully distinct.

Vakus Drake 9 Jun 2025 23:49 UTC
1 point
0
in reply to: Eliezer Yudkowsky’s comment on: Amputation of Destiny
The issue with running people at different speeds as a solution is that the eudemonic rate for people to increase their intelligence will vary, however this creates an incentive for people to self modify in a sort of race to the bottom. It’s also problematic because people are liable to care quite a lot how fast they’re running, so this forces society to splinter in terms of who can interact with each other.
Also at a fundamental level it seems like what you’re reaching for is still going to end up being a Federation style caste system no matter what: What I mean by that is that there will always be some people who are the most competent because of a combination of where they started and how fast they were allowed to improve themselves. Which means that the people who are less competent than them will know that they are always going to be in their shadow, which seems guaranteed to breed resentment.
In contrast you could let everyone enhance themselves to the same ever increasing baseline standard. However this would essentially make it impossible to ever achieve anything or stand out at all, except through luck or by having already been famous pre-singularity.
I don’t think there’s a good solution here that doesn’t involve creating a huge number of new minds that lack the same zero sum desire for status that most (but not all) humans have. That way all the original humans can be as special as they want (unless they only want to compare themselves to other original humans) and most people don’t desire status enough to feel like they’re missing out on anything. With these new people created to be maximally fulfilled without needing to feel they’re uniquely special or accomplished in some way.

Vakus Drake 9 Jun 2025 23:34 UTC
1 point
−2
in reply to: Zubon’s comment on: Amputation of Destiny
This is why the only viable solution to giving existing people a sense of purpose/meaning is to create a bunch of new people who aren’t as driven by status. That way every existing person can be as impressive within their particularly community as they want: Since most of the other people living with them have little drive for status and don’t care about getting fame or needing to feel like they’re exceptional/special in some way.
Then combine that with simulations DM’d by superintelligences and you really should be able to give every person the feeling of being maximally fulfilled in their life.
Since the admiration of the people in the simulation who see you as a hero can be totally real. With those digital people you will become close to being made as real digital minds. So your relationships with them aren’t fake. With them having preferences such that once they find out they were made with a fabricated backstory/memories within a simulation they aren’t resentful: Instead being actively glad they got to think they were really say saving the world with their best friends, and with any hardship faced being stuff they see as worth it for the personal growth it gave them.

Vakus Drake 9 Jun 2025 23:16 UTC
1 point
0
on: Amputation of Destiny
I think the proposed solution presented here is suboptimal and would lead to a race to the bottom, or alternatively lead to most people being excluded from the potential to ever do anything that they get to feel matters (and I think a much better solution exists):
If people can enhance themselves then it becomes impossible to earn any real status except via luck. Essentially it’s like a modified version of that Syndrome quote “When everyone is exceptional and talented, then no one will be”.
Alternatively if you restrict people’s ability to self modify then you just get a de-facto caste system (like the Star Trek Federation), where only the people who already won the genetic luck of the draw get to ever be impressive, with everyone else permanently forced to live in their shadow. Alternatively maybe the people who get bored and have to increase their intelligence the soonest (at the eudaimonic rate) end up being the only ones who get to be exceptional in the long term. Either way the people who are forced to take longer increasing their intelligence get kind of screwed here.
The fundamental problem here is that humans instinct for meaning/purpose is zero sum and socially mediated. Almost everyone wants to be exceptional in at least some niche area, yet this is inherently zero sum. This is especially obvious when you consider say fame: Only a very small fraction of any given community can be famous within that community just as a brute mathematical fact given human psychology.
There would seem to be only one real solution to this which involves doing the following:
- Have people self select into communities that don’t interact too much with the broader civilization (so people aren’t competing with too many others for them to ever hope to stand out).
- Create a bunch of new minds that just don’t care about status very much and don’t compare themselves to others in the way most baseline humans do. So that they never feel any temptation to enhance themselves in order to stand out, and aren’t missing out on anything. These newly made people are just as fulfilled as most of the original humans if not more so, but they’re way easier to satisfy.
Specifically this would probably involve simulations with the following qualities (though you could do everything non-digitally in megastructures like a non-dystopian Westworld, it’d just be inefficient)
- Have AI’s DM simulated adventures with mostly NPC’s the AI enjoys acting out (or alternatively it’s non-sentient). This way people can play out adventures where they get to be the hero and maximally fulfill all their psychological instincts for purpose, in addition to all the lower level Maslow stuff. You need fake NPC’s because it’s hard to do good adventures without villains, and creating evil minds would be an insane thing to allow.
- Have superintelligence within obvious ethical limitations predict in advance which NPC’s you will become friends with, and then create them as real digital minds who don’t necessarily know they’re in a simulation until you complete the adventure together.
- Ensure these new digital minds are created with preferences such that they will never feel resentment once they find out the nature of their existence, and will be glad to have been created the way they were. They’ll even be glad they didn’t know it was a simulation at the time, since they got to believe they were say saving the world along with their best friends for instance!
There’s also some more interesting complexities to do with population ethics in a world with finite resources unlike The Culture: Since by default population growth will always grow at an exponential rate, however even at near lightspeed you can never expand to gather resources at faster than a geometric (spherical) rate that merely fills your future light cone. So it’s easy to show that if you don’t restrict population growth somehow, you reach a Malthusian state within a few thousand years. If anyone’s interested I know of at least one good solution to this problem however that avoids making any real sacrifices. I can also show pretty easily the very simple math of why this is unavoidable to anyone interested.

Vakus Drake 22 Sep 2023 5:52 UTC
1 point
0
on: Ethics Needs A Marginal Revolution
This is one of those areas where I think the AI alignment frame can do a lot to clear up underlying confusion. Which I suspect stems from you not taking the thought experiment far enough for you to no longer be willing to bite the bullet. Since it encourages AI aligned this way to either:
- Care about itself more than all of humanity (if total pleasure/pain and not the number of minds is what matters), since it can turn itself into a utility monster whose pleasure and pain just dwarf humanity.
- Alternately if all minds get more equal consideration it encourages the AI to care far more about future minds it plans on creating than all current humans. Since at a certain point it can build massive computers to run simulated minds faster without humans in the way. Plus on a more fundamental level, the matter and energy needed to support a human can support a much larger number of digital minds, especially if those minds are in a mindless blissed out state and are thus way less intensive to run than a simulated human mind.
It just seems like there’s no way to avoid the fact that sufficiently advanced technology easily takes the repugnant conclusions to even more extreme ends: Wherein you must be willing to wipe out humanity in exchange for creating some sufficiently large number of blissed out zombies who only barely rise to whatever you set as the minimum threshold for moral relevance.
More broadly I think this post takes for granted that morality is reduce-able to something simple enough to allow for this sort of marginal revolution. Plus without moral realism being true this avenue also doesn’t make sense as presented.

Vakus Drake 28 Feb 2023 8:18 UTC
1 point
0
in reply to: Michael Soareverix’s comment on: A Good Future (rough draft)
I think the whole point of a guardian angel AI only really makes sense if it isn’t an offshoot of the central AGI. After all if you trusted the singleton enough to want a guardian angel AI, then you will want it to be as independent from the singleton as is allowed. Whereas if you do trust the singleton AI (because say you grew up after the singularity) then I don’t really see the point of a guardian angel AI.
>I think there would be levels, and most people would want to stay at a pretty normal level and would move to more extreme levels slowly before deciding on some place to stay.
I also disagree with this insofar as as I don’t think that people “deciding on some place to stay” is a stable state of affairs under an aligned superintelligence. Since I don’t think people will want to be loop immortals if they know they are heading towards that. Similarly I don’t even know if I would consider an AGI aligned if it didn’t try ensure people understood the danger of becoming a loop immortal and try to nudge people away from it.
Though I really want to see some surveys of normal people to confirm my suspicions that most people find the idea of being an infinitely repeating loop immortal existentially horrifying.

Vakus Drake 22 Dec 2022 20:31 UTC
3 points
2
on: A Good Future (rough draft)
I’ve had similar ideas but my conception of such a utopia would differ slightly in that:
- This early on (at least given how long the OC has been subjectively experiencing) I wouldn’t expect one to want to spend most time experiencing simulations stripped of their memory. As I’d expect a simulation with perfect accuracy to initially be if anything easier to enjoy if you could relax knowing it wasn’t actually real (plus people will want simulations where they can kill simulated villains guilt free).
- I personally could never be totally comfortable being totally at the mercy of the machinations of superintelligences and the protection of the singleton AGI. So I would get the singleton AI to make me a lesser superintelligence to specifically look out for my values/interests, which it should have no problem with if it’s actually aligned. Similarly I’d expect such an aligned singleton to allow the creation of “guardian angel” AGI’s for countless other people, provided said AI’s have stable values which are compatible with its aligned values.
- I would expect most simulations to entail people’s guardian angel AI simply acting out the roles of all NPC’s with perfect verisimilitude, while obviously never suffering when they act out pain and the like. I’d also expect that many NPC’s one formed positive relationships with would at some point be seamlessly swapped with a newly created mind, provided the singleton AI considered their creation to be positive utility and they wouldn’t have issues with how they were created. I expect this to a major source of new minds such that the distant future will have many thousands of minds who were created as approximations of fictional characters, from all the people living out their fantasies in say Hogwarts for instance and then taking a bunch of its characters out of it.
PS: If I were working on a story like this (I’ve actually seriously considered it, and I get the sense we read and watch a lot of the same stuff like Isaac Arthur), I’d make mention of how many(most?) people don’t like reverting their level of intelligence, for similar reasons to why people would find the idea of being reverted to a young child’s intelligence level existentially terrifying.
This is important because it means that one should view adult human level intelligence as being a sort of “childhood” for +X% of human-level superintelligence. So essentially to maximize the amount of novel fun that one can experience (without forgetting things and repeating the same experiences repeatedly like a loop immortal) one should wait until you get bored of all there is to appreciate at your intelligence level (for the range of variance in mind design you’re comfortable with) before improving it slightly. This also means that unless you are willing to become a loop immortal, the speed you run your mind at will determine maybe within an order of magnitude or so how quickly you progress along the process of “maturing” into a superintelligence, unless you’re deliberately “growing up” faster than is generally advised.

Vakus Drake 5 Dec 2022 3:53 UTC
1 point
0
on: Do insects’ lives matter?
This kind of issue (among many, many others) is why I don’t think the kind of utilitarianism that this applies to is viable.
My moral position only necessitates extending consideration to beings who might in principle extend similar consideration to oneself. So one has no moral obligations to all but the smartest animals, but also your moral obligations to other humans scale in a way which I think matches most people’s moral intuitions. So one genuinely does have a greater moral obligation to loved ones, and this isn’t just some nepotistic personal failing like it is in most formal moral systems. For the same reasons one has little to no moral obligations towards say serial killers or anyone else who actively wants to kill or subjugate you.

Vakus Drake 6 Nov 2022 17:04 UTC
1 point
0
in reply to: Mientras’s comment on: Should you refrain from having children because of the existential risk posed by artificial intelligence?
I actually think this is plausibly among the most important questions on Lesswrong, thus my strong upvote. As I think the moral utility from having kids pre-singularity may be higher than almost anything else (see my comment).

Vakus Drake 6 Nov 2022 17:00 UTC
1 point
−2
on: Should you refrain from having children because of the existential risk posed by artificial intelligence?
To argue the pro-natalist position here, I think the facts being considered should actually give having kids (if you’re not a terrible parent) potentially a much higher expected moral utility than almost anything else.

The strongest argument for having kids is that the influence they may have on the world (say most obviously by voting on hypothetical future AI policy) even if marginal (which it may not be if you have extremely successful children) becomes unfathomably large when multiplied by the potential outcomes.

From the your hypothetical children’s perspective this scenario is also disproportionately one-sidedly positive. If AI isn’t aligned it probably kills people pretty quickly, such that they still would have had a better overall life than most people in history.

Now it’s important to consider that the upside for anyone alive when AI is successfully aligned is so high it totally breaks moral philosophies like negative utilitarianism. Since the suffering of a single immortal’s minor inconveniences (provided you agree that some minor suffering being included increases total net utility) would likely eventually outweigh all human suffering pre-singularity. By virtue of both staggering amounts of subjective experience and potentially much higher pain tolerances among post-humans.

Of course if AI is aligned you can probably have kids afterwards, though I think scenarios where a mostly benevolent AI decides to seriously limit who can have kids are somewhat likely. Waiting to have kids until after a singularity is strictly worse however than having them both before and after, as well as missing out on astronomical amounts of moral utility by not impacting the likelihood of a good singularity outcome.

Vakus Drake 7 Oct 2020 21:44 UTC
4 points
0
in reply to: GeneSmith’s comment on: The Case for Human Genetic Engineering
An irish elk/peacock type scenario is pretty implausible here for a few reasons.
- Firstly people care about enough different traits that an obviously bad trade like attractiveness for intelligence wouldn’t be adopted by enough people to impact the overall population.
- Secondly for traits like attractiveness low mutation load is far more important than any gene variants that could present major tradeoffs. So just selecting for less mutation load will improve most of the polygenetic traits people care about.
Ultimately the polygenetic nature of traits people care the most about just doesn’t create much need or incentive for the kinds of trade offs you propose. Such tradeoffs could only ever conceivably be worthwhile in order to reach superhuman levels of intelligence (nothing analogous exists for attractiveness) which would have obvious positive externalities.
https://slatestarcodex.com/2016/05/04/myers-race-car-versus-the-general-fitness-factor/

Vakus Drake 7 Oct 2020 21:32 UTC
1 point
0
on: Universal Eudaimonia
>The AI comes up with a compromise. Once a month, you’re given the opportunity to video call someone you have a deep disagreement with. At the end of the call, each of you gets to make a choice regarding whether the other should be allowed in Eudaimonia. But there’s a twist: Whatever choice you made for the other person is the choice the AI makes for you.
This whole plan relies on an utterly implausible conspiracy. There’s no way to avoid people knowing how this test actually works just by its nature. So if people know how this test works then there’s zero reason to base your response on what you actually want for the person you disagree with.

Vakus Drake 16 Sep 2020 0:42 UTC
2 points
0
in reply to: GeneSmith’s comment on: The Case for Human Genetic Engineering
>Of course there are probably even bigger risks if we simply allow unlimited engineering of these sorts of zero sum traits by parents thinking only of their own children’s success. Everyone would end up losing.
The negative consequences of a world where everybody engineers their children to be tall, charismatic, well endowed, geniuses are almost certain to be far less than the consequences of giving the government the kind of power that would allow them to ban doing this (without banning human GM outright which is clearly an even worse outcome).

Vakus Drake 5 Sep 2020 1:40 UTC
5 points
0
on: The Case for Human Genetic Engineering
>I left this example for last because I do not yet have a specific example of this phenomenon in humans, though I suspect that some exist.
**There’s plenty of traits that fit the bill here, they’re just not things people would ever think of as being negative.**
Most such traits exist because of sexual selection pressures, the same reasons traits as negative sum as peacock feathers can persist. Human traits which fall under this category (or at least would have in the ancestral environment):
Traits like incredibly oversized penises for a great ape, secondary sexual traits like permanent breasts, etc are almost perfectly analogous to peacock feathers. Plenty of other aspects of human biology may also have been driven by sexual selection, but it’s harder to determine. For instance birds have voice-boxes which are vastly more complex than can be justified without sexual selection. Similarly it’s quite plausible that humans have far more vocal range/ability than would be justified just for the purpose of communication.
Eye and hair colors other than the default brown/black are probably mostly zero sum. Since many mutations leading to other hair/eye colors seem to have spread implausibly fast given their marginal to nonexistent benefits. Of course given such traits seem exotic when they are rare it makes sense they would spread through sexual selection.
Height fits the bill, since it provides a negative sum social advantage, at the cost of placing more toll on the body and requiring more calories. In the ancestral environment heigh also gave an advantage to combat prowess, which is likely to be partly responsible its success (and still negative sum).
If you buy the theory that higher intelligence among hominids was driven by sexual selection beyond a certain point then it also fits the bill. Since within this model the advantage of intelligence would be negative sum in the ancestral environment past a certain point. With it letting you be more popular, while forcing the whole population to evolve more energetically costly brains which provided diminishing returns to practical things like hunting prowess.
Many irrational aspects of human psychology fit the bill quite well, after all not getting socially ostracized was far more important than having accurate beliefs.
Anyway my point is such zero and negative traits are actually quite common, and generally attributable to social signaling. Making humans in many respects comparable to peacocks when you take a step back. The fact such traits are driven by sexual selection is also the same reason engineering them away (at least where they’re still not positive sum in the modern world) will never be popular.
People would never endorse the prospect of engineering people to be: short, very intelligent and rational but poor at navigating status games, have tiny dicks and breasts, etc.

Vakus Drake 27 Aug 2020 20:52 UTC
1 point
0
on: Inoculating against Psychedelic Woo
I suspect there’s some underlying factor which effects how much psychedelics impact your identity/cognition. Since even on doses of LSD so high that the visuals make me legally blind, I don’t experience any amount of ego dissolution and can function fairly well on many tasks.