Dach

• Dach 18 Oct 2022 4:56 UTC

  12 points

  0

  on: De­ci­sion the­ory does not im­ply that we get to have nice things

  But, sure, if you’re somehow magically unhackable and very good at keeping the paperclipper boxed until you fully understand it, then there’s a chance you can trade, and you have the privilege of facing the next host of obstacles.

  Now’s your chance to figure out what the next few obstacles are without my giving you spoilers first. Feel free to post your list under spoiler tags in the comment section.

  Ideas:

  1. Someone else definitely builds and deploys an UFAI before you finish studying Clippy. (This would almost always happen?)

  2. Clippy figures out that it’s in a prisoner’s dilemma with the other cobbled-together UFAI humanity builds, wherein each UFAI is given the option to shake hands with Humanity or pass 100% of the universe to whichever UFAI Humanity eventually otherwise deploys. Clippy makes some models, does some decision theory, predicts that if it defects and handshakes other UFAIs are more likely to defect too based on their models, and decides to not trade. The multiverse contains twice as many paperclips.

  3. The fact that you’re going to forfeit half of the universe to Clippy leaks. You lose, but you get the rare novelty Game Over screen as compensation?

  Interlocutor: Well, maybe we can train the infant paperclipper in games of increasing complexity, so that it’s never quite sure whether it’s in reality yet. The paperclipper will then be uncertain about whether we humans are simulating it, and will want to cater to our preferences to some extent.

  Me: Uh, yeah, your paperclipper will be able to tell when it is finally in the real world.

  Interlocutor: What? How?

  Me: I suggest maybe spending five minutes thinking about your own answer to that question before I give mine.

  Ideas:

  1. It could just act friendly for enough time to be sure it’s not in a simulation on the grounds that a civilization that could simulate what it was doing on its computers wouldn’t simulation-fakeout it for non-exotic reasons. Imagine Clippy mulling over its galaxy-sized supercomputing cluster and being like “Hm, I’m not sure if I’m still in those crude simulations those stupid monkeys put me in or I’m in the real world.”

  2. I would be surprised if we’re able to build a simulation (before we build AGI) that I couldn’t discern as a simulation 99.99% of the time. Simulation technology just won’t advance fast enough.

• Dach 4 Dec 2021 18:50 UTC

  4 points

  on: Biol­ogy-In­spired AGI Timelines: The Trick That Never Works

  Is Humbali right that generic uncertainty about maybe being wrong, without other extra premises, should increase the entropy of one’s probability distribution over AGI, thereby moving out its median further away in time?

  Writing my response in advance of reading the answer, for fun.

  One thought is that this argument fails to give accurate updates to other people. Almost 100% of people would give AGI medians much further away than what I think is reasonable, and if this method wants to be a generally useful method for getting better guesses by recognizing your uncertainty then it needs to push them towards shorter timelines, to whatever degree I trust short timelines.

  In fact, this argument seems to only be useful for people whose AGI timelines are shorter than whatever the true timeline ends up being. If this were a real comment I would say this revealed behavior was unsurprising because the argument was generated to argue someone towards longer timelines and thus I couldn’t trust it to give reality-aligned answers.

  It strikes me that such a system probably doesn’t exist? At the very least, I don’t know how to turn my “generic uncertainty about maybe being wrong, without other extra premises” into anything. You need to actually exert intelligence, actually study the subject matter, to get better probability distributions. Suppose I have a random number generator that I think gives 0 10% of the time and 1 90% of the time. I can’t improve this without exerting my intelligence- I can’t just shift towards 50% 0 and 50% 1 with no further evidence. That would rely on the assumption that my uncertainty signals I’m biased away from the prior of 50% 0 and 50% 1, which is completely arbitrary.

  Note that if you have reason to think your guess is biased away from the prior, you can just shift in the direction of the prior. In this case, if you think you’re too confident relative to a random distribution over all years, which basically means you think your timeline is too short, you can shift in the direction of a random distribution over all years.

  In this context, you can’t get better AGI estimates by just flattening over years. You need to actually leverage intelligence to discern reality. There are no magical “be smarter” functions that take the form of flattening your probability distribution at the end.

• Dach 2 Dec 2021 20:16 UTC

  2 points

  in reply to: AnthonyC’s comment on: How com­mon are abio­ge­n­e­sis events?

  If I try to answer that question as written, I’d say that any time I see a probability estimate with on-the-order-of-hundreds of zeroes, when I know that event actually happened (at least) once in Earth’s past light cone, I’m going to assume there is an error in the model that generated the estimate, whether I know what it is or not.

  I would agree for pretty much any other topic. This is an event required for people to be around to observe it. Imagine a universe in which abiogenesis events really were absurdly rare- unlikely to ever occur in a given observable universe sized area. Every observer in this universe would still look back and see an abiogenesis event occurring in their past! Having observed exactly one event is completely required and provides no evidence. This is essentially the weak anthropic principle.

  The fact that we observe only exactly one event is thus bayesian evidence in the direction of rare abiogenesis. You bring up the point that the fact we observe only exactly one event in earth’s history isn’t that strong of evidence because e.g. events being concealed by significantly more advanced competitors. I certainly don’t disagree- that was just unfortunate overemphasis on my part.

  So what I way trying to point to is that if a catalytic cycle of many (much smaller) RNA strands was sufficient for an abiogenesis event, that could lower the probability estimate enough to make such events more likely by enough that there could have been multiple even just on Earth without straining credulity, and the world today would likely look basically the same either way since the more-competitive biochemistry would have long since reach fixation (and/​or the lineages could have merged in some analog of later endosymbiosis events).

  I was thinking of systems more complex than the random coalescence of nucleotides when I wrote my post, but I didn’t know how to productively model that. Someone else had a similar argument and I responded that I slightly shifted away from rare-abiogenesis because I wasn’t thinking of complicated groups of cross-catalyzing RNA in particular, which are dumb enough to have no modern analogues but not so dumb as to not be competitive against single-strand solutions which require the random coalescence of another 100 base pairs, or whatever it is. It’s unclear to what degree this should affect my model, given a lack of understanding of how such systems work.

• Dach 30 Nov 2021 1:41 UTC

  2 points

  in reply to: AnthonyC’s comment on: How com­mon are abio­ge­n­e­sis events?

  I’m not aware of an argument that there was only on abiogenesis event on Earth, just the observation that all known surviving lineages come from a universal common ancestor fairly early on. In principle that would be compatible with any number of initial events. It’s just that once a given lineage evolved enough adaptions/​improvements, it would spread and take over, and then no new lineage would be able to compete/​get started.

  Your observation is an argument for only one abiogenesis event, and your claim that one would spread and take over and no new lineages would be able to compete is a counterargument. I think mentally using this wording is helpful for properly probabilistic thinking.

  I didn’t mean for the opening comments of my question to be interpreted as justification for the structure of my vague estimate. Said estimate (although plausibly off by several hundred orders of magnitude) says abiogenesis events are cosmically unlikely, and this notion was used in the presentation of the question.

• Dach 27 Nov 2021 21:32 UTC

  1 point

  in reply to: YimbyGeorge’s comment on: How com­mon are abio­ge­n­e­sis events?

  While cool, I didn’t expect indefinite self-replication to be hard under these circumstances. The enzymes work by combining two halves of the other enzyme- i.e. they are not self-replicating using materials we would expect to ever naturally occur, they are self-replicating using bisected versions of themselves.

  I’ve slightly downgraded my estimate for the minimum viable genome size for self-replicating RNA because I wasn’t thinking about complicated groups of cross-catalyzing RNA.

[Question] How com­mon are abio­ge­n­e­sis events?

• Dach 19 Oct 2021 13:17 UTC

  21 points

  in reply to: vV_Vv’s comment on: My ex­pe­rience at and around MIRI and CFAR (in­spired by Zoe Curzi’s writeup of ex­pe­riences at Lev­er­age)

  Scott: if you believe that people have auras that can implant demons into your mind then you’re clearly insane and you should seek medical help.

  Also Scott: beware this charismatic Vassar guy, he can give you psychosis!

  These so obviously aren’t the same thing- what’s your point here? If just general nonsense snark, I would be more inclined to appreciate it if it weren’t masquerading as an actual argument.

  People do not have auras that implant demons into your mind, and alleging so is… I wish I could be more measured somehow. But it’s insane and you should probably seek medical help. On the other hand, people who are really charismatic can in fact manipulate others in really damaging ways, especially when combined with drugs etc. These are both simultaneously true, and their relationship is superficial.

  Scott 2015: Universal love, said the cactus person

  Scott 2016: acritically signal boosts Aella talking about her inordinate drug use.

  Scott 2018: promotes a scamcoin by Aella and Vinay Gupta, a differently sane tech entrepreneur-cum-spiritual guru, who apparently burned his brain during a “collaborative celebration” session.

  Personally, when I read the cactus person thing I thought it was a joke about how using drugs to seek “enlightenment” was dumb, and aside from that it was just entertainment? That Aella thing is a single link in a sea of 40 from 5 years ago, so I don’t care. I don’t know who Vinay Gupta is- from reading Scott’s comments on that thread I get the impression he also didn’t really know who he was?

  I’ll add a fourth silly piece of evidence to this list for laughs. In Unsong, the prominent villain known as the Drug Lord is evil and brainwashes people. Must be some sort of hidden message about Michael Vassar, huh? He warned us in advance!

• Dach 19 Oct 2021 10:56 UTC

  1 point

  in reply to: acylhalide’s comment on: Hu­mans are the uni­ver­sal eco­nomic bottleneck

  Why is this limit unique? Why can’t we be working on “distribution inefficiencies and similar” for the next 100 years?

  In the case of real GDP per capita per hour worked, this limit is exactly unique- “distribution inefficiencies and similar” doesn’t apply. Indeed, this is tautologically true as you say. Think about what it would look like for an increase in real GDP per capita per hour worked to not have the form of “Something allowed for more work to be done per hour per person”. It wouldn’t look like anything- that doesn’t make any sense.

  I would completely ignore my comment on “distribution inefficiencies and similar” until you know what I mean by this. To explain my comment, real GDP per capita per hour worked is not the same as the nebulous “prosperity” I was referring to, which also contains some level of preference for how material goods are distributed.

  Maybe I’ll ask this, does your statement regarding universal bottleneck apply explicitly to humans? Or does it also apply to apes and bacteria and AI?

  Because we particularly care about how much work humans do, and how wealthy they are. We do not really care about the work hours or prosperity of bacteria. Economic productivity is measured relative to how much money people have and how much they must work to get it. Just read my previous comment and/​or the post again- this would seem to be a really basic sort of confusion that I can’t fix for you.

• Dach 19 Oct 2021 6:47 UTC

  1 point

  in reply to: MikkW’s comment on: Hu­mans are the uni­ver­sal eco­nomic bottleneck

  Self-driving technology is advancing and will soon(ish) allow us to move cars without humans being directly involved, except in terms of maintenance and management. This will be a major boon because it will partially remove humans from the equation- the bottleneck is partially removed. This has no real bearing on the title statement- I even remark about this in my post.

  The “universality” here is trivial- here is a copy-paste of part of my response to a similar comment:

  For everyone to become richer without working harder, we must develop technologies that allow more work to be done per man-hour. Aside from working out distribution inefficiencies and similar, this is the unique limit on prosperity. This is what I mean by “humans are the universal bottleneck”- we only have so many man-hours, so any growth is going to be of the form “With the same amount of hours, we do more”.

  Imagine if every area of economic activity was automated- humans were fully removed. This would look very sci-fi: think of von Neumann probes. In this situation there is no practical limit- the probes will expand and convert our entire light cone. Assuming constant population, per capita wealth would approach 50 billion stars, I guess.

• Dach 19 Oct 2021 6:33 UTC

  1 point

  in reply to: acylhalide’s comment on: Hu­mans are the uni­ver­sal eco­nomic bottleneck

  For everyone to become richer without working harder, we must develop technologies that allow more work to be done per man-hour. Aside from working out distribution inefficiencies and similar, this is the unique limit on prosperity. This is what I mean by “humans are the universal bottleneck”- we only have so many man-hours, and any growth is going to be of the form “With the same amount of hours, we do more”.

  Some segments of the economy have not had as much growth in the above department. For example, houses are assembled manually- all major parts must be done by hand, many with the assistance of only hand-held tools. Because we require shelter and the choice way of getting that shelter is owning a manually built house, this is a drag on the economy.

  Domains such as this which have not been as revolutionized by automation have scaling costs due to how directly they are pegged to the price of labor- in particular, houses of constant size cost the same general fraction of our income (if not more) despite real GDP per capita having grown significantly over time, because richer workers demand more pay. Why aren’t houses cheap? This is one of many reasons, and doing some quick googling on how many man-hours are required to build a house, the main factor in many areas of the united states.

  Is this way of thinking “new”? Surely not to humanity, but hopefully to the reader.

• Dach 18 Oct 2021 14:34 UTC

  1 point

  in reply to: Yair Halberstadt’s comment on: Hu­mans are the uni­ver­sal eco­nomic bottleneck

  Interesting, thank you.

  Is the quadrupling of drag and octupling of rolling resistance related to the assumption that drag is proportional to the surface area of the side on which the drag is produced, and that rolling resistance is proportional to weight? Either way, cost would still decrease due to larger and more complex engines, as rolling resistance per kg would not change.

  Of course, railway sizes are fixed, so there is little to be done. I was just speculating where the relative efficiency of cargo ships comes from. I made an edit at the end of the post which contains a very rough approximation of how large savings on wages are in the case of cargo container ships.

• Dach 18 Oct 2021 13:24 UTC

  3 points

  in reply to: Yair Halberstadt’s comment on: Hu­mans are the uni­ver­sal eco­nomic bottleneck

  I would expect fuel efficiency to be related to the size and complexity of the engine. Producing some amount of force is going to require the same amount of fuel assuming energy loss due to resistance/​friction is the same, and the engine is the same.
  
  If true, we could e.g. have absurdly large trains on lots of rails? I would expect energy loss due to rubbing on rails and changing elevation to be similar to energy loss due to rubbing on water.

Hu­mans are the uni­ver­sal eco­nomic bottleneck

• Dach 17 Oct 2021 1:59 UTC

  2 points

  in reply to: cousin_it’s comment on: What Do GDP Growth Curves Really Mean?

  This seems wrong. Imagine some country doesn’t have unobtainium, a mineral which is rare and also not particularly useful. You can’t get it at any price. Then it finds some, and soon enough many citizens have unobtainium paper holders. Does this mean GDP has grown by a factor of infinity? Hell no, most people would gladly exchange their paper holders for something more useful but also previously obtainable.

  Think about it this way. Suppose we have some device that was moderately valuable which everyone needed to own exactly one of, and it costs $100 per year. I don’t know- maybe all humanity has some heart disease and we need to have this medical device or we die. How much “better” does this situation get if we figure out how to cure this problem for essentially free? Say someone develops a pill that costs $.01 and it lasts for a year. In some sense, this is 10,000 times better. No one’s lives are even approximately 10,000 times better, though- people are only $99.99 richer per year.

  Ignoring diminishing marginal utility, this can be attributed to the fact that people’s budgets are not completely saturated with such innovations- housing isn’t sufficiently decoupled from things we can’t dramatically improve, like the skill of workers who operate power tools, zoning laws, and the surface area of cities.

  If everything got cheaper and better at the same rate as computers, I think it would be fair to assign some ridiculous multiplier to the increase in productivity over the past 60 years. Houses would cost cents and particle colliders would circle the solar system.

  Isn’t it the same for computers then? You’d exchange an iphone for a house in an instant, and you can’t exchange your iphone for a hundred houses. So it’s just not that valuable to you, and real GDP hasn’t grown by a factor of hundreds.

  This example seems bad? Few of us are wealthy enough to purchase a hundred houses. More directly, I would pay up to all of my discretionary income (i.e. not required to sustain my life or support my income) for access to computers and internet.

  The situation is not such that the degree to which computers are valuable to me is appropriately represented in my decision making- computers are extremely cheap, and I am wealthy enough to not have to make any sort of compromise to own one.

  I agree that real GDP or rather human productivity in the intuitively relevant sense hasn’t grown by a factor of 100 since 1960.

• Dach 5 Oct 2021 23:16 UTC

  1 point

  on: Dis­solv­ing the Ex­pe­rience Ma­chine Objection

  Is that your real disagreement with the experience machine?

  I think if you accept the premise that the machine somehow magically truly simulates perfectly and indistinguishably from actual reality, in such a way that there is absolutely no way of knowing the difference between the simulation and the outside universe, then the simulated universe is essentially isomorphic to reality, and we should be fully indifferent. I’m not sure it even makes sense to say either universe is more “real”, since they’re literally identical in every way that matters (for the differences we can’t observe even in theory, I appeal to Newton’s flaming laser sword). Our intuitions here should be closer to stepping into an identical parallel universe, rather than entering a simulation.

  The experience machine does not need to be an exact simulation of the entire observable universe, it merely needs to be a passably high quality simulation of your brain and how it would change. You can’t see the fundamental particles around you, or the evolution of the universal wavefunction. A superintelligence would thus not need to perfectly fake reality to perfectly deceive you and leave you with the same level of confidence that you’re in a simulation as you have right now.

  In fact, this is the original construction of the argument- it didn’t involve perfectly simulating the universe, it just involved deceiving the subject to such a degree that they wouldn’t be smart enough to figure out which was real.

  However, I think it’s not actually possible to have such a perfect experience machine, which would explain our intuition for not wanting to step inside. First, if this machine simulates reality using our knowledge of physics at the time, it’s entirely possible that there are huge parts of physics you would never be able to find out about inside the machine, since you can never be 100% sure whether you really know the Theory of Everything. Second, this machine would have to be smaller than the universe in some sense, since it’s part of the universe. As a result, the simulation would probably have to cut corners or reduce the size of the simulated universe substantially to compensate.

  This can’t “explain” our intuitions, because the fact that physically simulating the entire universe in exact detail is impossible is evidently not what’s causing those intuitions. People who disagree with the experience machine include those who have no idea what a “wave function” is, those who think the earth is flat and the entire universe is in a small crystal dome, leading physicists, philosophers who think building the experience machine as described is possible, and I allege many of the readers of this post before they knew that simulating the universe in exact detail was physically impossible.

  Can it explain your disagreement? That’s something for you to think about yourself.

  I personally would not step in, because I am at least mildly robust to attempts to hijack my senses. Whenever the map could become disentangled from the territory due to the plots of mad philosophers, I will try to avoid this.

• Dach 3 Nov 2020 1:37 UTC

  1 point

  in reply to: Scott Garrabrant’s comment on: What are good elec­tion bet­ting op­por­tu­ni­ties?

  I can confirm that this still works. Sum of the price of all Nos is $14.77, payoff is $15.

Sur­face Thoughts Suck

• Dach 20 Oct 2020 4:46 UTC

  1 point

  on: As a Washed Up Former Data Scien­tist and Ma­chine Learn­ing Re­searcher What Direc­tion Should I Go In Now?

  So, I guess the question boils down to, how seriously should I consider switching into the field of AI Alignment, and if not, what else should I do instead?

  I think you should at least take the question seriously. You should consider becoming in involved in AI Alignment to the extent that you think doing so will be the highest value strategy, accounting for opportunity costs. An estimate for this could be derived using the interplay between your answers to the following basic considerations:

  • What are your goals?

  • What are the most promising methods for pursuing your various goals?

    • What resources do you have, and how effective would investing those resources be, on a method by method and goal by goal basis?

  An example set of (short and incomplete) answers which would lead you to conclude “I should switch to the field of AI Alignment” is:

  • In addition to “normal” every day goals, I want extreme things. I care about the broad scope future of Humanity to a degree even roughly corresponding to what basic math suggests.

  • The best way to achieve my extreme goals is to break the current limitations on intelligence (brains need to fit in very small boxes, synapses fire in slow motion, you need to train each individual human, training time for individual humans is bad, I/​O speed is complete trash, etc.) and to safely exploit this.

    • I have the personal aptitude to work on this problem. It would be better for me to devote my personal time to this project than to pay other people to work on this project for me, for example. My normal ever day goals can also be effectively pursued by moving to the field.

  Like should I avoid working on AI at all and just do something fun like game design, or is it still a good idea to push forward ML despite the risks?

  If you’re not doing bleeding edge research (and no one doing bleeding edge research is reading your papers), your personal negative impact on AI Alignment efforts can be more effectively offset by making more money and then donating e.g. $500 to MIRI (or related) than changing career.

  And if switching to AI Alignment should be done, can it be a career or will I need to find something else to pay the bills with as well?

  AI Alignment is considered by many to be literally the most important problem in the world. If you can significantly contribute to AI Alignment, you will be able to find someone to give you money.

  If you can’t significantly personally contribute to AI Alignment but still think the problem is important, I would advise advancing some other career and donating money to alignment efforts, starting a youtube channel and spreading awareness of the problem, etc.

  I am neither familiar with you nor an alignment researcher, so I will eschew giving specific career advice.

• Dach 15 Oct 2020 10:58 UTC

  1 point

  in reply to: Said Achmiz’s comment on: In­dus­trial literacy

  You were welcome to write an actual response, and I definitely would have read it. I was merely announcing my advanced intent to not respond in detail to any following comments, and explaining why in brief, conservative terms. This is seemingly strictly better- it gives you new information which you can use to decide whether or not you want to respond. If I was being intentionally mean, I would have allowed you to write a detailed comment and never responded, potentially wasting your time.

  If your idea of rudeness is constructed in this (admittedly inconvenient) way, I apologize.

• Dach 15 Oct 2020 9:45 UTC

  −6 points

  in reply to: Said Achmiz’s comment on: In­dus­trial literacy

  Whether the future “matters more than today” is not a question of impersonal fact. Things, as you no doubt know, do not ‘matter’ intransitively; they matter to someone. So the question is, does “the future” (however construed) matter to me more than “today” (likewise, however construed) does? Does “the future” matter to my hypothetical friend Alice more than today does, or to her neighbor Bob? Etc.

  And any of these people are fully within their right to answer in the negative.

  Eh… We can draw conclusions about the values of individuals based on the ways in which they seem to act in the limit of additional time and information, the origins of humanity (selection for inclusive genetic fitness), by constructing thought experiments to solicit revealed beliefs, etc.

  Other agents are allowed to claim that they have more insight than you into certain preferences of yours- they often do. Consider the special cases in which you can prove that the stated preferences of some humans allow you to siphon infinite money off of them. Also consider the special cases in which someone says something completely incoherent- “I prefer two things to one another under all conditions”, or some such. We know that they’re wrong. They can refuse to admit they’re wrong, but they can’t properly do that without giving us all of their money or in some sense putting their fingers in their ears.

  These special cases are just special cases. In general, values are highly entangled with concrete physical information. You may say that you want to put your hand on that (unbeknownst to you) searing plate, but we can also know that you’re wrong. You don’t want to do that, and you’d agree if only you knew that the plate was searing hot.

  They are fully within their right to answer in the negative, but they’re not allowed to decide that they’re correct. There is a correct answer to what they value, and they don’t necessarily have perfect insight into that.

  Note that you’re making a non-trivial claim here. In past discussions, on Less Wrong and in adjacent spaces, it has been pointed out that our ability to predict future consequences of our actions drops off rapidly as our time horizon recedes into the distance. It is not obvious to me that I am in any particularly favorable position to affect the course of the distant future in any but the most general ways (such as contributing to, or helping to avert, human extinction—and even there, many actions I might feasibly take could plausibly affect the likelihood of my desired outcome in either the one direction or the other).

  You don’t need to be able to predict the future with omniscient accuracy to realize that you are in an unusually important position for affecting the future.

  If it’s not obvious, here we go: You’re an above average intelligence person living in the small period directly before Humanity is expected (By top experts- and with good cause) to develop artificial general intelligence. This technology will allow us to break the key scarcities of civilization:

  1. Allowing vastly more efficient conversion of matter into agency through the fabrication of computer hardware. This process will, given the advent of artificial general intelligence, soon far surpass the efficiency with which we can construct Human agency. Humans take a very long time to make, and you must train each individual Human- you can’t directly copy Human software, and the indirect copying is very, very slow.

  2. Allowing agents with intelligence vastly above that of the most intelligent humans (whose brains must all fit in a container of relatively limited size) in all strategically relevant regards- speed, quality, modularity, I/​O speed, multitasking ability, adaptability, transparency, etc.

  3. Allowing us to build agents able to access a much more direct method of recursively improving their own intelligence by buying or fabricating new hardware and directly improving their own code, triggering an extremely exploitable direct feedback loop.

  The initial conditions of the first agent(s) we deploy which possesses these radical and simultaneously new options which will, on account of the overwhelming importance of these limitations on the existing state of affairs, precisely and “solely” determine the future.

  This is a pretty popular opinion among the popular rationalist writers- I pass the torch on to them.

  Sorry, no. There is a categorical difference between bringing a person into existence and affecting a person’s future life, contingent on them being brought into existence. It of course makes sense to speak of doing the latter sort of thing “for” the person-to-be, but such isn’t the case for the former sort of thing.

  I was aware of the difference. The point (Which I directly stated at the end- convenient!) is that “It does make sense to say that you’re doing things ‘for’ people who don’t exist.” If this doesn’t directly address your point, the proper response to make would have been “Ok, I think you misunderstood what I was saying.” I think that I did misunderstand what you were saying, so disregard.

  Aside from that, I still think that saying bringing someone into existence “for” them makes sense. I think you saying the thing doesn’t “make sense” is unfairly dismissive and overly argumentative. If someone said that they weren’t going to have an abortion “for” their baby, (or, if you disagree with me about the lines of what constitutes a “person”) that they were stopping some pain relieving experimental drug that was damaging their fertility “for” their future children, you’d receive all of the information they meant to convey about their motivations. It would definitely make sense. You might disagree with that reasoning, but it’s coherent. They have an emotional connection with their as of yet not locally instantiated children.

  I personally do happen to disagree with this reasoning for reasons I will explain later- but it does make sense.

  To the contrary: your point hinges on this. You may of course discuss or not discuss what you like, but by avoiding this topic, you avoid one of the critical considerations in your whole edifice of reasoning. Your conclusion is unsupportable without committing to a position on this question.

  It isn’t, and I just told you that it isn’t. You should have tried to understand why I was saying that before arguing with me- I’m the person who made the comment in the first place, and I just directly told you that you were misinterpreting me.

  My point is: “It’s that we should be spending most of our effort on planning for the long term future.” See later for an elaboration.

  Quite so—but surely this undermines your thesis, rather than supporting it?

  No- I’m not actually arguing for the specific act of ensuring that future humans exist. I think that all humans already exist, perhaps in infinite supply, and I thus see (tentatively) zero value in bringing about future humans in and of itself. My first comment was using a rhetorical flair that was intended to convey my general strategy for planning for the future; I’m more interested in solving the AI alignment problem (and otherwise avoiding human extinction/​s-risks) than I am about current politically popular long term planning efforts and the problems that they address, such as climate change and conservation efforts.

  I think that we should be interested in manipulating the relative ratios (complicated stuff) of future humans, which means that we should still be interested in “ensuring the existence” (read: manipulating the ratios of different types of) of “Future Humanity”, a nebulous phrase meant to convey the sort of outcome that I want to see to the value achievement dilemma. Personally, I think that the most promising plan for this is engineering an aligned AGI and supporting it throughout its recursive self improvement process.

  Your response was kindof sour, so I’m not going to continue this conversation.