Just this guy, you know?
Dagon
Karma: 8,807
I don’t think we’ll do anything then—the aliens kill or quarantine us. If they’re weak, but somehow also smart enough to convince us of this (quibble: do they convince every human, or just some?), I suspect we’ll try to change our behaviors based on this new knowledge rather than destroying ourselves.
It’s possible that they only need to convince a minority of rich/powerful humans, who then destroy all humans. I don’t know if that qualifies for this question. Note that it doesn’t even have to be true, nor come from aliens: there’re always some humans doing things with a chance of mass destruction. I also don’t know if this qualifies for your question.
“Some people believe humans are net-negative for the earth, and later some people destroy the planet”. This is plausible, but not quite what you’re asking.
I presume the same decision/action mechanisms that destroy all value in the universe are the same ones deciding whether to isolate/suicide/whatever. So there’s a correlation in predicting this. There’s no way to voluntarily avoid that fate, or it wouldn’t be a fate.
I can’t follow your example. Does the AI have a goal in terms of the quantum wavefunction of the universe, or a goal in terms of abstractions like “cow neurochemistry”? But either way, is this utopia full of non-aligned, but not “actively evil” humans just another modeled and controlled part of the wavefunction, or are they agents with goals of their own (and if so, how does the AI aggregate those into it’s own)?
And more importantly for the post, what does any of that have to do with non-causal-path coordination?
Upvoted for interesting ideas, but I’m completely unconvinced that this is possible, desirable, or even coherent. I think a more formal mathematical definition of “coordination” would be required to change my mind.
Part of my confusion is your focus on “key decision-makers”. Do you mean “every agent in the civilization”? If not, what defines the path of an isolated group/civilization? But most of my confusion is what actions specifically are being coordinated, if communication is impossible. Acausal trade is already pretty theoretical, and completely unnecessary if both parties are ALREADY aligned in their utility functions over each observable part of the universe. And any causal agreement is impossible without communication.
Coordination across vast distances of spacetime would be necessary to remain a united civilisation (as opposed to fracturing into a multitude of [potentially very divergent] civilisations)
Note that this prevents improvements as much as it prevents degradation. And more importantly, this seems like “control” or “limitations imposed on children” rather than “coordination”. Unless you model “civilization” as an ant-like single utility function and a single timeless agent, which sub-agents merely comprise. No independence or misalignment or even uncertainty of goals can exist in such a picture, and I’ll pre-commit to finding the weakness that brings the whole thing down, just for pure orneriness.
I think a lot of people (including experts, sometimes) have a subconscious or semi-legible karma or external-judgement quasi-religious aspect to their beliefs too.
Humans caused this with their greed and shortsightedness and unwillingness to give proper respect to the right beliefs. Thus, there’s no possible human ingenuity or trickery that can avoid it, that’s just more hubris.
This is just another example of something that humans with the assistance of tool-AI can do as well or better than AI on it’s own. Which makes it a good topic to watch for—how much (if any) porn revenue is computer-generated? Nonzero, I’m sure, but I don’t follow closely enough to know if it’s significant.
It’s also another example of something that the median performer just doesn’t make much, and is pretty well saturated with humans trying their hand (or other parts) at it. For a long time yet, humans will be cheaper, at any quality/revenue level, than AI.
It’s possible that those that choose the social sciences do so because the questions are more important, or at least more salient to their social and moral universe. How do people behave, and how do we make them happier, more productive, less violent, whatever certainly SOUNDS more useful than why does the thin-surface of drying fluids avoid micro-wrinkles (yes, I know someone with a Ph.D. in watching paint dry).
It’s also possible that the longer (or often, nonexistent) feedback cycles are a feature that are sought by some—it’s kind of brutal finding out you’re wrong on little details over and over, especially when that feedback doesn’t really tell you whether you’re right or wrong about the bigger picture.
[Note: I’m an AGI skeptic, at least compared to the median LW poster. I expect decades of improving tool-AI that never quite reaches human levels of out-of-domain judgement and exception handling. ]
There’s really not much advantage to pure automation over AI-assisted humans (and as they get more advanced, human-oversight of AI). And there’s a lot of downsides, especially around adversarial manipulation of environment and ECM/capture to turn weapons against their own side. The very things that make self-contained AI weapons better than remote pilots (or at least remote weapons-unlock) are the things that make it terrifying to the users, not just the targets.
There’s an interesting parallel to landmines—they’re just too indiscriminate to be terribly effective on today’s battlefields. AI drones don’t have the problem of extending danger past the conflict (unless they do), but there’s a LONG way before they’re better than humans at deciding whether to take the shot.
I wasn’t an initial downvoter (just saw it now), but I am now (I vote for effect, not direction, and I think ~30 is a good target—I’ll remove my downvote if it gets much below that). I put the reasons for my concern in a comment so they can be addressed, which would also cause me to change down- to upvote.
Where’s the budget for this coming from? Free flights and lodging, plus $100 for referrals seems … extravagant for the stated agenda and purpose. Especially on fairly short notice, which just increases my suspicion that this isn’t fully described here. And even more especially with no remote options (even access to talks or material) or publishable support documents.
I have no concrete basis to be worried on anyone’s behalf, but seems suspicious in today’s funding environment and cult-like accusations for some sub-groups. It pattern-matches to timeshare sales tactics, or cult recruiting. I presume it’s NOT that, but you might want to publish some more information or open up the content a bit so it’s more clearly legitimate.
I don’t think you can ignore property-rights origin/maintenance/enforcement on this topic. “who has the right to exclude people from using” is the fundamental question you’re trying to judge, and it’s intimately tied to validity of ownership.
Upvoted because it’s pointing at a real source of pain, and it’s very good to talk about. But I suspect there’s a lot of typical mind fallacy in the parts that sound more universal and less “here’s what happened to and worked for me”.
For me, I went through my doomsday worries in my teens and twenties, long before AI was anything to take seriously. Nuclear war or environmental collapse (or one causing the other) were assumed to be the forms of destruction to expect. Over the course of a decade or two, I was able to accept that, for me, “memento mori” was the root of the anxiety. I don’t want to die, and I probably will anyway. There may be no actual outside meaning to my life, or by extension to anyone else’s. And that doesn’t prevent me from caring about other people (both individuals and groups, though not by any means equally), nor about my own experiences. These are important, even if they’re only important to me (and, I hope, to some other humans).
I’m surprised, as a poker player, you didn’t include the example that it’s pretty well known that the most important skill in poker is game selection. Playing well, knowing the odds, and reading your opponents are important, but your win rate is almost completely determined by whether you’re playing in soft games, and leaving when the lineup is tough.
Separately, I think you should introduce into this framework the idea of comparative advantage (https://en.wikipedia.org/wiki/Comparative_advantage). In most pursuits, it matters not just how good your product is, but also how much better YOUR product is than the competition. Absolute goodness is rarely enough, in the world of plenty we live in. This means that even if many people prefer a thai-style omlette, the fact that it’s much less demanding to produce than a french omlette means it’s not as profitable/useful to be good at it.
I’m pretty skeptical of acausal trade, so I don’t think I’m the best one to answer this. But my understanding of decision theories that engage in it do so because they want to be in the universe/multiverse which contains this increased utility.
I’m not sure humans have a commitment framework that would work for this. No matter what we say we’ll do, we’re going to shut down and destroy things that look dangerous enough.
Gah. Serious false dichotomy—almost all behavior is a mix of the two. Is charging a toll for the bridge you built OK? What if you used up the only good crossing spot, putting the ferry out of business and preventing any competing bridge being built? What if your great-grandparents claimed the crossing point from the natives, who considered it common property?
Who gets to decide how much of the value is morally directable to what participants? How far back do you go in determine what’s a “legitimate” property right on which to extract rents?
Information asymmetry, done intentionally and to a great degree, is spelled S C A M. But that’s not really the main problem with most crypto products (“crypto” is a pretty wide field, and there may be some coins or innovations that provide long-term value).
The vast majority of crypto products are pretty clearly not intrinsically valuable, and there’s not much asymmetry to that knowledge. There’s just the greater sucker theory, where all investors are of the mistaken impression that they’re on the right side of the ignorance/timing divide.
I think we mostly agree.
Probably, but precision matters. Mixing up mean vs sum when talking about different quantities of lives is confusing. We do agree that it’s all about how to convert to utilities. I’m not sure we agree on whether 2x the number of equal-value lives is 2x the utility. I say no, many Utilitarians say yes (one of the reasons I don’t consider myself Utilitarian).
game which maximizes log utility and still leaves you with nothing in 99% of cases.
Again, precision in description matters—that game maximizes log wealth, presumed to be close to linear utility. And it’s not clear that it shows what you think—it never leaves you nothing, just very often a small fraction of your current wealth, and sometimes astronomical wealth. I think I’d play that game quite a bit, at least until my utility curve for money flattened even more than simple log, due to the fact that I’m at least in part a satisficer rather than an optimizer on that dimension. Oh, and only if I could trust the randomizer and counterparty to actually pay out, which becomes impossible in the real world pretty quickly.
But that only shows that other factors in the calculation interfere at extreme values, not that the underlying optimization (maximize utility, and convert resources to utility according to your goals/preferences/beliefs) is wrong.
expected value return of . It’s clear that if you play this game long enough, you will wind up with nothing.
That’s only clear if you define “long enough” in a perverse way. For any finite sequence of bets, this is positive value. Read SBF’s response more closely—maybe you have an ENORMOUSLY valuable existence.
tl;dr: it depends on whether utility is linear or sublinear in aggregation. Either way, you have to accept some odd conclusions.
For most resources and human-scale gambling, the units are generally assumed to have declining marginal value, most often modeled as logarithmic in utility. In that case, you shouldn’t take the bet, as log(2) .51 + log(0) * .49 is negative infinity. But if you’re talking about multi-universe quantities of lives, it’s not obvious whether their value aggregates linearly or logarithmically. Is a net new happy person worth less than or exactly the same as an existing identically-happy person? Things get weird when you take utility as an aggregatable quantity.
Personally, I bite the bullet and claim that human/sentient lives decline in marginal value. This is contrary to what most utilitarians claim, and I do recognize that it implies I prefer fewer lives over more in many cases. I additionally give some value to variety of lived experience, so a pure duplicate is less utils in my calculations than a variant.
But that doesn’t seem to be what you’re proposing. You’re truncating at low probabilities, but without much justification. And you’re mixing in risk-aversion as if it were a real thing, rather than a bias/heuristic that humans use when things are hard to calculate or monitor (for instance, any real decision has to account for the likelihood that your payout matrix is wrong, and you won’t actually receive the value you’re counting on).
The federation protocol makes some weird choices about trust and bandwidth.
JWZ’s snarky commentary: https://www.jwz.org/blog/2022/11/mastodon-stampede/
Federation and what’s local vs remote: https://medium.com/@kris-nova/experimenting-with-federation-and-migrating-accounts-eae61a688c3c
It’s a very odd mix of low-trust but also many things not cryptographically validated to provide proof of intent (just https cert validation of the server). I give it 3 months before a rogue server figures out how to hijack or spoof follows, and then the spam whack-a-mole really begins.