I think this question is best understood as part of a broader question about how good intellectual processes are developed, of which science is a subset. And of which “why not china?” is a sub-question.
Raemon
Karma: 25,556 (LW), 10 (AF)
Meanwhile, the point of the thought experiment is not for us to figure out the answer with any kind of definitiveness, but to tease out whether the thought experiment is exploring factors that should even be part of our model at all. (the answer to which be no)
At the very least, you can have some sense of whether you value things that you are unlikely to directly interact with (and/or, how confused you are about that, or how confused you are about how reliably you can tell when you might interact with something)
Sure, but bgaesop’s “I don’t believe” is disregarding the thought experiment, which is the part I’m responding to. (I’m somewhat confused right now how much you’re speaking for yourself, and how much you’re speaking on behalf of your model of bgaesop or people like him)
I think that’s totally fair, but in that case I think it makes more sense to say upfront “this conversation doesn’t seem to be meaningful right now” or “for the time being I only base my moral system on things I’m quite confident of” or some such, rather that expressing particular opinions about the thought experiment.
Either you have opinions about the thought experiment, in which case you’re making your best guess and/or preferred meta-strategy for reaching reflective equilibrium or some-such, or you’re not, in which case why are you discussing it at all?
[Note: this comment comes with small amounts of attempted mindreading, which I think one should be careful with when arguing online. If this doesn’t feel like a fair stab at what you felt your underlying reasoning was, apologies]
If I (Raemon) had said the sentence you just said, my motivation would have been more likely to be a defense against clever, manipulative arguers (a quite valuable thing to have a defense against) than an attempt to have a robust moral framework that can handle new information I might learn about weird edge cases.
Say that rather than a person coming to you and giving you a hypothetical example, the person reflecting upon the hypothetical elephants is you, after having existed for a long enough time that you’ve achieved all your most pressing goals, and you’ve actually studied cosmology yourself, and come the conclusion that the hypothetical elephants most likely exist.
I think it makes sense for people not to worry about moral quandaries that aren’t relevant to them when they have more pressing things to worry about. I think it’s important not to over-apply the results of thought experiments (i.e. in real life there’s no way you could possibly know that pushing a fat man off a bridge will stop a trolley and save five lives).
But insofar as we’re stepping into the domain of “figure out ethics for real, in a robust fashion”, it seems useful to be able to seriously entertain thought experiments, so long as they come properly caveated with “As long as I’m running on human hardware I shouldn’t make serious choices about hypothetical elephants.”
I’d be somewhat surprised if the bgaesop-who’s-studied-cosmology, had decided that ironing out their moral edgecases was their top priority and wanted to account for moral uncertainty and so actually did their own research to figure out whether elephants-outside-the-lightcone existed or mattered… would end up saying that the reason they don’t matter is the same reason fictional elephants don’t matter. (Although I can more easily imagine hypothetical future you deciding they didn’t matter for other reasons)
Do you have an intuition that the last 1000 elephants (or humans, if you consider elephants to have weight zero or some such), are more valuable than 1000 elephants/humans if there are a million of them)?
(It’s not obvious you should have such an intuition, just that it seemed like the conversation was predicated on that, and the most obvious reason it’d be the case is that one might care about preserving a species existing at all [for various possible reasons] than being slightly more numerous)
Yeah, I agree that at the very least this consideration shouldn’t get swept under the rug.
[epistemic status: mulling over my moral intuitions in realtime, almost certainly will not endorse this upon further reflection]
When Many Worlds, Big Universe, or “Lots of Simulations” comes into the picture I become very confused about how to aggregate experiences.
But my current best guess intuitions are:
suffering adds up linearly (but if suffering exists in insects or bacteria or other Brian Tomasikian considerations like simple feedback loops, it doesn’t make sense to deal with that until humanity has their collective shit in order, and can think real hard about it). It is always morally commendable to reduce unnecessary suffering, but not always morally obligatory.
positive experiences add up less linearly, and “variety of positive experiences” matters more.
I have some sense that the positive experiences of rats adds up to a finite value, which I’d guess is greater than the value of a single human but less than some finite number of humans (whose positive experiences can converge on a higher finite number because there’s a wider variety of experiences accessible to them)
By contrast, if bacteria had anything that I’d classify as a positive experience, I think it’d cap out at less valuable than one human.
there almost certainly exist beings whose capacity for both positive and negative experiences exceeds humanity
some degree of contractualism matters, which includes acausal contracts I might wish I had made with simulators, in this universe or others. I want to have some kind of consistent contractual policy, in which simulators or aliens who are more morally valuable than me still have contractual motivation to treat me well (in return for which they might be treated better by other hypothetical simulators or aliens).
I think this implies making reasonable efforts to treat well beings that I think are less morally valuable than me (but where “reasonable” might mean “first get humanity to something approximating a post scarcity shit-together situation”)
Would you get most of the benefit from this if you were able to ban advertising specifically? (not sure if you can easily do it that easily, people can probably do various under-the-table things to trade advertisements, and selling user data for advertisements offline might be an issue.)
Patreon-style funding schemes seem more likely to be sane-in-the-limit to me.
I always have mixed feelings when I read book reviews that largely are quotes from the book.
I think that’s a useful service when the book is long (especially if it’s overly long) and there is some actual intellectual labor in figuring out how to abridge it down to the essentials.
In this case I think the original chapter was pretty close to maximally compact, and if I were to have done lots of quotes it’d have essentially just been a link-post, and I’d feel uncomfortable copying the chapter whole cloth.
I do encourage people who were interested in this post to read the whole thing if they’re interested in more details.
(Legal Systems as a whole is quite long and I think individual posts that highlight particular chapters are quite valuable. My reading of the book was directly downstream of this previous, recent LessWrong post about legal systems created by prison inmates, was another chapter that illuminated key group rationality concepts for me)
No, why? I think it’s fine if people waste time online.
Huh, I didn’t mean for “wasting time online” to be a key element in my sentence.
(I think I’m about as confused about the inference you made in your comment about my comment about your comment as you were about mine)
It seemed like a natural consequence that if you remove financial motivators from the attention economy, most of what you’re left with is people using the internet to either do things that they personally were motivated to do, or are paid to by someone directly motivated to sink money into the system)
(There’d probably be a reorganization of currently attention-economy websites into “store” websites, which I think would be fine. There’s a talk transcript somewhere about how there was a window where the internet might have been built around microtransactions and then for some reason wasn’t, which would have enabled a very different incentive landscape)
This post feels to me like it’s roughly doing the same, but instead of it reviewing a book or a scientific field...
Well, to be fair, it is also a book review. :P
I think a better version of this post would have expanded more on the cultural review aspects, and possible summarized some more insights into what the Amish actually practice, possibly with some more concrete examples.
Fair, although worth noting that such an expanded version of the post would simply be the entire chapter from Legal Systems Different From Ours (available here, although I think this looks slightly different than the version I got on kindle).
I’m currently stewing in a weird of mixture of this post, with it’s reflections on what it’s like to be smart but confused during periods where people barely understand complex motions (with all the mysticism and aesthetic preferences and perfect circles thrown in)...
...and Scott Alexander’s latest post boggling at the fact that so much human collection action seems to reduce to simple functions that can be mapped onto straight lines.
Figuring out what’s up with that seems like a major puzzle of our time. I’m chuckling at future hypothetical historians who might read old Scott Alexander posts and be like “hmm, this guy was grappling with the thing we now mostly understand, but was weirdly fixated on the Gods of the Straight Lines and maybe confused about it in ways that are now clear to us. Also, what the hell is up with that Unsong book that he wrote?”
I’m not sure I parsed this comment thread, wondering if you could explain in a bit more detail what you think happened?
Is the idea here that the internet becomes a place only for people who are intrinsically motivated to do whatever they’re doing?
I could imagine this turning out to be a good outcome but it seems politically like a non-starter.
Most of this didn’t seem new to my thinking, but I appreciated this post as a comprehensive writeup of the various issues here.
(This post also motivates me to work on a Table Of Contents view that is more optimized as a primary reading experience. Because most of the points where things I’d heard before, I found myself preferring to skim the ToC and then click to zoom into particular arguments that seemed new or interesting)
One interesting sub-problem – I think FB does a good job of keeping in touch with most friends. It does an actively bad job of me keeping in touch with my parents and some relatives, because most of what they post is political memes, so I unsubscribed from them.
Most of things I’d expect to miss wouldn’t be in the form of notifications, but in terms of posts I hadn’t read, and potentially friends that I lost touch with.
Sub-questions that I’d like to answer:
1. What value am I actually getting out of social media, and is that value good?
Keeping in Touch With Friends
I have a perception, similar to Dr_Manhattan, that I am actually using facebook for a fairly reasonable thing (that it seems to actually aspire to be good at) which is keeping in touch with friends, in a way that most other platforms do not enable.
I can imagine this turning out to either:
Not be true – it’s simply an illusion that the thing I’m doing is “keeping in touch with friends”, and I’d be much better off if I just “actually kept in touch with friends” using some other process.
True, but a much weaker version of the thing than it should be – Maybe facebook actually helps me keep in touch with friends, but naturally forces it into a milquetoast version itself. Maybe I’d be better off if I had a small number of friends that I was closer to, and didn’t even pretend to keep in touch with the others. Maybe I could keep loose connections going more easily if facebook didn’t have a few behaviors that actively sabotaged it.
(both of the above are separate from what other costs FB might be imposing)
Marketing Products and Ideas
FB enables certain kinds of grassroots marketing that’s hard to do otherwise. I’ve most-obviously used this to promote the Secular Solstice. I also sometimes use it to recommend products that actually served me well that I want to reward.
I have a sense that this is actually good when done at small scales, encouraging entrepreneurship that generates new valuable things for the world. And meanwhile people have a fair degree of control over whether to subscribe to a given person’s self-promotion, so there’s incentives for the promotion to be actually useful. (Insofar as people don’t have that control it’s because FB is separately manipulating people with skinner box tech)
I have a sense that this is bad, when it reaches certain thresholds of commercial-ness, or simplified-for-the-masses-ness (which I think my solstice promotion has sometimes reached), where the tails come apart.
There’s a similar thing for the marketing of ideas, where there’s a low-barrier-to-entry way to share your thoughts and have it naturally scale. At low doses this is good and fruitful. But supercharged with goodhart’s law it turns into bad political memes.
I’m less confident that these things are net good. And, if I’m correct that they are good at low doses but bad at high-volume, less confident that there’s a principled way to fix that.
–
What are the current alternatives? How hard is it to build a real alternative?
Assuming there is value to social media that isn’t an illusion and is net-positive....
First, are there actual alternatives that do all the things well? (I don’t know, feel free to pitch me on a thing I’d want to coordinate to move my friends and my family to that already exists. Zvi, insofar as you’ve gotten people to leave facebook, where did they go?)
The Ferrett recently wrote a post on how commercial social media wants lots of users, which intrinsically pushes against the ability to make choices to maintain high quality.
Could a nonprofit funded by people with good-judgment be incentivized to make a “good” social media platform? I have a sense that this would be challenging but mostly a matter of executing well on the obvious things, and would be a good project for someone in the rationalsphere to just do. (I’d default expect it to fail for the generic competition-is-hard-reasons, but given the right founder I could imagine it having enough chance of working to be worth it)
Yeah. Rereading the thread I agree it’s not as relevant to this as I thought.
I think a dedicated response would be good.
I do think, when/if such a response comes, it would be valuable to take the opportunity to frame the debate more in terms of “sharp vs smooth takeoff” or “discontinuous vs continuous”.
(See the section on clarifying the question)