The ‘p’ is silent, as in “ptarmigan”, “psalm”, “phthisis”, or “pshrimp”.
pjohn
Karma: 131
So: a robot-assisted encyclopedia project that might prevent the collapse of civilisation. This may well be the most Asimovian idea I’ve ever heard....
Can this particular objection not be gotten-around by using the person’s prior internal state-of-knowledge?
If Alice’s pior internal state is “Known-ignorance of Bob’s bank details” and “Strong-belief that Charles isn’t sleeping with anybody else”, then Bob’s remaining silent causes Alice to maintain a perfectly valid internal state, whereas Charles’ remaining silent causes Alice to maintain an internal state that Charles knows to not match reality (along a dimension about which Alice might reasonably be expected to care).
I’m not claiming Charles is necessarily morally obliged to tell Alice he’s sleeping with other people—just that I think this particular objection might not hold if we’re allowed to factor-in Alice’s prior internal state-of-knowledge.
The most we can say using an argument from bandwidth is “If the ideal moral setpoint (between “true-but-trivial; not morally necessary to share” and “true-and-significant; morally necessary to share”) is low enough a person acting morally would have bandwidth to spare, and if the setpoint is high enough a person acting morally would saturate their available bandwidth”. It can’t actually say anything about where the optimal setpoint actually ought to be.
I think you and I might be in agreement that the optimal setpoint does indeed leave plenty of bandwidth headroom for ordinary communication, but I’m not sure that @Fernand0 is actually arguing for a setpoint so high it would saturate all available bandwidth: the “I had an affair” and “I’m sleeping with other people” examples do seem intentionally-chosen to meet the ‘true-and-significant’ test.
“[Like] Bronze Age peasants attempting to build the Tower of Babel, humanity is messing with forces we cannot hope to understand or control”
To be fair, Bronze Age peasants built some pretty impressive towers, and judging purely from the robustness and longevity of their structures compared to the robustness and longevity of modern buildings, one might suppose them to have had a better understanding of the forces involved than we do...
“why should my children have to bear this incredible burden?”
Nevermind the economic burden of looking after anybody else: I can think of few greater psychological burdens than knowing I was brought into the world primarily in order to look after just my parents when they were old, or—perhaps even worse—because my parents figured that they and their peers might enjoy their public spaces more if they were decorated with children. I would resent parents who thought like that a great deal, I think.
“They see their 90 neighbours holding out their hands expecting to be looked after”
I think it’s the other way around! People deliberately having children in order to have somebody around to do all the work when they are elderly seem to be the ones “expecting to be looked-after”!
Suppose a terminally-ageing population such that each generation is 50% the size of the previous one: each adult would need to generate an economic surplus of 100% during their working life so that the next generation had sufficient resources to care for them. This seems difficult, I know, and I am glad we have social safety nets for people who fail to do this (paid-for by the people generating more than 100% economic surplus), but:
Firstly, it’s not as difficult as it seems: if the work you do only earns you £20 per hour but your employer generates a profit of £40 per hour of your labour—in other words your £20 of labour generates £40 of goods or services—then you have probably generated the requisite 100% surplus. Note that dividing the annual profits of a medium-sized company by their employee count generally yields a surplus figure of several thousand percent: the fact that only 2 to 8 percent of this 3000-odd percent is typically set-aside by the company (in the form of a pension contribution) to provide for the employee at the end of their lifetime of labour seems to me like a problem with how companies work, not a problem with some other entirely unrelated people not having enough children.
Secondly, no matter how difficult generating sufficient surplus was, intentionally planning to generate less and use the labour of some large number of children to make-up the shortfall seems like an incredibly selfish choice, to me. (Difficult to do “explain, not persuade” on this one, I admit! I think it just seems wrong as essentially a final, non-instrumental value, and I don’t think “it’s natural”, “it’s just the way of the world”, or “there’s no other way” are sufficient justification for inflicting that burden on the beings one is supposed to love and protect above all others.)
Separately: we absolutely have the resources to support our ageing population! The UK’s purchasing-power-adjusted GDP per capita is just over £50,000. Meanwhile, the highest birthrates and largest family sizes are almost-universally found in regions with catastrophically low domestic product per capita. Speaking for myself, I would prefer solutions involving better distribution of our really-quite-plentiful resources over solutions involving manufacturing people and expecting them to do the work for us.
Okay, it sounds like we’re pretty much in full agreement here. (And for the parts where neither of us know how reality works—we seem to nevertheless agree on what we don’t know and why we don’t know it!)
For what it’s worth I did have a bit of trouble parsing your original message in places, and I clearly misunderstood parts of it (sorry!) - but it absolutely didn’t appear rude or impolite at all to me; just as arguing (politely) for something that, it turns-out, you weren’t actually arguing for. I’m sure it’s my error as much as yours!My own English is, I’m sure, nothing to write home about ..pun fully intended.. but for what it’s worth your writing does seem to include lots of “it’s not X, it’s Y” and other such sentence structures that lots of English speakers interpret as indications of AI-written text. I don’t think it’s fair for people to interpret it like this—cf. “I’m [a non-native English Speaker]: I don’t write like AI. AI writes like me.”—and so long as the author understands and agrees with what is written I don’t think there’s anything wrong with AI-assisted writing anyway, but I just thought I’d mention it lest you encounter such an accusation!
I disagree with many of the Tegmarks’ re-labellings—in that I think in many cases the original label is more fair, accurate, and appropriate than their proposed new label—but I do not think it is obnoxious at all to say “here are the labels you probably currently use; here are some others you might consider”.
“you should always use language which prioritises my particular idea of what the bad results of your actions are and implicitly bakes in my worldview”
I think this is a pretty uncharitable characterisation of what the authors were trying to do. A more fair characterisation would be, “If you agree with our view of the situation, don’t get drawn-in to using language that implies a different situation; that language is chosen intentionally to support that outlook, so if you wish to support our outlook you should be intentional with your language too”.
Even though I disagree pretty strongly with the actual outlook proposed in many of the examples I think this is essentially a form of intellectual honesty, not “obnoxious” or “annoying”, and I could absolutely come up with non-AI-related examples of the same phenomenon I’ve seen in business (for example employees being referred to as “people” when the company is doing something nice to them but as “resources” when the company is doing something unpleasant to them, “pay rise” used when it would be more accurate to say “pay calculation restructuring that’s on net a pay cut for practically everybody”, etc. I’m sure you’ve seen many such examples, too!)It’s explaining a subtle mechanism for manufacturing agreement at a less-than-fully-concious level and showing how you can carefully use language to maintain the position you’d previously arrived at through rational thought rather than being caught-up by the language and allowing yourself to be unconsciously diverted into the other party’s opinion: that’s pretty classic LessWrong!
‘seems to fail one of the three stated requirements for being on the front page, to wit, “aim to explain, not persuade”.’
I don’t read it as trying to persuade the reader of AI harms. I read it as being aimed at readers who are already persuaded of AI harms, explaining how to respond to those harms (showing how responses to other research-level harms worked and where they failed, etc.)
Perhaps the article could have been more explicit about saying this (and, as I say, personally I’m absolutely not persuaded of many of the harms being presented) but I don’t think an objective reader could really mistake it for an article trying to persuade the reader of harms, and I think it’s entirely fair to assume the intended audience (AI capabilities researchers on LessWrong) would have already encountered arguments for and against AI harms and formed a position, without needing the argument re-explained to them from scratch yet again. (Although a few hyperlinked citations would have been nice, of course..!)
More generally, I think it’s reasonable to say “Given problem P, here’s what I think we should do” and it would be impractical and counterproductive to require every such post on LessWrong to first present a full argument for P from first principles.
“Vasily Arkhipov was criticized after vetoing a Soviet nuclear strike against the US”
My goodness, I think this might be the most beautifully understated claim in all of philosophy!
In addition to the “enduring criticism” angle, I think people in large corporations are under social and economic pressure to follow-along with the corporation’s position and are often expected to make moral decisions whilst enduring personal discomforts (lack of sleep, pressure of deadlines, family/life worries, etc.), and Arkhipov also provides an excellent example/inspiration for thinking for oneself whilst under immense pressure (...in Arkhipov’s case literally as well as figuratively...) and despite intolerable personal discomforts (in his case sleep-deprivation, oxygen-deprivation, separation from his family, fear of imminent death, etc.)
“Hannah Arendt wrote about “the banality of evil”. arguing that the greatest harms are often done not by malice, but by obedient and conscientious technocrats who don’t think about the bigger picture.”
“There are hardly any excesses of the most crazed psychopath that cannot easily be duplicated by a normal kindly family man who just comes in to work every day and has a job to do.”
-- Sir Terry Pratchett, “Small Gods”
(Convergent evolution, or had Pratchett read Arendt..?)
[Edit: What follows has turned-out to be a fairly strenuous disagreement with your position. Sorry—I only noticed how strenuous it was after the words/thoughts were done pouring out of my head! I know you’re new ..I’m pretty new around these parts myself.. so I thought I’d message to apologise in advance and ask you to not take it personally. Even though I do very much disagree with you I’m nevertheless interested in your opinion and glad to have the discussion!]
I think this is false because the premise “muscles are internal whereas morality is external” is:
1) False, in that morality and moral behaviour can be thought of as “applied conscience”, and conscience exists within the mind/brain, which is internal.
2) Irrelevant, in that there doesn’t seem to be any practical reason why being internal or external matters; what matters is whether the thing can be trained and strengthened.
You can strengthen muscles -internal- by going to the gym, and you can strengthen a building -external- by adding extra structural supports, digging deeper foundations, etc. Both are important and it doesn’t really matter whether strengthening morality is more like strengthening muscles or more like strengthening a building.
Separately, I think it is self-inconsistent: in one place you say “morality is external”, and in another place you say “morality is trained since childhood”. Since the stuff we train in childhood must be internal (since we don’t bring our external childhood environments along with us into adulthood, only our internal selves ….my beloved teddy bear obviously notwithstanding....), I don’t think it’s possible to simultaneously claim that we train morality in childhood but that morality is external.
(Sure, the training is external: most moral positions we train-on come from other people, we didn’t figure them out for ourselves. A gym-goer also didn’t invent press-ups and bench-presses and dumb-bells and gymnasia for themself either—but all those external ideas and tools nevertheless improve them internally...)
Despite this, I do think it’s entirely legitimate to ask “Is there an objective, universally-true morality that isn’t just some guy’s opinion?”. Philosophers have been debating that for millennia and it’ll probably be millennia more before we’ve made any headway on this question.
But—if we all acted as though there was no objective morality, it was all just some guy’s opinion, and literally everything was fundamentally equally acceptable… I’m pretty sure that what remains of civilisation wouldn’t last long enough to answer the question either way...
Personally I think that even though it’s difficult to know exactly what is and isn’t moral in a universally true, non-childhood-dependent, non-external-factor-dependent way, it’s nevertheless quite plain to see that an objective universal morality can exist:
Suppose a world full of torture and mutilation and suffering and loneliness and hate and utter, abject misery for every thinking being, all the time, with no respite, for all eternity—a “hell”, in other words. I think it takes an unsupported assumption (or a “leap of faith” if you prefer!) to say that such a world would be bad and that we should avoid it and not create it—but it requires the tiniest, most reasonable and most fair possible assumption in all of philosophy.
Once you make that, it’s possible to (in theory!) measure the distance between this hell-world and any other world in world-space (not physical space but the theoretical parameter-space describing all possible worlds), and assess how similar or dissimilar other possible worlds are to the hell-world, and upon which axes. From that, I think it’s possible to bootstrap an entire objective universal morality.
...in principle! Of course in practice navigating the parameter-space is utterly computationally intractable and the best we can do with our limited tools and intellects is flawed navigation heuristics like “utilitarianism”, “virtue ethics”, etc. that -much like general relativity- work perfectly in some situations but break-down in others. These heuristics being flawed doesn’t mean that there’s no objective morality, even if we can’t yet figure-out what it says we should do in some situation.
(What about any beings that don’t make even the tiniest and most reasonable of assumptions that the hell-world is bad, and would be either supportive of or indifferent to making the real world like that?
On a philosophical level I would say that if they maintain that all worlds are equally acceptable and it literally does not matter what the world is like at all then they don’t have moral patienthood—in other words what they want by definition can’t matter, since everything is equally acceptable to them.
On a personal level, of course, I would say that such people are ☠️the enemy☠️ and must be opposed with every fibre of our beings if we want the world to ever not be a horrible place for everybody!)
Great read. Thanks for sharing!
....How did the three decisions turn-out? Do you eat meat, use cutlery, and/or use chairs today?
For me it was a motorcycle. I didn’t want to Interrail because, even though I adore the railway as an institution, I didn’t like the idea of being a passenger and being passively being carried to different places without somehow “achieving them” by my own hand (I realise this is an absurd and ridiculous decision… not a reasonable and sensible decision like refusing to sit in chairs...)
So, I loaded-up my ER5 (an objectively boring, mundane, unsexy motorcycle—but a faithful and trusty steed, to me!) and took the ferry to Dieppe with a hand-written list of directions worked out from paper maps and sellotaped to my left arm, and to the bag strapped to my petrol tank (I think Android OS had only-just been released but, if it had, it certainly hadn’t yet made its way down the economic food-chain to the likes of me..) My theory was that enough sellotape, being plastic, would waterproof the paper. It partially worked!
I only got as far as exploring parts of France and Belgium, and later a fair amount of Britain. (I always wanted to ride along the Zeeland-Holland coast: there are some long bridges and viaducts there that, even on large-scale paper maps, really stood-out as something unusual. Teenage-me imagined myself riding over them, no land on either side, no scenery, no visual distractions, just self + bike + road. It sounded amazing! But I never got that far. Perhaps someday I shall go back..)
Along the way I had some amazing experiences that I still often think about even today, saw many fascinating things I didn’t even know existed (I’d never really been to another country before, except as a small child who didn’t really notice anything beyond the superficial), met some really wonderful, interesting, and kind people, exposed myself to ludicrous amounts of unnecessary risk, challenged myself (physically, intellectually, spiritually -by which I mean ‘l’Espirit Humain’, not as in ‘religiously’- and in my ability to pilot, navigate, and maintain a motorcycle under adverse conditions), learned loads -through comparison- about my home country, and learned a fair bit about myself.
These days I mostly just ride my motorcycle (no longer an ER5, thankfully—but also sadly..) to work, to the supermarket, etc. - but I still have a relationship with it, in a way most people don’t seem to with most machines, and I think that those tours I did, two decades ago, are part of the reason why.
If you had a perfectly massless car, at rest in perfectly still air, and you accelerated, would it go forwards (because the tyres are nevertheless still in contact with the ground albeit with 0 force pushing them into it)? Or would the wheels just spin (because since there’s a 0 term in the friction equation)? Or (let’s say it’s rear-wheel drive) would the wheel stay in-place and the car rotate around it, like a motorcycle doing a wheelie?
I would’t describe downforce only when cornering as “magical”: seems eminently achievable with active aero. A control-surface active aero system was patented by BMW in 2024 and a jet-assisted version was demonstrated in 2025.
If the car could be powered by a propeller (or for that matter a rocket..) we could simply vector the thrust to give us whatever balance of forces we want. Vector more upwards thrust to push the car into the road and give more grip, vector more downwind thrust to compensate for wind, vector all thrust dead astern for maximum acceleration, &c.
You’re absolutely right that maritime law works this way—but actual shipping companies manage to get around it all the time.
1) Poorer nations compete with one another to have the absolute most permissive maritime regulations they possibly can so as to attract shipping companies registering with them as a flag state. (The money from such registry ain’t great but it makes a significant difference to certain economies).
2) The shipping companies register ships under one flag state then, if they’re ever forced to submit to regulations or go to court or anything they simply re-flag the vessel and say “Sorry, we’re Panamanian now, not Nigerian, we’re out of your jurisdiction”. Within a few years the same vessel will fall afoul of Panama’s authorities and be re-flagged as Liberian, then Bermudan, and so on.
3) When you’re deep sea, you can do all sorts of illegal stuff—under both international law and the laws of even the most permissive flag states—including gross environmental damage, forced labour, human and animal rights abuses, and nobody will ever know. This stuff happens all the time (source: spent half my life at sea). There’s very little money, political will, public demand, and practical ability to police the behaviour of vessels on the other side of the planet, the best part of a thousand miles from the nearest inhabited land and ten thousand miles from your country’s nearest government asset.
The EU is experimenting with detecting certain kinds of common deep-sea environmental crimes by satellite remote sensing—but A) this has the same jurisdictional problems as everything else and you can be sure the rest of the world won’t ever spend money on it, and B) it’s only effective for detecting one type of crime, and even then only when the resulting pollution is big enough to be seen from space. The Bermudan government developing the capability to remotely detect illegal activity inside a USA-headquartered (but Bermuda-flagged) megacorp’s satellite seems even more unlikely.
4) You don’t even need to go to sea (or into space) for this sort of thing. Note for example that Meta’s digital sweatshops and workforce-wide human rights abuses are located in Kenya and Ghana rather than San Francisco.
I really hope you’re right and that flag-state controls can be relied-upon to prevent illegal activity in space—but I’m afraid maritime law (and the behaviour of entirely terrestrial corporations..) doesn’t offer a very promising case-study.
“If the universe were infinite, there would be infinite humans”
As discussed elsewhere in this thread, I think there are entirely plausible—perhape even probable! - cosmologies wherein an infinite universe can yield infinite Boltzmann Brains but not infinite humans. Having infinite humans would require infinite negentropy (or if you prefer infinite matter and energy), not just infinite spacetime. Boltzmann Brains could exist in an infinitely long, post-stellar-epoch, post-heat-death, maximum-entropy, infinite de-Sitter-space universe with no remaining free energy sources, but humans couldn’t.
“BBs with years worth of highly ordered pseudo-memories would be very rare compared to ones with chaotic pseudo-memories”
I agree, but if both types of Boltzmann Brains are infinite (even though one type is much more rare than the other) yet humans are finite, we’d expect most beings with our experiences to be those rare-type Boltzmann Brains, not humans. (I don’t use this to argue in favour of our being Boltzmann Brains, but as part of the paradox I set-out in a previous reply.)
It sounds like you understood physics pre-puberty that I’m only-just starting to speculate upon in middle-age (damn you!)
I speculated on this elsewhere—just didn’t realise that’s what you were describing here! Ingenious technique; I shall absolutely try it!
I speculated on this mechanism in another comment (thanks for the confirmation!) - I just didn’t realise that jam jar lids were malleable enough to be levered-open from the side (I was imagining puncturing them). Thanks!
The discussion is about lids that unscrew (like on a jam jar) not lids that need to be levered-off (like on a paint tin).
I have seen jars of comestibles with paint tin style lever-off lids, but they’re pretty rare where I live. Possibly they’re more common where you are—which would explain the confusion?
(And personally I’d prefer to use a flat-blade screwdriver -I have quite fancy delicate teaspoons!- but if the choice is exclusively between a teaspoon and a knifeblade, I’d pick the teaspoon too!)
I don’t think the lids are hard to open because they’re screwed on tightly, but because there’s a vacuum in the jar? (cf. those Age of Enlightenment experiments where large teams of horses couldn’t open vacuum-sealed containers.. history doesn’t record whether the horses ever stopped pulling in teams and tried just unscrewing them...)
Possibly you could make them easier to open despite the vacuum by making the thead pitch finer—but I suspect that since the threads are cut into glass rather than steel there’s a limit to how fine-pitch they can be made and still be robust.
But anyways—why is there a vacuum in the jar? To preserve the jam? Isn’t jam a preserve? Like, I thought the whole raison d’etre of jam was as a way to make fruit keep, unrefrigerated, through the winter? Why must we preserve the preserve? Quis custodiet ipsos custodes?
I’m pretty sure that when old ladies from the Women’s Institute make jam to sell at the village fête [at a manufacturing cost of £2 per jar.. which they then sell at a cost of 50p a jar.. to raise money for.. old ladies..] they don’t vacuum-seal the jars, and their jam seems to keep just fine.
Lastly (probably well-worn common knowledge—apologies if so—but I haven’t seen anybody else mention this so I thought I would just in case): To make it easier to unscrew a jar you can increase the coefficient of friction, increase the torque, or (if I’m right about the vacuum thing!) increase the pressure inside the jar to bring it closer to atmospheric pressure:
If your hands are moist or oily, wrapping something like a tea towel or (dry) dishcloth around the lid can increase the coefficient of friction; this also increases the effective diameter of the lid (thus increasing the torque).
A thin rubber sheet or rubber apron would do this even better (but presumably less useful if one opens one’s jam in a kitchen and not a sex dungeon). You can get a special tool (that looks a bit like an automotive oil filter strap-wrench) to increase the torque, from kitchenware shops, but having yet another single-purpose kitchen gadget lying around is pretty annoying.
Running the jar under a hot tap would increase the internal pressure (but seems a bit wasteful of energy/water). Piercing the lid with an awl would increase the internal pressure (and an awl seems like a less annoying and more useful thing to have around the house than a jam-jar-strap-wrench-thingy) but is it a problem for shelf-life if the lid has a small hole in it? If jam genuinely is a preserve, possibly not!
Partially agree. If the universe were both infinite in size and contained infinite negentropy (or, if you prefer, infinite matter and energy) then sure, I’d agree that there would likely be vastly more human brains than Boltzmann brains.
However, if the universe were infinite in size but didn’t contain an infinite amount of negentropy (for example, if the universe started with some fixed amount of negentropy -like all the matter and energy present at the Big Bang- and then became infinite in size by inflating for an infinite amount of time, but after the initial Big Bang no additional matter/energy was added) then I’d say the infinite size alone wouldn’t be sufficient to make human brains more probable.
I don’t know enough about cosmology to know whether there’s any sort of consensus on whether the amount of negentropy actually is finite or infinite. I was (as you probably deduced from my previous comment!) just assuming a universe that’s infinite in size (both spatially and temporally) but not in negentropy (hence the maximum-entropy de Sitter ‘heat death’ epoch).
“It’s hard to compare infinities”
True, but that doesn’t mean we can’t make any comparisons between them! For example, I think we can say “If the universe is infinite in size and contains an infinite amount of negentropy/matter, then the “universe size” infinity must be a larger infinity than the “amount of matter” infinity, otherwise we would observe infinite density throughout the universe”.
There are purer examples: Cantor’s infinite series maths, where by matching-up elements in sets you can compare not only the relative sizes of infinite sets but also the “size classes” of types of infinite set. So eg. you can compare the size of the set of positive even numbers (let’s call that Set A) to the size of the set of positive integers (Set B) by saying “if we match up the 2 from Set A with the 2 from Set B, the 4 from Set A with the 4 from Set B, the 6 from Set A with the 6 from Set B, and so on all the way to infinity, we’d still have 50% of the numbers in Set B left over, therefore Set A is 50% the size of Set B” and you can say “If Set C is ‘the set of all the numbers between 0.0 and 1.0’ there’s no way to match-up the numbers in Set C with the numbers in Set B, therefore Set C is a bigger ‘size class’ than Set B”.
(Confession: pretty sure Cantor didn’t call this sort of thing a “size class” but I have no idea what it is called, so I made up the “size class” term..)
Can we make use of this in cosmology? I think so: my “infinite density” argument above basically works like this. More pertinently, if we could assess the probability of a human brain existing (per unit spacetime, per unit negentropy) as (say) 10^-50 and the probability of a Boltzmann Brain existing as 10^-52, we would be able to say “There are around 100 times more human brains than Boltzmann brains”, even though both could be infinite if the spacetime and negentropy were infinite.
What are the implications of this for FLOSS? Might it mean lots of open-source projects stall because their volunteer developers can make software that perfectly meets their personal needs and so they no longer need the wider project? Or might it make no real difference (perhaps because not many volunteer developers would want to develop software only for themselves, or else perhaps enough FLOSS is actually developed by professional, employed devs)? Or might existing projects that have already achieved a critical mass carry-on okay, but we won’t see so many new ones starting-up any more? Or something else entirely?
Writing plugins and figuring-out obscure config languages seem like the sorts of things AI could make much quicker and easier, too! I would suppose that AI would reduce the barrier to these (and to forking existing software) far more than it reduces the barrier to writing software from scratch.
(That being said—the idea of having programs re-written in pure Rust without twenty years of cruft is rather appealing!)