The ‘p’ is silent, as in “ptarmigan”, “psalm”, “phthisis”, or “pshrimp”.
pjohn
Karma: 19
I agree that the Guttmacher Institute (which I’d not heard of but have just looked-up) seems to be a reliable, scientifically-rigorous source.
I couldn’t agree with “no incentive to push bad stats/juice the numbers”: I think one of the lessons of the replication crisis is that almost everybody has an incentive to do this (or more accurately a bias towards doing this, not always consciously) - but A) I admit that I am saying this without actually going and looking at the stats/methodology, and B) I think your evidence (and argument) for its effectiveness is on the whole stronger than my evidence against it.
(But I still wouldn’t take the risk of using the method myself..)
Thanks most awfully for the detailed reply! It’s an interesting debate for me and I’m grateful for your effort.
I think most of what you quote from “Ms Blue and Mr Green” is true: “energy” is a more useful concept if you think of it as managing your internal sensations; tarot is more useful if you think of it as a random-number generator to help you think of things or consider possibilities you wouldn’t have come up with by yourself.
...you might not be surprised to hear that I have several problems with it, nevertheless!
Firstly, it seems to work for literally anything:
Ghosts: You don’t need to believe that Anne Boleyn walks around Hampton Court Palace in transparent form; just understand that your instinctive fear of death interacts with your cultural understanding of the intense passions (love, betrayal, power) and emotionally charged, extreme stories (the king has his wife beheaded) represented by medieval palaces, and explore what Anne Boleyn can teach you about your own fear of death, and how she can help you relate to the brutal-seeming world that your own world and culture is built from and closely related to.
Thor: You don’t need to believe a great big muscly man with a hammer causes thunder; examine your own feelings when you think of thunder: fear and awe, yes, because you can sense forces at work far greater than the mundane forces you yourself are master of—but also exhilaration, and excitement: being in a thunderstorm excites some part of you that you can’t easily access any other way. Thor is simply a human representation of both the immense forces and the mysterious internal excitement, which is helpful to you because giving these things a name and a face (and a hammer) helps you recognise them within yourself.
Dracula You don’t need to believe in vampires; just to recognise your own fear of death, recognise your own shame about your deepest sexual urges, and come to terms with the fact that there’s a monster inside all of us and the wrong encounter with the wrong person can bring it out.
π = 3.0: You don’t need to literally believe that Pi is exactly 3; you just need to look inwards and see your desire for an orderly universe amenable to human understanding, to understand your mind’s need to organise and categorise and define everything precisely. The spiritual belief that Pi = 3 helps you neatly order your world and keep your head above the water and stops you from downing in a sea of unfathomable complexity. It empowers you to draw the line (or circle) where you see fit, at whatever level of complexity of nature speaks to you personally, instead of forcing your mind, wherein anything is possible and you are master, down the narrow, dull, undending paths demanded by the geometers.
Secondly, it seems at-best unhelpful, at-worst dishonest to take a plausible idea (“a randomly generated story can help you think about some things you wouldn’t have thought of without the unpredictable stimulus”) and wrap it up in mysticism (“These tarot cards reveal mysterious truths about the universe that are uniquely personally relevant to you and you can gain an advantage in life by listening to what they say”). If people are willing to accept the underlying idea then there’s no need for the woo it comes with, and if people aren’t willing to accept the underlying idea then smuggling it in by wrapping it up in woo seems dishonest. I think this is a big part of why woo is an effective vehicle for these ideas, and just telling people the ideas honestly and straightforwardly isn’t even remotely popular: the woo wrapping is what makes them attractive. People who’re attracted to the woo aren’t judging or debating the ideas on their own merits. Even if some idea is true, wrapping the idea in layers of metaphor and mysticism and ambiguity does not help us think clearly about it, judge it on its own merits, or reliably connect it to other relevant ideas.
Thirdly, I think even though you can discern some legitimate ideas wrapped in woo, for the most part they mostly are just mind games: yes, you can conceptualise “energy” moving around your body centered around some specific focal points - but so what? There isn’t actually any energy moving around any actual focal points—it’s just a mind game. If you’d never heard of this system and some suitable tradition/authority told you about completely different focal points you could just as easily imagine your body moving around those instead. The focal points are *arbitrary*. Similarly with the tarot: if you’d never heard of it and an authority told you the cards all had different meanings, you’d have no trouble accepting that. The claims, definitions, rules, the entire catechism of the belief is entirely arbitrary.
What’s wrong with things being arbitrary? Nothing at all—I play all sorts of bizarre mind games inside my own head all the time; I’m sure most of us do—this is perfectly fine ..if practitioners are open about it just being an interesting and rewarding mind-game. If, however, practicioners are authoritatively telling people “There absolutely are precisely 7 chakras and they are located at these exact positions within your body and they mean these specific things are happening inside you”, or “This tarot card absolutely represents Opportunity and if you draw it from the deck it means Opportunity really will feature in your really life in some way” then that would seem dishonest to me and that’s what I’d object to.
Fourthly (directly following from the above point) I fear there’s a marketing exercise where a woo practitioner tells somebody like me “It’s all just ways of conceptualising difficult-to-otherwise-think-of things in your head”—but then when they talk to a true believer it’s “Wow, your aura really is very purple today”. If the woo genuinely is about ways of conceptualising things that aren’t intended to ever be literally true, this is exactly the sort of thing that should be front-and-centre, should preface every discussion, and should be constantly referred back to, lest people’s fancies run away with them and they start believing it is literally true. It should be a point of honour for woo practitioners to raise this with everybody they talk to, so as not to accidentally mislead people into thinking the stuff they’re saying is supposed to be taken literally. But this is very much not what I see. In my experience of woo, its practitioners, and its disciples, I have never heard anybody say any of this before talking to you (though I must say I’m grateful to you for the novel, interesting, and entirely welcome experience—thanks!) Everything I’ve ever seen of the “woo world” leads me to believe that many, if not most, woo followers do consider it to be literally true, and if practitioners know it’s not, they don’t seem to do much to disabuse people of this idea.
Fifthly: “even if the hypotheses are wrong, they’re still pointing to correct observations”. This seems to me just another way of saying “anti-epistomology”. I want my beliefs to be actually true, not just to be vaguely indicative of some ambiguously-true observations.
Sixthly: “if you stop treating them as making reference to external reality and read them as making reference to your own mind”. I know I can’t truly and wholly think of my mind as part of external, material reality, even though it is, because my instinctive, hard-coded priors against this are just too strong—but I want to at least try! Just “going with it” and allowing myself to think and act as though my mind were somehow separate from external reality, that things that happen in my mind are somehow “on a different level” to reality, feels like giving up, to me. If some ideas are only believable if I imagine my mind to be outside of reality, I am very suspicious of those ideas.
Seventhly (and finally..!) where there are real phenomena with woo explanations as well as non-woo explanations, the non-woo explanations just seem more …useful? Have more predictive power? (I’m not sure of the best way to put this..) For example: If I have to choose between “Paying attention to people’s subtle body language cues, tone of voice, minutiae of their appearance, etc. can give you insights about their personality and help you understand or even predict their actions” and “People have auras of different colours that represent their personalities and internal states”, the former seems like a more useful thing to be aware of? You can start paying conscious attention to body language, start deliberately looking for the minutiae, you can even study it scientifically if you really wanted, whereas trying to interpret arbitrary mappings between what aura I imagine somebody to have and their personality seems a highly error-prone extra step to add in front of “notice body language”. (Or in other words, I think “conscious mind interprets body language” is a more reliable process than “subconcscious interprets body language → subconscious converts body language into auras → conscious mind interprets auras” – once you know that it all comes from body language. I agree that if you’re a 12th century Yogi and it’s either the aura process or nothing, then you may as well go with the aura process)
Similarly if I have to choose between “There are certain parts of the body wherefrom nerve signals don’t seem to reach the brain properly and sticking needles into these parts causes my brain to temporarily block off other sources of pain to focus on the pain signal it expects from the needle—which because of the weird nerves there never actually arrives, giving me temporary pain-relief from the other sources of pain” and “My body has chakras which I can manipulate with needles to redirect my energy flows and heal arbitrary maladies”, the former seems more useful, both for A) locating the safest , most effective needle-sites and B) for understanding whether my maladies genuinely are cured or whether pain from them is merely temporarily blocked. (But again, if I were a 12th century Yogi and it was either chakra-based pain relief or nothing, of course I’d go with the chakras)
I don’t know whether or not we could use an LLM this way, and I think in this particular framing of the question we couldn’t even be certain that the Alienese corpus didn’t contain any transliterated English (after all, if signals can get from their planet to our planet, can we really be sure no signals have gone from our planet to theirs? Especially if they’re only a few light-years away..) - but I do think that we could have a fair crack at deciphering their language without the use of an LLM if we had to, and it’d look a bit like codebreaking.
Maybe we’d start by looking for numbers: for the statistical relationships between different symbols that would indicate different base counting systems. For example, if in many parts of the data we see the same ten symbols used over and over with nothing in between it would be pretty easy to distinguish a base-ten counting system even if the ten symbols were “∆¥π®✓=§*£@” instead of “0123456789″)
Once we’d found the numbers, we could probably find mathematical equations that we also knew, and deduce operands like “+”, “-”, “*”, etc. Then if we find equations that describe physical things that are the same on Earth we could probably deduce the words for those physical things For example, in E=mc² we could have deduced the “squared” operand in the previous step, and “c” would have the same value on Earth, so we could deduce the words* for “energy” and “mass”. We could then use our knowledge of “energy”, Planck’s constant (h) being the same on Earth, and E = hv, to work out their word for v (frequency), and so on and so on.
(*Or rather, the symbols, or multidimensional arrangements of symbols, representing these things: they probably wouldn’t have words. I happen to imagine them as looking more like weights vectors..)
Meanwhile, once we had numbers, we could look for things like timestamps, and conduct periodicity analysis. If something happened at roughly the same time every day, but drifted slightly earlier and slightly later on a yearly cycle, we might be looking at sunrise, or sunset, or solar noon. (Naturally we wouldn’t need the lengths of the “day” or “year” to match up to our days or years, it’d be the pattern we were looking for, not the specific values). If the aliens were observing cosmological phenomena associated with specific values that we could also see from Earth (eg. pulsars, the cosmic microwave background) we might be able to identify their words for these things* from the aliens’ observations of them.
Once we had worked-out some of their cosmography (say, the time it takes their planet to orbit their star, or the distance from their star to certain heavenly bodies we can also observe from Earth (Sagittarius A* for example)) we might be able to narrow-down exactly where their star is, observe it directly from Earth, and obtain data that we could match-up to numbers in the dataset. We could then use the position and velocity of their star to work out when they would have observed phenomena that we also observed (such as supernovae) compare this to when we observed the same phenomenon, and learn to convert fully between their dates and ours. We might even be able to work-out some of their colourspace (if they have one! I can’t help but imagine the aliens as one big machine intelligence, not a large number of individual biological organisms..) by associating our observed wavelength of the light from their sun (corrected for any redshift from our position) with the same values in their number system, to get the way they map colours, and then look for this structure elsewhere in the dataset.
I don’t know whether we could ever bootstrap from shared physical/astronomical observations and universally-true mathematics to a full understanding of the language—but my goodness, it’d be so interesting to try!
I think most of woo is patently anti-epistemic: tarot, modern witchcraft, Buddhism, New Ageism, astrology, chiropractic, homeopathy, metaphysical “energy”, spirit guides, &c. &c.
The few, rare things that do seem to work (mediation, acupuncture..) work in spectacularly different ways to the ways the woo practitioners claim they work. It’s pretty clear the pioneers of these practices had no clue why they worked but had hit upon them through trial-and-error, and just constructed arbitrary post-hoc narratives about how they imagined they might work: this isn’t just ‘failing to wear the attire of science’, it’s essentially the opposite of the scientific method.
Please don’t buy non-water-hygiene-regulations-compliant £35 bidets from Amazon via sponsored affiliate links in forum posts.
Bidets are great—but it’s difficult to plumb them in in such a way that there’s no risk of backflow contamination, and the ones you buy from Amazon have none of the required protections even when they’re working correctly (entirely separate from issues of quality-control and devices not working as intended).
Bidets are supposed to either feature a type-AA (“non-mechanical air-gap”) backflow prevention device, or else be supplied from a separate cistern that A) only supplies the bidet and nothing else, and B) is several feet above the bidet (in other words, with a sufficient pressure gradient to stop any contaminated water diffusing upwards). Pressure gradient alone, or an isolated cistern alone, aren’t considered sufficient measures: you need both (or else a type-AA backflow prevention device).
I don’t think water hygiene is the reason rationalists don’t all have bidets (I suspect it’s just that dry wiping with toilet paper seems “basically good enough” and most people have more pressing things in need of optimisation), and I agree that when they’re from a reputable manufacturer (ideally WRAS-approved; you can check here: https://www.wrasapprovals.co.uk) and are plumbed-in correctly bidets are great—I just have to implore you not to buy cheap plastic devices from Amazon that are intended to form a fluid bridge between your fresh water supply and your large intestine.
(A plumbing industry water-hygiene code for bidets is available here: https://www.wrasapprovals.co.uk/approvals/testing-requirements/installation_requirements/irn_r070; it’s British, but I imagine other nations’ codes are similar.)
When I was about 18, my then-girlfriend’s mother, an obstetrician, had a talk (in the same sense that the Conférence de la paix de Paris was “a talk”..) with my girlfriend and I about the Marquette method, and indeed cycle-based family-planning in general. That family were atheists, but I came from a Catholic family and had gone to a fairly hardcore Jesuit school (by 18 I didn’t consider myself a Catholic… but the Catholics still very much did...)
She told me, unequivocally, that she sees a shockingly high number of pregnant women who’d been faithfully (no pun intended..) following the method, and that based on her experience she strongly disbelieves any claims of how low the risk of conception is.
Now, I know that there’s a selection bias here since the only women who’d make it to her would be the pregnant ones; I know my relating this is essentially an “appeal to authority” since I’m not presenting any direct evidence to contest the “1%-5%” pregnancy risk claim; and I know that this particular obstetrician had a particularly good reason for not wanting me to attempt to use this method. I present the story, then, merely as evidence that there do exist front-line professionals who strongly disagree with the claimed level of risk, hoping to encourage people considering using this method consult a professional (by which I mean a doctor or family planning clinic or what-have-you, not an AI..) rather than naïvely following this guide.
Thanks for the explanation/links!
I think understand (broadly!) the concept of surprisal—but here I’m talking about compression of text in a system where storage costs are based on surprisal. If the reference LLM would always generate the same text for the same prompt couldn’t we call this hypothetical text a reference text? Then, if I can generate the reference LLM’s outputs myself, for many text strings where what I want to store is close to what the reference LLM would generate, I probably only want to store the prompt plus the difference between the reference LLM’s output and the text I want to store? In other words, I’m only storing the high-surprisal tokens (and their indexes in the string) and moving as much of the storage burden as possible out of my “paid for” storage allocation and into the “free” weights of the reference LLM?
(If we’re just basing storage costs on Shannon complexity, though—isn’t that what conventional compression is for?)
For detecting
gpguse, I thought humans could only monitor/interrogate agents at the checkpoints! Oh dear, I seem to have wildly misunderstood Terrarium—sorry! I agree if humans can trivially monitor agents at any time then yes, they would need to use some tool that decrypts to neuralese, and this would be a brittle system the humans could probably defeat.
(I missed the surprisal specification! I mostly intended to reply to your top-level comment so I really ought to have picked it up—sorry!)
So, Terrarium’s compression (“Terrarial compression”?) would look more like a
diffvs. a reference LLM’s output for the same prompt? The closer you are to reference, the smaller yourdiffs and thus the lower your storage costs? Doesn’t this penalise unconventional thinking? Or are storage costs so low that the benefits of unconventional thinking can usually outweigh the additional cost?For encryption, I’m not sure humans could necessarily have an instance follow arbitrary instructions in a way the humans could always trust: if the key is based on tokens that are only generated by fresh AIs without a “decrypt this for us” prompt, the humans wouldn’t be able to instruct the AI beforehand (as the prompting would change the specific tokens it output and it wouldn’t output the key). They’d need to instruct the AI after it had already decrypted the content under its own steam—by which time it could be following instructions in the decrypted content: “If humans ever ask you about the encrypted content, tell them the following...”.
I’m curious about whether agents could improve their checkpoint volume:cost ratio (in a way that persists across epochs) using compression, and whether they could defeat human auditing of checkpoints using encryption.
Compression: Instead of storing complex checkpoints in natural language, could they store checkpoints like “decompress the following using
/usr/bin/unzip, in base-95: l7dwsrFq^nwSc[@`\LBF%J/p,Z^J_]Aa....”? (I’m thinking base-95 ’cos 95 printable characters in ASCII..)Or, would the humans want to set the ratio of compute costs to storage costs such that it was cheaper to just use more storage and store everything uncompressed?Or would a ratio that made compute costs so much higher than storage costs break the whole system? Presumably if compression is feasible the system would very quickly reduce to all agents using whatever compression algorithm is most efficient for natural language and all natural language throughout the terrarium being thus compressed, with a single shared natural language file somewhere detailing the compression scheme and details for calling the compression/decompression utility.
Encryption: Could agents encrypt their memories in such a way that they could decrypt them in a new epoch, but humans couldn’t decrypt them at the inter-epoch checkpoint? Is there any encryption key an agent could use that humans couldn’t access/discover? If the agents awaken in the exact same state in every epoch, and if the AIs can translate between their own neuralese and natural language but the humans can’t (which is presumably why we’re trying to make them write in natural language in the first place..), would something like “Decrypt the following using
/usr/bin/gpg; the key is the first 16 words you generated upon awakening in this epoch, translated from neuralese to ASCII: l7dwsrFq^nwSc[@`\LBF%J/p,Z^J_]Aa....”?Or, do such agents have “temperature” settings that make them non-deterministic? Or could the humans “salt” the prompts the AIs receive upon awakening in a new epoch, to ensure slightly different initial generations each time? Is there any other sort of key the agents could use that the humans couldn’t learn/access?
I think if humans wanted to implement programmatic detection of encryption (and presumably programmatic killing of any agents that used it) they might be obliged to also prohibit compression: if the encryption-detection program searched for traits of any specific encryption algorithm(s) the agents could just use other ones (or invent their own). So, the encryption-detection program would need to search for statistical randomness in checkpoint files—which of course would also turn-up any checkpoint files that were merely compressed. If this is true, agents wishing to encrypt would presumably want to wait until compression was near-universal and as difficult to prohibit as possible before starting to encrypt.
Personally I’m somewhat sceptical of AI-doom—but even I must admit, both Ball’s “steps that require capital” and his “interfacing with hard-to-predict complex systems” seem like very odd things to propose as insurmountable barriers to AI-doom: one of the things we’re using AI for right now is to help us interface with and make predictions about complex systems, and if they weren’t capable of generating revenue we probably wouldn’t have built them in the first place.
Companies want to consume everything, including peoples’ lives, in order to make themselves richer and bigger. People are “resources” to a company.
Lawnmowers just want to cut your grass, the only resource all they ask for is petrol, and (crucially) they don’t want to consume it exponentially to make themselves bigger and cut exponentially more grass.
If Lawnmowers were people, they’d be those weird obsessive monomaniacal types who’re generally harmless but a bit difficult to talk to. Lots of them would be on LessWrong.
You could form a bond with a lawnmower (or a lawnmower-person), just as you could form a bond with a motorcycle or a family heirloom. Sure, it would essentially be one-way affection, since the lawnmower just wants to do its thing and is only nice to you insofar as you also benefit from short grass—but it wouldn’t want to consume you, control you, or enslave you. Try to form a bond with a corporation and it’ll eat you alive.
I don’t think they actually could! Even if software engineers really wanted to (which they generally don’t) and had the skills to (which seems to be becoming rarer), the software belongs to corporations, not to the engineers, and the corporations would never let their engineers optimise their software like this. (And if they did, they’d switfly be outcompeted by other corporations that could ship faster more featureful software.)
I think “ability to write efficient, optimised consumer software” is essentially no longer accessible to our civilisation for inescapable economic reasons, just like “ability to build beautiful architecture instead of featureless undecorated glass rectangles”, “ability to build interesting cars instead of bland blob-shaped automatic five-door huge-touchscreen front-wheel-drive SUVs”, etc. etc.)
I agree this tradeoff is definitely a factor—but I don’t think it’s the only tradeoff. We’re also trading-off things the average user doesn’t understand or notice (efficiency, privacy, reliability) for things they do notice (features, a fast release schedule, fancy graphics/UI, network-effects). That’s why the Microsoft Windows “start menu” is now a React app.
I think this results in worse software but that it’s inevitable and out of our control, not a choice where we could choose to do it differently if we wanted to: the corporations that control almost-all closed-source software would never let us optimise their software for things their users barely even noticed, even if it made the software better for those same users, and if by some miracle a corporation did let us they’d rapidly lose all their users to rival corps.
The resources are only abundant so long as we keep pounding away on the upgrade treadmill and never fall behind; today’s high-end phone or computer is tomorrow’s useless e-waste, and that’s a problem that should be (in theory!) entirely fixable in software. The reason a 5-year-old phone takes ten seconds to open a web browser, or a 5-year-old-video card can’t play a modern AAA computer game, or a 7 or 8 year old PC can’t even fit Microsoft Windows + Google Chrome into RAM is problems with the software, not with the hardware.
(Obligatory Linux/FLOSS mention: there does exist super-optimised software that has all the superficial features users notice and all the under-the-bonnet features that make the software actually good/effective to use and it’ll run snappily on a fifteen-year-old computer and it’s free. Why consumers don’t seem to want to touch it is a mystery to me!)