I may have discovered an interesting tool against lethargy and depression [1]: This morning, in place of my usual caffeine pill, I made myself a cup of hot chocolate (using pure cacao powder / baking chocolate from the supermarket), which made me very energetic (much more energetic than usual), which stood in sharp contrast to the past 4 days, which have been marked by lethargy and intense sadness. Let me explain:
Last night, I was reflecting on the fact that one of the main components of chocolate is theobromine, which is very similar in structure to caffeine (theobromine is the reason why chocolate in poisonous to dogs & cats, for reasons similar to how caffeine was evolved to kill insects that feed on plants), and is known to be the reason why eating chocolate makes people happy. Since I have problems with caffeine, but rely on it to have energy, I figured it would be worthwhile to try using chocolate instead as a morning pick-me-up. I used baking chocolate instead of Nesquick or a hot chocolate packet because I’m avoiding sugar these days, and I figured having as pure chocolate as possible would be ideal for my experiment.
I was greeted with pleasant confirmation when I became very alert almost immediately after starting to drink the chocolate, despite having been just as lethargic as the previous days until I drank the chocolate. It’s always suggestive when you form a hypothesis based on facts and logic, then test the hypothesis, and exactly what you expected to happen, happens. But of course, I can’t be too confident until I try repeating this experiment on future days, which I will happily be doing after today’s success.
[1]: There are alternative hypotheses for why today was so different from the previous days: I attended martial arts class, then did some photography outside yesterday evening, which meant I got intense exercise, was around people I know and appreciate, and was doing stuff with intentionality, all of which could have contributed to my good mood today. There’s also the possibility of regression to the mean, but I’m dubious of this since today was substantially above average for me. I also had a (sugar-free) Monster later in the morning, but that was long after I had noticed being unusually alert, and now I have a headache that I can clearly blame on the Monster (Caffeine almost always gives me a headache) [1a].
[1a]: I drink energy drinks because I like the taste of them, not for utilitarian reasons. I observe that caffeine tends to make whatever contains it become deeply associated with enjoyment and craving, completely separated from the alertness-producing effects of the chemical. A similar thing happened with Vanilla Café Soylent [1b], which I absolutely hated the first time I tried it, but a few weeks later, I had deep cravings for, and could not do without.
[1b]: Sidenote, the brand Soylent has completely gone to trash, and I would not recommend anyone buy it these days. Buy Huel or Plenny instead.
I think I want to try this. What was your hot cocoa recipe? Did you just mix it with hot water? Milk? Cream? Salt? No sugar, I gather. How much? Does it taste any better than coffee? I want to get a sense of the dose required.
Just saw this. I used approximately 5 tablespoons of unsweetened cocoa powder, mixed with warm water. No sweetener, no sugar, or anything else. It’s bitter, but I do prefer the taste over coffee.
I just tried it. I did not enjoy the taste, although it does smell chocolatey. I felt like I had to choke down the second half. If it’s going to be bitter, I’d rather it were stronger. Maybe I didn’t stir it enough. I think I’ll use milk next time. I did find this: https://criobru.com/ apparently people do brew cacao like coffee. They say the “cacao” is similar to cocoa (same plant), but less processed.
I found this abstract suggesting that theobromine doesn’t affect mood or vigilance at reasonable doses. But this one suggests that chocolate does.
Subjectively, I feel that my cup of cocoa today might have reduced my usual lethargy and improved my mood a little bit, but not as dramatically as I’d hoped for. I can’t be certain this isn’t just the placebo effect.
The first linked study tests 100, 200, and 400 mg Theobromine. A rough heuristic based on the toxic doses of the two chemicals suggests 750 mg, maybe a little more (based on subjective experience) is equivalent to 100mg caffeine or a cup of coffee (this is roughly the dose I’ve been using each day), so I wouldn’t expect a particularly strong effect for the first two. The 400 mg condition does surprise me; the sample size of the study is small (n = 24 subjects * 1 trial per condition), so the fact that it failed to find statistical significance shouldn’t be too big of an update, though.
Can you clarify your Soylent anti-recommendation? I don’t use it as an actual primary nutrition, more as an easy snack for a missed meal, once or twice a week. I haven’t noticed any taste difference recently—my last case was purchased around March, and I pretty much only drink the Chai flavor.
A] Meal replacements require a large amount of trust in the entity that produces it, since if there’s any problems with the nutrition, that will have big impacts on your health. This is less so in your case, where it’s not a big part of the nutrition, but in my case, where I ideally use meal replacements as a large portion of my diet, trust is important.
B] A few years ago, Rob Rhinehart, the founder and former executive of Soylent, parted ways with the company due to his vision conflicting with the investor’s desires (which is never a good sign). I was happy to trust Soylent during the Rhinehart era, since I knew that he relied on his creation for his own sustenance, and seemed generally aligned. During that era, Soylent was very effective at signaling that they really cared about the world in general, and people’s nutrition in general. All the material that sent those signals no longer exists, and the implicit signals (e.g. the shape of and branding on the bottles, the new products they are developing [The biggest innovation during the Rhinehart era was caffeinated Soylent, now the main innovations are Bridge and Stacked, products with poor nutritional balance targeted at a naïve general audience, a far cry from the very idea of Complete Food], and the copy on their website) all indicate that the company’s main priority is now maximizing profit, without much consideration as to the (perceived) nutritional value of the product. In terms of product, the thing is probably still fine (though I haven’t actually looked at the ingredients in the recent new nutritional balance), but in terms of incentives and intentions, the management’s intention isn’t any better than, say, McDonald’s or Jack In The Box.
Since A] meal replacements require high trust and B] Soylent is no longer trustworthy: I cannot recommend anyone use Soylent more than a few times a week, but am happy to recommend Huel, Saturo, Sated, and Plenny, which all seem to still be committed to Complete Food.
(As far as flavour, I know I got one box with the old flavor after the recent flavor change, the supply lines often take time to get cleared out, so it’s possible you got a box of the old flavor. I don’t actually mind the new flavour, personally)
I recommend Ample (lifelong subscriber). It has high quality ingredients (no soy protein), fantastic macro ratios (5/30/65 - Ample K), and an exceptional founder.
I’m no longer sure that I buy dutch book arguments, in full generality, and this makes me skeptical of the “utility function” abstraction
Thesis: I now think that utility functions might be a pretty bad abstraction for thinking about the behavior of agents in general including highly capable agents.
[Epistemic status: half-baked, elucidating an intuition. Possibly what I’m saying here is just wrong, and someone will helpfully explain why.]
Over the past years, in thinking about agency and AI, I’ve taken the concept of a “utility function” for granted as the natural way to express an entity’s goals or preferences.
Of course, we know that humans don’t have well defined utility functions (they’re inconsistent, and subject to all kinds of framing effects), but that’s only because humans are irrational. To the extent that a thing acts like an agent, it’s behavior corresponds to some utility function. That utility function might not be explicitly represented, but if an agent is rational, there’s some utility function that reflects it’s preferences.
Given this, I might be inclined to scoff at people who scoff at “blindly maximizing” AGIs. “They just don’t get it”, I might think. “They don’t understand why agency has to conform to some utility function, and an AI would try to maximize expected utility.”
Currently, I’m not so sure. I think that talking in terms of utility functions is biting a philosophical bullet, and importing some unacknowledged assumptions. Rather than being the natural way to conceive of preferences and agency, I think utility functions might be only one possible abstraction, and one that emphasizes the wrong features, giving a distorted impression of what agents, in general, are actually like.
I want to explore that possibility in this post.
Before I begin, I want to make two notes.
First, all of this is going to be hand-wavy intuition. I don’t have crisp knock-down arguments, only a vague discontent. But it seems like more progress will follow if I write up my current, tentative, stance even without formal arguments.
Second, I don’t think utility functions being a poor abstraction for agency in the real world has much bearing on whether there is AI risk. As I’ll discuss, it might change the shape and tenor of the problem, but highly capable agents with alien seed preferences are still likely to be catastrophic to human civilization and human values. I mention this because the sentiments expressed in this essay are casually downstream of conversations that I’ve had with skeptics about whether there is AI risk at all. So I want to highlight: I think I was mistakenly overlooking some philosophical assumptions, but that is not a crux.
Is coherence overrated?
The tagline of the “utility” page on arbital is “The only coherent way of wanting things is to assign consistent relative scores to outcomes.”
This is true as far as it goes, but to me, at least, that sentence implies a sort of dominance of utility functions. “Coherent” is a technical term, with a precise meaning, but it also has connotations of “the correct way to do things”. If someone’s theory of agency is incoherent, that seems like a mark against it.
But it is possible to ask, “What’s so good about coherence anyway? Maybe
The standard reply of course, is that if your preferences are incoherent, you’re dutchbookable, and someone will pump you for money.
But I’m not satisfied with this argument. It isn’t obvious that being dutch booked is a bad thing.
Suppose I tell you that I prefer pineapple to mushrooms on my pizza. Suppose you’re about to give me a slice of mushroom pizza; but by paying one penny ($0.01) I can instead get a slice of pineapple pizza (which is just as fresh from the oven). It seems realistic to say that most people with a pineapple pizza preference would probably pay the penny, if they happened to have a penny in their pocket. 1
After I pay the penny, though, and just before I’m about to get the pineapple pizza, you offer me a slice of onion pizza instead—no charge for the change! If I was telling the truth about preferring onion pizza to pineapple, I should certainly accept the substitution if it’s free.
And then to round out the day, you offer me a mushroom pizza instead of the onion pizza, and again, since I prefer mushrooms to onions, I accept the swap.
I end up with exactly the same slice of mushroom pizza I started with… and one penny poorer, because I previously paid $0.01 to swap mushrooms for pineapple.
This seems like a qualitatively bad behavior on my part.
Eliezer asserts that this is “qualitatively bad behavior.” But I think that this is biting a philosophical bullet.
As an intuition pump: In the actual case of humans, we seem to get utility not from states of the world, but from changes in states of the world. So it isn’t unusual for a human to pay to cycle between states of the world.
For instance, I could imagine a human being hungry, eating a really good meal, feeling full, and then happily paying a fee to be instantly returned to their hungry state, so that they can enjoy eating a good meal again.
This is technically a dutch booking (which do they prefer, being hungry or being full?), but from the perspective of the agent’s values there’s nothing qualitatively bad about it. Instead of the dutchbooker pumping money from the agent, he’s offering a useful and appreciated service.
Of course, we can still back out a utility function from this dynamic: instead of having a mapping of ordinal numbers to world states, we can have one from ordinal numbers to changes from world state to another.
But that just passes the buck one level. I see no reason in principle that an agent might have a preference to rotate between different changes in the world, just as well as rotating different between states of the world.
But this also misses the central point. I think you can always construct a utility function that represents some behavior. But if one is no longer compelled by dutch book arguments, this begs the question of why we would want to do that. If coherence is no longer a desiderata, it’s no longer clear that a utility function is that natural way to express preferences.
And I wonder, maybe this also applies to agents in general, or at least the kind of learned agents that humans are likely to build via gradient descent.
Maximization behavior
I think this matters, because many of the classic AI risk arguments go through a claim that maximization behavior is convergent. If you try to build a satisficer, there are a number of pressures for it to become a maximizer of some kind. (See this Rob Miles video, for instance)
I think that most arguments of that sort depend on an agent acting according to an expected utility maximization framework. And utility maximization turns out not to be a good abstraction for agents in the real world, I don’t know if these arguments are still correct.
I posit that straightforward maximizers are rare in the multiverse, and that most evolved or learned agents are better described by some other abstraction.
If not utility functions, then what?
If we accept for the time being that utility functions are a warped abstraction for most agents, what might a better abstraction be?
I don’t know. I’m writing this post in the hopes that others will think about this question and perhaps come up with productive alternative formulations.
I’ll post some of my half-baked thoughts on this question shortly.
Utility functions are especially problematic in modeling behaviour for agents with bounded rationality, or those where there are costs of reasoning. These include every physically realizable agent.
For modelling human behaviour, even considering the ideals of what we would like human behaviour to achieve, there are even worse problems. We can hope that there is some utility function consistent with the behaviour we’re modelling and just ignore cases where there isn’t, but that doesn’t seem satisfactory either.
I’ve long been somewhat skeptical that utility functions are the right abstraction.
My argument is also rather handwavy, being something like “this is the wrong abstraction for how agents actually function, so even if you can always construct a utility function and say some interesting things about its properties, it doesn’t tell you the thing you need to know to understand and predict how an agent will behave”. In my mind I liken it to the state of trying to code in functional programming languages on modern computers: you can do it, but you’re also fighting an uphill battle against the way the computer is physically implemented, so don’t be surprised if things get confusing.
And much like in the utility function case, people still program in functional languages because of the benefits they confer. I think the same is true of utility functions: they confer some big benefits when trying to reason about certain problems, so we accept the tradeoffs of using them. I think that’s fine so long as we have a morphism to other abstractions that will work better for understanding the things that utility functions obscure.
Alyssa Vance asked, “What great classes could be taught using ideas that might be seen on the Internet, but aren’t part of a standard curriculum yet?”.
My answer:
Deep learning (especially recent ideas like graph neural nets, transformers, GPT-3, deep learning applied to science), online advertising, cryptocurrency, contemporary cybersecurity, the internet in China (seems valuable for people outside China to understand), CRISPR, human genetics (e.g. David Reich’s work), contemporary videogames (either from technological or cultural/artistic perspective), contemporary TV, popular music in the age of Spotify, internet culture (e.g. Reddit, social media, memes).
superforecasting: the best class would involve people actually doing forecasting on something like Metaculus or on a prediction market with financial bets.
real-world practical applications of deep learning: considering the technical and economic/ethical aspects
immersive sociology/anthropology of internet cultures: you can’t do traditional anthropological fieldwork in an undergrad class but you can lurk or participate in any one of innumerable online subcultures.
immersion in country X: using machine translation, it’s possible to consume newspapers, Twitter, TV, etc from another country without speaking the language. someone who knows country X well could build an engaging class around this.
cooking: there are not many college courses on cooking (harvard’s famous class is an exception). youtube is pretty great for demonstrations.
How would this be different from movie music? There are some examples of music dynamically adapting according to what happens, but most games don’t go very deep into that. Stylistically, of course, due to the historical separation between the mediums, video game music often sounds different from movie music. But practically I suspect there’s not much difference.
Movie music wasn’t listed either, just popular music.
That said,
But practically I suspect there’s not much difference.
musical numbers in games might be intended to cover more time, and be more flow workable. But this might start to get into, ‘What type of game?’, ‘What type of movie?‘, and ‘What part of the work is the song in?’ (They might seem most similar in the trailers for each, because there they are serving similar roles/purposes.)
The lethal dose of caffeine in adult humans is approximately 10 grams, while the lethal dose of theobromine (the main psychoactive chemical in chocolate, nearly identical structurally to caffeine, with similar effects) in humans is 75 grams (this is much lower in most animals, which is why you should never give chocolate to your pets). This can motivate a rough heuristic of 7.5 mg theobromine is roughly equal to 1 mg caffeine, and 750 mg theobromine is equivalent to one cup of coffee.
Therefore, to replace coffee with cocoa or chocolate, 6 spoons of unsweetened cocoa powder should replace a cup of coffee. 11 cups of hot chocolate (that’s a lot) or 2 bars of dark chocolate should also work.
I’ve long been aware of the concept of a “standard drink”, a unit for measuring how much alcohol a person has had, regardless of what they are drinking, so one “drink” of wine contains less liquid than one “drink” of beer, but more than one drink of vodka. When I started experimenting with chemicals other than ethanol, I intuitively wanted to extend this notion to other chemicals. For example, in my mind, I roughly equate 10 mg of Tetrahydracannabinol with one drink of ethanol. While the effects of these two chemicals are quite different, and work in different ways, they both have relaxing and depressant (not “depressing”) effects, so there is some meaningful comparison—if I want to incapacitate myself to a certain extent, I can use the concept of an “depressant unit” to calculate a dose of either THC or ethanol, or similarly with diphenhydramine (ZZZQuil) or Nyquil.
Clearly, in most cases, I would not want to compare the strength of an alcoholic beverage with the strength of a caffeinated beverage. But I would want to be able to use a “stimulating unit” to compare, say, amounts of caffeine to amounts of theobromine (cocoa) or to other stimulating chemicals (for example, Adderall).
Another unit that would be worth using would be an “entheogenic unit”, which would allow one to compare doses of LSD, Psilocybin, THC (regarding its quasi-psychedelic, not depressant qualities), and so on, in terms of their ability to change the way one thinks.
Question: Is it possible to incorporate Caffeine into DNA? Caffeine is structurally similar to Adenine, one of the four DNA nucleobases (and the A in ATCG). But looking at the structure, the hexagonal ring (which is the part of the DNA that bonds A to T and C to G) doesn’t look very promising—there are two oxygen atoms that can bond, but they are a bit too far apart, and there are no hydrogens, and since DNA is held together by hydrogen bonds, the hydrogen will have to be provided by whatever it is paired to. Theobromine looks more promising, since a CH3 group is replaced by an H (otherwise it is identical to Caffeine), which provides a continuous run of bondable groups, and the H can be used for hydrogen bonding.
Probably for either Theobromine or Caffeine, they would have to be paired with another base that is not one of the usual ATCG bases, which is specially chosen to complement the shape of the molecules.
It’s now been about two years since I started seriously blogging. Most of my posts are on Lesswrong, and the most of the rest are scattered about on my substack and the Effective Altruist Forum, or on Facebook. I like writing, but I have an impediment which I feel impedes me greatly.
In short: I often post garbage.
Sometimes when I post garbage, it isn’t until way later that I learn that it was garbage. And when that happens, it’s not that bad, because at least I grew as a person since then.
But the usual case is that I realize that it’s garbage right after I’m done posting it, and then I keep thinking, “oh no, what have I done!” as the replies roll in, explaining to me that it’s garbage.
Most times when this happens, I just delete the post. I feel bad when this happens because I generally spend a lot of time writing and reviewing the posts. Some of the time, I don’t delete the post because I still stand by the main thesis, although the delivery or logical chain of reasoning was not very good and so I still feel bad about it.
I’m curious how other writers deal with this problem. I’m aware of “just stop caring” and “review your posts more.” But, I’m sometimes in awe of some people who seem to consistently never post garbage, and so maybe they’re doing something right that can be learned.
Sometimes I send a draft to a couple people before posting it publicly.
Sometimes I sit on an idea for a while, then find an excuse to post it in a comment or bring it up in a conversation, get some feedback that way, and then post it properly.
I have several old posts I stopped endorsing, but I didn’t delete them; I put either an update comment at the top or a bunch of update comments throughout saying what I think now. (Last week I spent almost a whole day just putting corrections and retractions into my catalog of old posts.) I for one would have a very positive impression of a writer whose past writings were full of parenthetical comments that they were wrong about this or that. Even if the posts wind up unreadable as a consequence.
I have a hope that with more practice, this gets better.
Not just practice, but also noticing what other people do differently. For example, I often write long texts, which some people say is already a mistake. But even a long text can be made more legible if it contains section headers and pictures. Both of them break the visual monotonicity of the text wall. This is why section headers are useful even if they are literally: “1”, “2″, “3”. In some sense, pictures are even better, because too many headers create another layer of monotonicity, which a few unique pictures do not. Which again suggests that having 1 photo, 1 graph, and 1 diagram is better than having 3 photos. I would say, write the text first, then think about which parts can be made clearer by adding a picture.
There is some advice on writing, by Stephen King, or by Scott Alexander.
If you post a garbage, let it be. Write more articles, and perhaps at the end of a year (or a decade) make a list “my best posts” which will not include the garbage.
BTW, whatever you do, you will get some negative response. Your posts on LW are upvoted, so I assume they are not too bad.
Also, writing can be imbalanced. Even for people who only write great texts, some of them are more great and some of them are less great than the others. But if they deleted the worst one, guess what, now some other articles is the worst one… and if you continue this way, you will stop with one or zero articles.
I’m 65% moving back to us in a month or two but haven’t lived there in like 5 years so am not sure what to expect nor where I’d like to go.
I’m mainly optimizing for friend-making/having/meeting (IRL) though I’m not sure how much I care about that since I haven’t had much chance to do that for the last few years. though also trees and stuff are nice too
I’m not really sure how to choose between cities in terms of satisfying this. vaguely seems like SF would be cool and have many cool people that would be nice to meet but is also expensive
Any recommendations on figuring out where to stay?
Shared with permission, a google doc exchange confirming Eliezer still finds the arguments for alignment optimism, slower takeoffs, etc. unconvincing:
Daniel Filan: I feel like a bunch of people have shifted a bunch in the type of AI x-risk that worries them (representative phrase is “from Yudkowsky/Bostrom to What Failure Looks Like part 2 part 1”) and I still don’t totally get why.
Eliezer Yudkowsky: My bitter take: I tried cutting back on talking to do research; and so people talked a bunch about a different scenario that was nicer to think about, and ended up with their thoughts staying there, because that’s what happens if nobody else is arguing them out of it.
That is: this social-space’s thought processes are not robust enough against mildly adversarial noise, that trying a bunch of different arguments for something relatively nicer to believe, won’t Goodhart up a plausible-to-the-social-space argument for the thing that’s nicer to believe. If you talk people out of one error, somebody else searches around in the space of plausible arguments and finds a new error. I wasn’t fighting a mistaken argument for why AI niceness isn’t too intractable and takeoffs won’t be too fast; I was fighting an endless generator of those arguments. If I could have taught people to find the counterarguments themselves, that would have been progress. I did try that. It didn’t work because the counterargument-generator is one level of abstraction higher, and has to be operated and circumstantially adapted too precisely for the social-space to be argued into it using words.
You can sometimes argue people into beliefs. It is much harder to argue them into skills. The negation of Robin Hanson’s rosier AI scenario was a belief. Negating an endless stream of rosy scenarios is a skill.
Caveat: this was a private reply I saw and wanted to share (so people know EY’s basic epistemic state, and therefore probably the state of other MIRI leadership). This wasn’t an attempt to write an adequate public response to any of the public arguments put forward for alignment optimism or non-fast takeoff, etc., and isn’t meant to be a replacement for public, detailed, object-level discussion. (Though I don’t know when/if MIRI folks plan to produce a proper response, and if I expected such a response soonish I’d probably have just waited and posted that instead.)
It didn’t work because the counterargument-generator is one level of abstraction higher, and has to be operated and circumstantially adapted too precisely for the social-space to be argued into it using words.
You could tell people to read certain books, but how they read and what they get out of them is quite unknown, a lot of times even to the readers themselves. There is no exact formula for how to learn things. You could take things at face value, and those usually only work for guided courses. As for reading a book, you have to pause and think about the stuff you read, and there is no general guidance on how deep of a thought tangent you want to explore. Most books are read sequentially, but optimizing what you get out of the things you read, doesn’t have to be books, is hardly a sequential task.
Also not all knowledge is expressible in linguistic terms. A typical example is kinesthetic knowledge, like muscle memories. Also as you read more, the speed of your reading increases. Usually they associate these type of information as skills. There is a clear distinction when we talk about knowledge and skill, but by their very nature, they might not be so distinct as we like to think of them. Musical improvisation is a mixture of multiple knowledge domains.
While your comment was clearly written in good faith, it seems to me like you’re missing some context. You recommend that EY recommend that the detractors read books. EY doesn’t just recommend people read books. He wrote the equivalent of like three books on the subjects relevant to this conversation in particular which he gives away for free. Also, most of the people in this conversation are already big into reading books.
It is my impression he also helped establish the Center for Applied Rationality, which has the explicit mission of training skills. (I’m not sure if he technically did but he was part of the community which did and he helped promote it in its early days.)
It is my impression he also helped establish the Center for Applied Rationality, which has the explicit mission of training skills. (I’m not sure if he technically did but he was part of the community which did and he helped promote it in its early days.)
Eliezer was involved with CFAR in the early days, but has not been involved since at least 2016.
Thinking out loud about some arguments about AI takeoff continuity:
If a discontinuous takeoff is more likely to be local to a particular agent or closely related set of agents with particular goals, and a continuous takeoff is more likely to be global, that seems like it incentivizes the first agent capable of creating a takeoff to make sure that that takeoff is discontinuous, so that it can reap the benefits of the takeoff being local to that agent. This seems like an argument for expecting a discontinuous takeoff and an important difference with other allegedly analogous technologies.
I have some trouble understanding the “before there are strongly self-improving AIs there will be moderately self-improving AIs” argument for continuity. Is there any reason to think the moderate self-improvement ability won’t be exactly what leads to the strong self-improvement ability? Before there’s an avalanche, there’s probably a smaller avalanche, but maybe the small avalanche is simply identical to the early part of the large avalanche.
Several very important to me people whom I love told me that they would rather die than live even a few hundred years or indefinitely, that they would not choose cryopreservation if life extension capabilities aren’t advanced enough by their “natural time”, and so on, when I asked them how they felt about immortality (scenario was: imagine that humanity figures out how to be immortal and there are no restrictions, anyone can have it if they want it, do you take it yes or no?).
There’s too much deathism in this world, aahhhhhhhhh. I’ve already started to mourn those people, and it hurts so fucking much, it literally is keeping me awake tonight...I was meditating then trying to sleep and reminders of their choice bubbled up and now here I am, typing away. Crying about and mourning the loss of loved ones who haven’t died yet, but ultimately said that that was their preference over life extension / immortality.
It hurts so bad.
Writing this felt helpful somewhat, at least I’ve channeled those feelings and temporarily diminished their intensity, somewhat...well, back to trying to sleep.
Talk is cheap. Someone who says “I want to die eventually” isn’t actually invested in the answer—it’s just them justifying to themselves why they’re not exercising, eating right, and otherwise planning for a long future.
This is very uncharitable. For many people, living forever is simply not a realistic option. Heck, many rationalists give it a chance around 10%, and that already involves a lot of belief in progress, which many people don’t have.
Also, people are not automatically strategic. For example, religious people believe that sin can bring them eternity in hell, and they still keep sinning.
You’re assuming a lot about other peoples’ experiences and motivations, the internal experience that my aforementioned love ones have described to me looks not at all similar to what you said. While their internal experiences and their desire for eventual death are alien thought processes, emotions, and experiences to me, I do notice that the people-space of people who prefer eventual death to immortality contains a pretty wide variety of reasonings and internal experiences for why they prefer that eventual death...including surprisingly well thought out and sophisticated and logically coherent answers. Some people genuinely want to die eventually rather than live indefinitely, and that mindset / preference is so alien to mine own that it’s a struggle to accept that people believe such things and have such preferences, but I keep encountering people who do so it seems to be true.
However, I can see how what you said might be an internal experience for some people within people-space, it does check out and pass my anecdotal experience test at least (I’ve encountered some people who, per their description of internal experience, are likely similar to the mindset you described).
I like to ask people their preferences on this matter, so I’ve heard a lot different answers to the “death vs immortality” questions, and while I’ve encountered some people who have a strong or neutral preference for immortality, I’ve encountered a surprisingly high amount of people who would prefer death, and that sucks.
On average, as you grow older, your health gets worse. I suspect that many people make an interpolation of this process, and their idea of a 1000 years old person is kind of a zombie in a wheelchair screaming in pain. Arguably, a fate worse than death. (And if you are religious, or unable to talk, then choosing death is not even an option here.) So perhaps it would be better to talk about “more decades of youth” rather than extension of life-as-we-know-it.
Another possible fear is of waking up in a bad future. (Which again may be worse than death, and suicide may not be an option.) I have no idea what are the actual probabilities here.
I suspect that a good deal of people make that assumption too, about what living past a certain age would be like. Or the bad future scenario. I’ve encountered people who believe either or both things, but once I frame the question and scenario as immortality with perpetual youth then the first concern almost always disappears. The majority of people I ask the question / scenario to, keep bringing up concerns about population and where all these immortality people are going to live. That’s not really something I’m worried about, because the universe is very large and an assumption I make is that humanity would spread throughout the stars if immortal, but I don’t have a great specific answer regarding the population concerns people have mentioned.
Actually, religious people with strong faith in their religion’s conception of an afterlife are the most likely to choose eventual death over immortality in my experience, because they believe that one their time is up on Earth, they simply die and go to their religion’s afterlife, and they find that very strongly preferable to living perpetually in the material world.
For the religious ones, perhaps a good frame would be “young for 1000 years”, so that they can still enjoy the afterlife. More time to do the earthly stuff, and the afterlife is supposed to be infinite anyway.
Population… the best case would be something like “people are young forever, but they can only have kids during the first few decades”. Anyway, with exponential growth we would run out of resources even without immortality. And if there is ever a law against exponential growth, like “only 2 kids per a pair of adults”, then immortality would mean a linearly growing population, which should be doable somehow. But yeah, this is difficult to explain, and requires some faith in either space travel or linear increases in food production.
I know some will, but that’s too optimistic and ignores the preferences / experiences of a huge amount of people, because there are categories of people who prefer death over immortality for whom the aging process doesn’t factor in to their choice on that matter. Especially people with strong faith in their religion’s afterlife.
What I mean is that they haven’t really considered it. As I’m sure you’re aware, your mind does not work like most people’s. When most people consider the question of whether they’d like to die someday, they’re not really thinking about it as if it were a real option. Even if they give detailed, logically coherent explanations for why they’d like to die someday, they haven’t considered it in in near mode.
I am very confident of this—once they have the option, they will not choose to die. Right now they see it as just an abstract philosophical conversation, so they’ll just say whatever sounds nice to them. For a variety of reasons, “I’d like to die someday; being immortal doesn’t appeal to me” sounds wise to a lot of people. But once they have the actual option to not die? Who’s going to choose to die? Only people who are suicidally depressed or who are in very extreme cults will choose that.
I used to think a primary job of a meetup/community organizer was to train their successor, and develop longterm sustainability of leadership.
I still hold out for that dream. But, it seems like a pattern is:
1) community organizer with passion and vision founds a community
2) they eventually move on, and pass it on to one successor who’s pretty closely aligned and competent
3) then the First Successor has to move on to, and then… there isn’t anyone obvious to take the reins, but if no one does the community dies, so some people reluctantly step up. and....
...then forever after it’s a pale shadow of its original self.
For semi-branded communities (such as EA, or Rationality), this also means that if someone new with energy/vision shows up in the area, they’ll see a meetup, they’ll show up, they’ll feel like the meetup isn’t all that good, and then move on. Wherein they (maybe??) might have founded a new one that they got to shape the direction of more.
I think this also applies to non-community organizations (i.e. founder hands the reins to a new CEO who hands the reins to a new CEO who doesn’t quite know what to do)
So… I’m kinda wondering if second-generation successors should just… err on the side of shutting the thing down when they leave, rather than trying desperately to find a replacement.
The answer isn’t obvious. There is value that continues to be created by the third+ generation. I think I’ve mostly gone from “having a firm opinion that you should be proactively training your successor” to “man, I dunno, finding a suitable successor is actually pretty hard, mrrr?”
Reading this makes me think of organizations which manage to successfully have several generations of competent leadership. Something that has struck me for a while is the contrast in long-term competence between republics (not direct democracies) and hereditary monarchies.
Reading through history, hereditary monarchies always seem to fall into the problem you describe, of incompetent and (physically and mentally) weak monarchs being placed at the head of a nation, leading to a lot of problems. Republics, in contrast, almost always have competent leaders—one might disagree with their goals, and they are too often appointed after their prime, when their health is declining [1], but the leaders of republics are almost always very competent people.
This makes life much better for the people in the republic, and may be in part responsible for the recent proliferation of republics (though it does raise the question of why that hasn’t happened sooner. Maybe the robust safeguards implemented by the Founding Fathers of the USA in their constitution were a sufficiently non-obvious, but important, social technology, to be able to make republics viable on the world stage? [2]).
A key difference between monarchies and republics is that each successive generation of leadership in a republic must win an intense competition to secure their position, unlike the heirs of a monarchy. Not only this, but the competitions are usually held quite often (for example, every 4 years in Denmark, every 3 years in New Zealand), which ensures that the competitive nature of the office is kept in the public mind very frequently, making it hard to become a de facto hereditary position. By holding a competition to fill the office, one ensures that, even if the leader doesn’t share the same vision as the original founder, they still have to be very competent to be appointed to the position.
I contend that the usual way of appointing successors to small organizations (appointment by the previous leader) and to corporations (elected, but by a small body in a usually non-competitive fashion that is more similar to being appointed on a personal basis) is insufficiently competitive, and so is more similar to a hereditary monarchy than a republic, in this way.
---
[1] - This (The fact that the leaders of republics are often elected when their health is in decline) makes me think it may be a good idea to have a constitutional maximum age, after which individuals cannot be elected to certain important offices, to ensure that only people who are in their prime (and hence likely sufficiently healthy) can lead the nation.
[2] - The existence of elective monarchies also is suggestive that the theory may be meaningful, but it again raises the question of why elective monarchies weren’t more prominent. Maybe in practice elective monarchies were too likely to become effectively hereditary monarchies in all but name (c.f. the Hungarian kingdom and the Holy Roman Empire), that they didn’t distinguish themselves enough to have a competitive advantage.
older people have better shown themself (more time in which to do so, accumulate trust, etc.)
?
Elections (by means of voters) intentionally choose old leaders because that limits how long they can hold the position, or forces them to find a successor or delegate?
[2]
George Washington’s whole, only twice thing, almost seems more deliberate here. Wonder what would have happened if a similar check had been placed on political parties.
Regarding [1], people tend to vote for candidates they know, and politicians start out with 0 name recognition, which increases monotonically with age, always increasing but never decreasing, inherently biasing the process towards older candidates.
The two-term limit was actually not intended by Washington to become a tradition, he retired after his second term because he was declining in health. It was only later that it became expected for presidents not to serve more than 2 terms. I do think the term limit on the presidency is an important guard in maintaining the competitive and representative nature of the office, and I think it’s good to wonder if extending term limits to other things can be beneficial, though I am also aware of arguments pushing in the opposite direction
From Wikipedia: George Washington, which cites Korzi, Michael J. (2011). Presidential Term Limits in American History: Power, Principles, and Politics page 43, -and- Peabody, Bruce G. (September 1, 2001). “George Washington, Presidential Term Limits, and the Problem of Reluctant Political Leadership”. Presidential Studies Quarterly. 31 (3): 439–453:
At the end of his second term, Washington retired for personal and political reasons, dismayed with personal attacks, and to ensure that a truly contested presidential election could be held. He did not feel bound to a two-term limit, but his retirement set a significant precedent. Washington is often credited with setting the principle of a two-term presidency, but it was Thomas Jefferson who first refused to run for a third term on political grounds.
A note on the part that says “to ensure that a truly contested presidential election could be held”: at this time, Washington’s health was failing, and he indeed died during what would have been his 3rd term if he had run for a 3rd term. If he had died in office, he would have been immediately succeeded by the Vice President, which would set an unfortunate precedent of presidents serving until they die, then being followed by an appointed heir until that heir dies, blurring the distinction between the republic and a monarchy.
My read on this is that it’s still obviously worthwhile to train a successor, but to consider giving them clear instructions to shut down the group when it’s time for them to move on, to avoid the problems that come with 3rd-generational leadership.
What’s different for the organizer and first successor, in terms of their ability to do the primary job of finding their successor? I also note the pattern you mention (one handoff mostly succeeds, community degrades rapidly around the time the first successor leaves with no great second successor). But I also have seen a lot of cases where the founder fails to hand off in the first place, and some where it’s handed off to a committee or formal governance structure, and then eventually dies for reasons that don’t seem caused by succession.
I wonder if you’ve got the causality wrong—communities have a growth/maintenance/decline curve, which varies greatly in the parameters, but not so much in the shape. It seems likely to me that the leaders/organizers REACT to changes in the community by joining, changing their involvement, or leaving, rather than causing those changes.
The founder has a complete vision for the community/meetup/company/etc. They were able to design a thing that (as long as they continue putting in energy) is engaging, and they instinctively know how to change it so that it continues being great for participants.
The first successor has an incomplete, operational/keep-things-running-the-way-they-were type vision. They cargo-cult whatever the founder was doing. They don’t have enough vision to understand the ‘why’ behind all the decisions. But putting your finger on their precise blind spot is quite hard. It’s their “fault” (to the extent that we can blame anyone) that things go off the rails, but their bad decision-making doesn’t actually have short term impacts that anyone can see. Instead, the impacts come all at once, once they disappear, and there becomes common knowledge that it was a house of cards the whole time.
(or something. my models are fairly imprecise on this.)
Anyway, why did the founder get fooled into anointing the first successor even though they don’t have the skills to continue the thing? My guess is that there’s a fairly strong selection effect for founders combined with “market fit”—founders who fail to reach this resonant frequency don’t pick successors, they just fail. Whatever made them great at building this particular community doesn’t translate into skills at picking a successor, and that resonance may not happen to exist in any other person. Another founder-quality person would not necessarily have resonated with the existing community’s frequency, so there could also be an anti-selection effect there.
My model differs from yours. In my view, the first successor isn’t the source of most problems. The first successor usually has enough interaction and knowledge transfer from the founder, that they are able to keep things working more-or-less perfectly fine during their tenure, but they aren’t able to innovate and create substantial new value, since they lack the creativity and vision of the founder. In your terms, they are cargo-culting, but they are able to cargo-cult sufficiently well to keep the organization running smoothly; but when the second (and nth) successor comes in, they haven’t interacted much directly with the original founder, but instead are basing their decisions based, at most, on a vague notion of what the founder was like (though are often better served when they don’t even try to follow in the footsteps of the founder), and so are unable to keep things working according to the original vision. They are cargo-culting a cargo-cult, which isn’t enough to keep things working the way they need to work, at which point the organization stops being worth keeping around.
During the reign of the founder, the slope of the value created over time is positive, during the reign of the first successor, the slope is approximately zero, but once the second successor and beyond take over, the slope will be negative.
Without context, such a request to taboo a word sounds like you are asking the other person to never use that word, to cleanse it from their vocabulary, to go through the rest of their life with that word permanently off-limits. That’s a very high, and quite rude, ask to make of someone. While that’s of course not what we mean by “taboo”, I have seen requests to taboo made where it’s not clear that the other person knows what we mean by taboo, which means it’s quite likely the receiving party interpreted the request as being much ruder than was meant.
Instead of saying “Taboo [X word]”, instead say “could you please say what you just said without using [X word]?”—it conveys the same request, without creating the potential to be misunderstood to be making a rude and overreaching request.
The Roman Kingdom and Roman Empire both fell because of ineffective leaders. The Roman Republic fell because of extremely competent, but autocratic, leaders.
The Kingdom was overthrown; the last kings were not particularly well-loved by the people, and when King Tarquin raped Lucretia, the wife of an important general, the people deposed him and established the Republic, in particular creating the rule that any man who tried to make himself king could be killed on the spot without reprecussions.
The Roman Republic gave way to the Empire not all at once, but over the course of several different periods of leadership (since the consuls, the main leaders of the Republic, were elected for 1 year terms that couldn’t be immediately repeated, there’s a long list of leaders for any era). Julius Caesar did not start the end of the Republic, but he put the final nails in the coffin, having led an army in insurrection against the government, and becoming a king in all but name by the end of his life. The assassination of Caesar led to a series of civil wars, which ended with his nephew Augustus becoming Emperor of Rome. Needless to say, Julius Caesar and Augustus were both very competent men, in addition to many of the men who rivaled them for power, and all involved (with the exception of Augustus, who inherited his influence from Caesar) owed their influence to having been elected by the people of Rome.
As for the fall of the Empire, really the history of the fall of the Empire is just the history of the Empire, period. Sure, there were good Emperors who ruled well and competently, and the fullest extent of the reach of the Empire was after the Republic had already been overthrown, but for every good Emperor, there’s another bad Emperor who treats his populace in the cruelest ways imaginable, and blunders away influence and soft power, to mirror him. Already as soon as the first Emperor Augustus died, we get Tiberius, who wasn’t exactly great, then Caligula, whose name has justly become synonymous with overflowing sadism and needless excess.
Rome grew to become the great power that it was during the Republic, and the story of the Empire is the story of that great power slowly crumbling and declining under the rule of cruel and incompetent leaders, punctuated by the occasional enlightened ruler who would slow that decline for another 20 or 30 years.
I would like to see a page like TalkOrigins, but about IQ. So that any time someone confused but generally trying to argue in good faith posts something like “but wasn’t the idea of intelligence disproved scientifically?” or “intelligence is a real thing, but IQ is not” or “IQ is just an ability to solve IQ tests” or “but Taleb’s article/tweet has completely demolished the IQ pseudoscience” or one of the many other versions… I could just post this link. Because I am tired of trying to explain, and the memes are going to stay here for a foreseeable future.
Household robots vs. household appliances: Score One for Team Narrow
Vehicles on roads vs. a network of pipes, tubes, and rails: Score one for Team General
Ships that can go anywhere vs. a trade network of ships optimized for one specific route: Score one for Team General
(On the ships thing—apparently the Indian Ocean trade was specialized prior to the Europeans, with cargo being transferred from one type of ship to another to handle different parts of the route, especially the red sea which was dangerous to the type of oceangoing ship popular at the time. But then the Age of Sail happened.)
Obviously this is just three data points, two of which seem sorta similar because they both have to do with transporting stuff. It would be good to have more examples.
I recommend The Meme Machine, it’s a shame it didn’t spawn a huge literature. I was thinking a lot about memetics before reading it, yet still I feel like I learned a few important things.
Anyhow, here’s an idea inspired by it:
First, here is my favorite right way to draw analogies between AI and evolution:
Evolution : AI research over time throughout the world
Gene : Bit of code on Github
Organism : The weights of a model
Past experiences of an organism : Training run of a model
With that as background context, I can now present the idea.
With humans, memetic evolution is a thing. It influences genetic evolution and even happens fast enough to influence the learning of a single organism over time. With AIs, memetic evolution is pretty much not a thing. Sure, the memetic environment will change somewhat between 2020 and whenever APS-AI is built, but the change will be much less than all the changes that happened over the course of human evolution. And the AI training run that produces the first APS-AI may literally involve no memetic change at all (e.g. if it’s trained like GPT-3).
So. Human genes must code for the construction of a brain + learning procedure that works for many different memetic environments, and isn’t overspecialized to any particular memetic environment. Whereas the first APS-AI might be super-specialized to the memetic environment it was trained in.
This might be a barrier to building APS-AI; maybe it’ll be hard to induce a neural net to have the right sort of generality/flexibility because we don’t have lots of different memetic environments for it to learn from (and even if we did, there’s the further issue that the memetic environments wouldn’t be responding to it simultaneously) and maybe this is somehow a major block to having APS capabilities.
More likely, I think, is that APS-AI will still happen but it’ll just lack the human memetic generality. It’ll be “overfit” to the current memetic landscape. Maybe.
This is a Humble Bundle with a bunch of AI-related publications by Morgan & Claypool. $18 for 15 books. I’m a layperson re the material, but I’m pretty confident it’s worth $18 just to have all of these papers collected in one place and formatted nicely. NB increasing my payment from $18 to $25 would have raised the amount donated to the charity from $0.90 to $1.25--I guess the balance of the $7 goes directly to Humble.
I’ve recently been looking through available Berkeley coworking places.
The main options seem to be WeWork, NextSpace, CoWorking with Wisdom, and The Office: Berkeley. The Office seems basically closed now, CoWorking with Wisdom seemed empty when I passed by, and also seems fairly expensive, but nice.
I took a tour of WeWork and Nextspace. They both provide 24⁄7 access for all members, both have a ~$300/m option for open coworking, a ~$375/m for fixed desks, and more for private/shared offices. (At least now, with the pandemic. WeWork is typically $570/month for a dedicated desk apparently).
Both WeWork and NextSpace were fairly empty when I visited, though there weren’t many private offices available. The WeWork is much larger, but its split among several floors that I assume barely interact with each other.
Overall the NextSpace seemed a fair bit nicer to me. The vibe was more friendly, the receptionist much more friendly, there were several sit/stand desks, and I think I preferred the private offices. (They were a bit bigger and more separated from the other offices). That said, the WeWork seemed a bit more professional and quiet, and might have had a nicer kitchen.
If you look at the yelp/reviews for them, note that the furniture of the NextSpace changed a lot in the last ~1.5 years, so many of the old photos are outdated. I remembered one NextSpace in SF that didn’t seem very nice, but this one seemed better. Also, note that they have a promotion to work there for 1 day for free.
Their attempt to strategically pivot away from being about remembering information is deeply flawed.
They update their app to a new design and for 3 months the app just crashed when I start it on my phone (I have a Google Pixel 3A which isn’t that non-standard).
This Sunday, the app didn’t save two notes I made, and now notes can’t be saved.
Sounds horrible—I’m happy that I mostly use textfiles, and sync them using whatever mechanism works best (currently, Git + iCloud, but that’s changed 6-8 times over the last few decades).
I find it interesting that you picked “mismanaged” as your root cause, as opposed to “incompetent” or just “failing”.
I don’t disagree, but “management problem” is an undifferentiated cause. You can say that everything that seems like a mistake from outside is a management problem. Calling it a QA problem would be more specific (though no more helpful in terms of actions that a bystander can take).
I may have discovered an interesting tool against lethargy and depression [1]: This morning, in place of my usual caffeine pill, I made myself a cup of hot chocolate (using pure cacao powder / baking chocolate from the supermarket), which made me very energetic (much more energetic than usual), which stood in sharp contrast to the past 4 days, which have been marked by lethargy and intense sadness. Let me explain:
Last night, I was reflecting on the fact that one of the main components of chocolate is theobromine, which is very similar in structure to caffeine (theobromine is the reason why chocolate in poisonous to dogs & cats, for reasons similar to how caffeine was evolved to kill insects that feed on plants), and is known to be the reason why eating chocolate makes people happy. Since I have problems with caffeine, but rely on it to have energy, I figured it would be worthwhile to try using chocolate instead as a morning pick-me-up. I used baking chocolate instead of Nesquick or a hot chocolate packet because I’m avoiding sugar these days, and I figured having as pure chocolate as possible would be ideal for my experiment.
I was greeted with pleasant confirmation when I became very alert almost immediately after starting to drink the chocolate, despite having been just as lethargic as the previous days until I drank the chocolate. It’s always suggestive when you form a hypothesis based on facts and logic, then test the hypothesis, and exactly what you expected to happen, happens. But of course, I can’t be too confident until I try repeating this experiment on future days, which I will happily be doing after today’s success.
[1]: There are alternative hypotheses for why today was so different from the previous days: I attended martial arts class, then did some photography outside yesterday evening, which meant I got intense exercise, was around people I know and appreciate, and was doing stuff with intentionality, all of which could have contributed to my good mood today. There’s also the possibility of regression to the mean, but I’m dubious of this since today was substantially above average for me. I also had a (sugar-free) Monster later in the morning, but that was long after I had noticed being unusually alert, and now I have a headache that I can clearly blame on the Monster (Caffeine almost always gives me a headache) [1a].
[1a]: I drink energy drinks because I like the taste of them, not for utilitarian reasons. I observe that caffeine tends to make whatever contains it become deeply associated with enjoyment and craving, completely separated from the alertness-producing effects of the chemical. A similar thing happened with Vanilla Café Soylent [1b], which I absolutely hated the first time I tried it, but a few weeks later, I had deep cravings for, and could not do without.
[1b]: Sidenote, the brand Soylent has completely gone to trash, and I would not recommend anyone buy it these days. Buy Huel or Plenny instead.
I think I want to try this. What was your hot cocoa recipe? Did you just mix it with hot water? Milk? Cream? Salt? No sugar, I gather. How much? Does it taste any better than coffee? I want to get a sense of the dose required.
Just saw this. I used approximately 5 tablespoons of unsweetened cocoa powder, mixed with warm water. No sweetener, no sugar, or anything else. It’s bitter, but I do prefer the taste over coffee.
I just tried it. I did not enjoy the taste, although it does smell chocolatey. I felt like I had to choke down the second half. If it’s going to be bitter, I’d rather it were stronger. Maybe I didn’t stir it enough. I think I’ll use milk next time. I did find this: https://criobru.com/ apparently people do brew cacao like coffee. They say the “cacao” is similar to cocoa (same plant), but less processed.
I found this abstract suggesting that theobromine doesn’t affect mood or vigilance at reasonable doses. But this one suggests that chocolate does.
Subjectively, I feel that my cup of cocoa today might have reduced my usual lethargy and improved my mood a little bit, but not as dramatically as I’d hoped for. I can’t be certain this isn’t just the placebo effect.
The first linked study tests 100, 200, and 400 mg Theobromine. A rough heuristic based on the toxic doses of the two chemicals suggests 750 mg, maybe a little more (based on subjective experience) is equivalent to 100mg caffeine or a cup of coffee (this is roughly the dose I’ve been using each day), so I wouldn’t expect a particularly strong effect for the first two. The 400 mg condition does surprise me; the sample size of the study is small (n = 24 subjects * 1 trial per condition), so the fact that it failed to find statistical significance shouldn’t be too big of an update, though.
Can you clarify your Soylent anti-recommendation? I don’t use it as an actual primary nutrition, more as an easy snack for a missed meal, once or twice a week. I haven’t noticed any taste difference recently—my last case was purchased around March, and I pretty much only drink the Chai flavor.
A] Meal replacements require a large amount of trust in the entity that produces it, since if there’s any problems with the nutrition, that will have big impacts on your health. This is less so in your case, where it’s not a big part of the nutrition, but in my case, where I ideally use meal replacements as a large portion of my diet, trust is important.
B] A few years ago, Rob Rhinehart, the founder and former executive of Soylent, parted ways with the company due to his vision conflicting with the investor’s desires (which is never a good sign). I was happy to trust Soylent during the Rhinehart era, since I knew that he relied on his creation for his own sustenance, and seemed generally aligned. During that era, Soylent was very effective at signaling that they really cared about the world in general, and people’s nutrition in general. All the material that sent those signals no longer exists, and the implicit signals (e.g. the shape of and branding on the bottles, the new products they are developing [The biggest innovation during the Rhinehart era was caffeinated Soylent, now the main innovations are Bridge and Stacked, products with poor nutritional balance targeted at a naïve general audience, a far cry from the very idea of Complete Food], and the copy on their website) all indicate that the company’s main priority is now maximizing profit, without much consideration as to the (perceived) nutritional value of the product. In terms of product, the thing is probably still fine (though I haven’t actually looked at the ingredients in the recent new nutritional balance), but in terms of incentives and intentions, the management’s intention isn’t any better than, say, McDonald’s or Jack In The Box.
Since A] meal replacements require high trust and B] Soylent is no longer trustworthy: I cannot recommend anyone use Soylent more than a few times a week, but am happy to recommend Huel, Saturo, Sated, and Plenny, which all seem to still be committed to Complete Food.
(As far as flavour, I know I got one box with the old flavor after the recent flavor change, the supply lines often take time to get cleared out, so it’s possible you got a box of the old flavor. I don’t actually mind the new flavour, personally)
Thanks for the detail and info!
I recommend Ample (lifelong subscriber). It has high quality ingredients (no soy protein), fantastic macro ratios (5/30/65 - Ample K), and an exceptional founder.
I’m no longer sure that I buy dutch book arguments, in full generality, and this makes me skeptical of the “utility function” abstraction
Thesis: I now think that utility functions might be a pretty bad abstraction for thinking about the behavior of agents in general including highly capable agents.
[Epistemic status: half-baked, elucidating an intuition. Possibly what I’m saying here is just wrong, and someone will helpfully explain why.]
Over the past years, in thinking about agency and AI, I’ve taken the concept of a “utility function” for granted as the natural way to express an entity’s goals or preferences.
Of course, we know that humans don’t have well defined utility functions (they’re inconsistent, and subject to all kinds of framing effects), but that’s only because humans are irrational. To the extent that a thing acts like an agent, it’s behavior corresponds to some utility function. That utility function might not be explicitly represented, but if an agent is rational, there’s some utility function that reflects it’s preferences.
Given this, I might be inclined to scoff at people who scoff at “blindly maximizing” AGIs. “They just don’t get it”, I might think. “They don’t understand why agency has to conform to some utility function, and an AI would try to maximize expected utility.”
Currently, I’m not so sure. I think that talking in terms of utility functions is biting a philosophical bullet, and importing some unacknowledged assumptions. Rather than being the natural way to conceive of preferences and agency, I think utility functions might be only one possible abstraction, and one that emphasizes the wrong features, giving a distorted impression of what agents, in general, are actually like.
I want to explore that possibility in this post.
Before I begin, I want to make two notes.
First, all of this is going to be hand-wavy intuition. I don’t have crisp knock-down arguments, only a vague discontent. But it seems like more progress will follow if I write up my current, tentative, stance even without formal arguments.
Second, I don’t think utility functions being a poor abstraction for agency in the real world has much bearing on whether there is AI risk. As I’ll discuss, it might change the shape and tenor of the problem, but highly capable agents with alien seed preferences are still likely to be catastrophic to human civilization and human values. I mention this because the sentiments expressed in this essay are casually downstream of conversations that I’ve had with skeptics about whether there is AI risk at all. So I want to highlight: I think I was mistakenly overlooking some philosophical assumptions, but that is not a crux.
Is coherence overrated?
The tagline of the “utility” page on arbital is “The only coherent way of wanting things is to assign consistent relative scores to outcomes.”
This is true as far as it goes, but to me, at least, that sentence implies a sort of dominance of utility functions. “Coherent” is a technical term, with a precise meaning, but it also has connotations of “the correct way to do things”. If someone’s theory of agency is incoherent, that seems like a mark against it.
But it is possible to ask, “What’s so good about coherence anyway? Maybe
The standard reply of course, is that if your preferences are incoherent, you’re dutchbookable, and someone will pump you for money.
But I’m not satisfied with this argument. It isn’t obvious that being dutch booked is a bad thing.
In, Coherent Decisions Imply Consistent Utilities, Eliezer says,
Eliezer asserts that this is “qualitatively bad behavior.” But I think that this is biting a philosophical bullet.
As an intuition pump: In the actual case of humans, we seem to get utility not from states of the world, but from changes in states of the world. So it isn’t unusual for a human to pay to cycle between states of the world.
For instance, I could imagine a human being hungry, eating a really good meal, feeling full, and then happily paying a fee to be instantly returned to their hungry state, so that they can enjoy eating a good meal again.
This is technically a dutch booking (which do they prefer, being hungry or being full?), but from the perspective of the agent’s values there’s nothing qualitatively bad about it. Instead of the dutchbooker pumping money from the agent, he’s offering a useful and appreciated service.
Of course, we can still back out a utility function from this dynamic: instead of having a mapping of ordinal numbers to world states, we can have one from ordinal numbers to changes from world state to another.
But that just passes the buck one level. I see no reason in principle that an agent might have a preference to rotate between different changes in the world, just as well as rotating different between states of the world.
But this also misses the central point. I think you can always construct a utility function that represents some behavior. But if one is no longer compelled by dutch book arguments, this begs the question of why we would want to do that. If coherence is no longer a desiderata, it’s no longer clear that a utility function is that natural way to express preferences.
And I wonder, maybe this also applies to agents in general, or at least the kind of learned agents that humans are likely to build via gradient descent.
Maximization behavior
I think this matters, because many of the classic AI risk arguments go through a claim that maximization behavior is convergent. If you try to build a satisficer, there are a number of pressures for it to become a maximizer of some kind. (See this Rob Miles video, for instance)
I think that most arguments of that sort depend on an agent acting according to an expected utility maximization framework. And utility maximization turns out not to be a good abstraction for agents in the real world, I don’t know if these arguments are still correct.
I posit that straightforward maximizers are rare in the multiverse, and that most evolved or learned agents are better described by some other abstraction.
If not utility functions, then what?
If we accept for the time being that utility functions are a warped abstraction for most agents, what might a better abstraction be?
I don’t know. I’m writing this post in the hopes that others will think about this question and perhaps come up with productive alternative formulations.
I’ll post some of my half-baked thoughts on this question shortly.
Utility functions are especially problematic in modeling behaviour for agents with bounded rationality, or those where there are costs of reasoning. These include every physically realizable agent.
For modelling human behaviour, even considering the ideals of what we would like human behaviour to achieve, there are even worse problems. We can hope that there is some utility function consistent with the behaviour we’re modelling and just ignore cases where there isn’t, but that doesn’t seem satisfactory either.
‘Or you will leave money on the table.’
You rotated ‘different’ and ‘between’. (Or a serious of rotations isomorphic to such.)
I’ve long been somewhat skeptical that utility functions are the right abstraction.
My argument is also rather handwavy, being something like “this is the wrong abstraction for how agents actually function, so even if you can always construct a utility function and say some interesting things about its properties, it doesn’t tell you the thing you need to know to understand and predict how an agent will behave”. In my mind I liken it to the state of trying to code in functional programming languages on modern computers: you can do it, but you’re also fighting an uphill battle against the way the computer is physically implemented, so don’t be surprised if things get confusing.
And much like in the utility function case, people still program in functional languages because of the benefits they confer. I think the same is true of utility functions: they confer some big benefits when trying to reason about certain problems, so we accept the tradeoffs of using them. I think that’s fine so long as we have a morphism to other abstractions that will work better for understanding the things that utility functions obscure.
Alyssa Vance asked, “What great classes could be taught using ideas that might be seen on the Internet, but aren’t part of a standard curriculum yet?”.
My answer:
Deep learning (especially recent ideas like graph neural nets, transformers, GPT-3, deep learning applied to science), online advertising, cryptocurrency, contemporary cybersecurity, the internet in China (seems valuable for people outside China to understand), CRISPR, human genetics (e.g. David Reich’s work), contemporary videogames (either from technological or cultural/artistic perspective), contemporary TV, popular music in the age of Spotify, internet culture (e.g. Reddit, social media, memes).
Some more ideas:
superforecasting: the best class would involve people actually doing forecasting on something like Metaculus or on a prediction market with financial bets.
real-world practical applications of deep learning: considering the technical and economic/ethical aspects
immersive sociology/anthropology of internet cultures: you can’t do traditional anthropological fieldwork in an undergrad class but you can lurk or participate in any one of innumerable online subcultures.
immersion in country X: using machine translation, it’s possible to consume newspapers, Twitter, TV, etc from another country without speaking the language. someone who knows country X well could build an engaging class around this.
cooking: there are not many college courses on cooking (harvard’s famous class is an exception). youtube is pretty great for demonstrations.
There’s also video game music (which might be different from music in general because it can have a particular purpose).
How would this be different from movie music? There are some examples of music dynamically adapting according to what happens, but most games don’t go very deep into that. Stylistically, of course, due to the historical separation between the mediums, video game music often sounds different from movie music. But practically I suspect there’s not much difference.
Movie music wasn’t listed either, just popular music.
That said,
musical numbers in games might be intended to cover more time, and be more flow workable. But this might start to get into, ‘What type of game?’, ‘What type of movie?‘, and ‘What part of the work is the song in?’ (They might seem most similar in the trailers for each, because there they are serving similar roles/purposes.)
The lethal dose of caffeine in adult humans is approximately 10 grams, while the lethal dose of theobromine (the main psychoactive chemical in chocolate, nearly identical structurally to caffeine, with similar effects) in humans is 75 grams (this is much lower in most animals, which is why you should never give chocolate to your pets). This can motivate a rough heuristic of 7.5 mg theobromine is roughly equal to 1 mg caffeine, and 750 mg theobromine is equivalent to one cup of coffee.
Therefore, to replace coffee with cocoa or chocolate, 6 spoons of unsweetened cocoa powder should replace a cup of coffee. 11 cups of hot chocolate (that’s a lot) or 2 bars of dark chocolate should also work.
I’ve long been aware of the concept of a “standard drink”, a unit for measuring how much alcohol a person has had, regardless of what they are drinking, so one “drink” of wine contains less liquid than one “drink” of beer, but more than one drink of vodka. When I started experimenting with chemicals other than ethanol, I intuitively wanted to extend this notion to other chemicals. For example, in my mind, I roughly equate 10 mg of Tetrahydracannabinol with one drink of ethanol. While the effects of these two chemicals are quite different, and work in different ways, they both have relaxing and depressant (not “depressing”) effects, so there is some meaningful comparison—if I want to incapacitate myself to a certain extent, I can use the concept of an “depressant unit” to calculate a dose of either THC or ethanol, or similarly with diphenhydramine (ZZZQuil) or Nyquil.
Clearly, in most cases, I would not want to compare the strength of an alcoholic beverage with the strength of a caffeinated beverage. But I would want to be able to use a “stimulating unit” to compare, say, amounts of caffeine to amounts of theobromine (cocoa) or to other stimulating chemicals (for example, Adderall).
Another unit that would be worth using would be an “entheogenic unit”, which would allow one to compare doses of LSD, Psilocybin, THC (regarding its quasi-psychedelic, not depressant qualities), and so on, in terms of their ability to change the way one thinks.
Question: Is it possible to incorporate Caffeine into DNA? Caffeine is structurally similar to Adenine, one of the four DNA nucleobases (and the A in ATCG). But looking at the structure, the hexagonal ring (which is the part of the DNA that bonds A to T and C to G) doesn’t look very promising—there are two oxygen atoms that can bond, but they are a bit too far apart, and there are no hydrogens, and since DNA is held together by hydrogen bonds, the hydrogen will have to be provided by whatever it is paired to. Theobromine looks more promising, since a CH3 group is replaced by an H (otherwise it is identical to Caffeine), which provides a continuous run of bondable groups, and the H can be used for hydrogen bonding.
Probably for either Theobromine or Caffeine, they would have to be paired with another base that is not one of the usual ATCG bases, which is specially chosen to complement the shape of the molecules.
It’s now been about two years since I started seriously blogging. Most of my posts are on Lesswrong, and the most of the rest are scattered about on my substack and the Effective Altruist Forum, or on Facebook. I like writing, but I have an impediment which I feel impedes me greatly.
In short: I often post garbage.
Sometimes when I post garbage, it isn’t until way later that I learn that it was garbage. And when that happens, it’s not that bad, because at least I grew as a person since then.
But the usual case is that I realize that it’s garbage right after I’m done posting it, and then I keep thinking, “oh no, what have I done!” as the replies roll in, explaining to me that it’s garbage.
Most times when this happens, I just delete the post. I feel bad when this happens because I generally spend a lot of time writing and reviewing the posts. Some of the time, I don’t delete the post because I still stand by the main thesis, although the delivery or logical chain of reasoning was not very good and so I still feel bad about it.
I’m curious how other writers deal with this problem. I’m aware of “just stop caring” and “review your posts more.” But, I’m sometimes in awe of some people who seem to consistently never post garbage, and so maybe they’re doing something right that can be learned.
Sometimes I send a draft to a couple people before posting it publicly.
Sometimes I sit on an idea for a while, then find an excuse to post it in a comment or bring it up in a conversation, get some feedback that way, and then post it properly.
I have several old posts I stopped endorsing, but I didn’t delete them; I put either an update comment at the top or a bunch of update comments throughout saying what I think now. (Last week I spent almost a whole day just putting corrections and retractions into my catalog of old posts.) I for one would have a very positive impression of a writer whose past writings were full of parenthetical comments that they were wrong about this or that. Even if the posts wind up unreadable as a consequence.
I have a hope that with more practice, this gets better.
Not just practice, but also noticing what other people do differently. For example, I often write long texts, which some people say is already a mistake. But even a long text can be made more legible if it contains section headers and pictures. Both of them break the visual monotonicity of the text wall. This is why section headers are useful even if they are literally: “1”, “2″, “3”. In some sense, pictures are even better, because too many headers create another layer of monotonicity, which a few unique pictures do not. Which again suggests that having 1 photo, 1 graph, and 1 diagram is better than having 3 photos. I would say, write the text first, then think about which parts can be made clearer by adding a picture.
There is some advice on writing, by Stephen King, or by Scott Alexander.
If you post a garbage, let it be. Write more articles, and perhaps at the end of a year (or a decade) make a list “my best posts” which will not include the garbage.
BTW, whatever you do, you will get some negative response. Your posts on LW are upvoted, so I assume they are not too bad.
Also, writing can be imbalanced. Even for people who only write great texts, some of them are more great and some of them are less great than the others. But if they deleted the worst one, guess what, now some other articles is the worst one… and if you continue this way, you will stop with one or zero articles.
I’m 65% moving back to us in a month or two but haven’t lived there in like 5 years so am not sure what to expect nor where I’d like to go.
I’m mainly optimizing for friend-making/having/meeting (IRL) though I’m not sure how much I care about that since I haven’t had much chance to do that for the last few years. though also trees and stuff are nice too
I’m not really sure how to choose between cities in terms of satisfying this. vaguely seems like SF would be cool and have many cool people that would be nice to meet but is also expensive
Any recommendations on figuring out where to stay?
Shared with permission, a google doc exchange confirming Eliezer still finds the arguments for alignment optimism, slower takeoffs, etc. unconvincing:
Caveat: this was a private reply I saw and wanted to share (so people know EY’s basic epistemic state, and therefore probably the state of other MIRI leadership). This wasn’t an attempt to write an adequate public response to any of the public arguments put forward for alignment optimism or non-fast takeoff, etc., and isn’t meant to be a replacement for public, detailed, object-level discussion. (Though I don’t know when/if MIRI folks plan to produce a proper response, and if I expected such a response soonish I’d probably have just waited and posted that instead.)
You could tell people to read certain books, but how they read and what they get out of them is quite unknown, a lot of times even to the readers themselves. There is no exact formula for how to learn things. You could take things at face value, and those usually only work for guided courses. As for reading a book, you have to pause and think about the stuff you read, and there is no general guidance on how deep of a thought tangent you want to explore. Most books are read sequentially, but optimizing what you get out of the things you read, doesn’t have to be books, is hardly a sequential task.
Also not all knowledge is expressible in linguistic terms. A typical example is kinesthetic knowledge, like muscle memories. Also as you read more, the speed of your reading increases. Usually they associate these type of information as skills. There is a clear distinction when we talk about knowledge and skill, but by their very nature, they might not be so distinct as we like to think of them. Musical improvisation is a mixture of multiple knowledge domains.
Why am I being downvoted? So strange.
While your comment was clearly written in good faith, it seems to me like you’re missing some context. You recommend that EY recommend that the detractors read books. EY doesn’t just recommend people read books. He wrote the equivalent of like three books on the subjects relevant to this conversation in particular which he gives away for free. Also, most of the people in this conversation are already big into reading books.
It is my impression he also helped establish the Center for Applied Rationality, which has the explicit mission of training skills. (I’m not sure if he technically did but he was part of the community which did and he helped promote it in its early days.)
Eliezer was involved with CFAR in the early days, but has not been involved since at least 2016.
Thinking out loud about some arguments about AI takeoff continuity:
If a discontinuous takeoff is more likely to be local to a particular agent or closely related set of agents with particular goals, and a continuous takeoff is more likely to be global, that seems like it incentivizes the first agent capable of creating a takeoff to make sure that that takeoff is discontinuous, so that it can reap the benefits of the takeoff being local to that agent. This seems like an argument for expecting a discontinuous takeoff and an important difference with other allegedly analogous technologies.
I have some trouble understanding the “before there are strongly self-improving AIs there will be moderately self-improving AIs” argument for continuity. Is there any reason to think the moderate self-improvement ability won’t be exactly what leads to the strong self-improvement ability? Before there’s an avalanche, there’s probably a smaller avalanche, but maybe the small avalanche is simply identical to the early part of the large avalanche.
Where have these points been discussed in depth?
Several very important to me people whom I love told me that they would rather die than live even a few hundred years or indefinitely, that they would not choose cryopreservation if life extension capabilities aren’t advanced enough by their “natural time”, and so on, when I asked them how they felt about immortality (scenario was: imagine that humanity figures out how to be immortal and there are no restrictions, anyone can have it if they want it, do you take it yes or no?).
There’s too much deathism in this world, aahhhhhhhhh. I’ve already started to mourn those people, and it hurts so fucking much, it literally is keeping me awake tonight...I was meditating then trying to sleep and reminders of their choice bubbled up and now here I am, typing away. Crying about and mourning the loss of loved ones who haven’t died yet, but ultimately said that that was their preference over life extension / immortality.
It hurts so bad.
Writing this felt helpful somewhat, at least I’ve channeled those feelings and temporarily diminished their intensity, somewhat...well, back to trying to sleep.
Arguably, self modification might be different from continuing to exist (comparatively) unchanged.
Talk is cheap. Someone who says “I want to die eventually” isn’t actually invested in the answer—it’s just them justifying to themselves why they’re not exercising, eating right, and otherwise planning for a long future.
This is very uncharitable. For many people, living forever is simply not a realistic option. Heck, many rationalists give it a chance around 10%, and that already involves a lot of belief in progress, which many people don’t have.
Also, people are not automatically strategic. For example, religious people believe that sin can bring them eternity in hell, and they still keep sinning.
Arguably that’s internal conflict. (Or, alternatively, what point is there in the religion if religious people are perfect?)
You’re assuming a lot about other peoples’ experiences and motivations, the internal experience that my aforementioned love ones have described to me looks not at all similar to what you said. While their internal experiences and their desire for eventual death are alien thought processes, emotions, and experiences to me, I do notice that the people-space of people who prefer eventual death to immortality contains a pretty wide variety of reasonings and internal experiences for why they prefer that eventual death...including surprisingly well thought out and sophisticated and logically coherent answers. Some people genuinely want to die eventually rather than live indefinitely, and that mindset / preference is so alien to mine own that it’s a struggle to accept that people believe such things and have such preferences, but I keep encountering people who do so it seems to be true.
However, I can see how what you said might be an internal experience for some people within people-space, it does check out and pass my anecdotal experience test at least (I’ve encountered some people who, per their description of internal experience, are likely similar to the mindset you described).
I like to ask people their preferences on this matter, so I’ve heard a lot different answers to the “death vs immortality” questions, and while I’ve encountered some people who have a strong or neutral preference for immortality, I’ve encountered a surprisingly high amount of people who would prefer death, and that sucks.
On average, as you grow older, your health gets worse. I suspect that many people make an interpolation of this process, and their idea of a 1000 years old person is kind of a zombie in a wheelchair screaming in pain. Arguably, a fate worse than death. (And if you are religious, or unable to talk, then choosing death is not even an option here.) So perhaps it would be better to talk about “more decades of youth” rather than extension of life-as-we-know-it.
Another possible fear is of waking up in a bad future. (Which again may be worse than death, and suicide may not be an option.) I have no idea what are the actual probabilities here.
I suspect that a good deal of people make that assumption too, about what living past a certain age would be like. Or the bad future scenario. I’ve encountered people who believe either or both things, but once I frame the question and scenario as immortality with perpetual youth then the first concern almost always disappears. The majority of people I ask the question / scenario to, keep bringing up concerns about population and where all these immortality people are going to live. That’s not really something I’m worried about, because the universe is very large and an assumption I make is that humanity would spread throughout the stars if immortal, but I don’t have a great specific answer regarding the population concerns people have mentioned.
Actually, religious people with strong faith in their religion’s conception of an afterlife are the most likely to choose eventual death over immortality in my experience, because they believe that one their time is up on Earth, they simply die and go to their religion’s afterlife, and they find that very strongly preferable to living perpetually in the material world.
For the religious ones, perhaps a good frame would be “young for 1000 years”, so that they can still enjoy the afterlife. More time to do the earthly stuff, and the afterlife is supposed to be infinite anyway.
Population… the best case would be something like “people are young forever, but they can only have kids during the first few decades”. Anyway, with exponential growth we would run out of resources even without immortality. And if there is ever a law against exponential growth, like “only 2 kids per a pair of adults”, then immortality would mean a linearly growing population, which should be doable somehow. But yeah, this is difficult to explain, and requires some faith in either space travel or linear increases in food production.
They’ll almost definitely change their minds once we have good treatments for aging.
I know some will, but that’s too optimistic and ignores the preferences / experiences of a huge amount of people, because there are categories of people who prefer death over immortality for whom the aging process doesn’t factor in to their choice on that matter. Especially people with strong faith in their religion’s afterlife.
What I mean is that they haven’t really considered it. As I’m sure you’re aware, your mind does not work like most people’s. When most people consider the question of whether they’d like to die someday, they’re not really thinking about it as if it were a real option. Even if they give detailed, logically coherent explanations for why they’d like to die someday, they haven’t considered it in in near mode.
I am very confident of this—once they have the option, they will not choose to die. Right now they see it as just an abstract philosophical conversation, so they’ll just say whatever sounds nice to them. For a variety of reasons, “I’d like to die someday; being immortal doesn’t appeal to me” sounds wise to a lot of people. But once they have the actual option to not die? Who’s going to choose to die? Only people who are suicidally depressed or who are in very extreme cults will choose that.
A thing I might have maybe changed my mind about:
I used to think a primary job of a meetup/community organizer was to train their successor, and develop longterm sustainability of leadership.
I still hold out for that dream. But, it seems like a pattern is:
1) community organizer with passion and vision founds a community
2) they eventually move on, and pass it on to one successor who’s pretty closely aligned and competent
3) then the First Successor has to move on to, and then… there isn’t anyone obvious to take the reins, but if no one does the community dies, so some people reluctantly step up. and....
...then forever after it’s a pale shadow of its original self.
For semi-branded communities (such as EA, or Rationality), this also means that if someone new with energy/vision shows up in the area, they’ll see a meetup, they’ll show up, they’ll feel like the meetup isn’t all that good, and then move on. Wherein they (maybe??) might have founded a new one that they got to shape the direction of more.
I think this also applies to non-community organizations (i.e. founder hands the reins to a new CEO who hands the reins to a new CEO who doesn’t quite know what to do)
So… I’m kinda wondering if second-generation successors should just… err on the side of shutting the thing down when they leave, rather than trying desperately to find a replacement.
The answer isn’t obvious. There is value that continues to be created by the third+ generation. I think I’ve mostly gone from “having a firm opinion that you should be proactively training your successor” to “man, I dunno, finding a suitable successor is actually pretty hard, mrrr?”
What if the replacement isn’t a replacement? If only a different person/people with a different vision/s can be found then...why not that?
Or, what does the leader do, that can’t be carried on?
Reading this makes me think of organizations which manage to successfully have several generations of competent leadership. Something that has struck me for a while is the contrast in long-term competence between republics (not direct democracies) and hereditary monarchies.
Reading through history, hereditary monarchies always seem to fall into the problem you describe, of incompetent and (physically and mentally) weak monarchs being placed at the head of a nation, leading to a lot of problems. Republics, in contrast, almost always have competent leaders—one might disagree with their goals, and they are too often appointed after their prime, when their health is declining [1], but the leaders of republics are almost always very competent people.
This makes life much better for the people in the republic, and may be in part responsible for the recent proliferation of republics (though it does raise the question of why that hasn’t happened sooner. Maybe the robust safeguards implemented by the Founding Fathers of the USA in their constitution were a sufficiently non-obvious, but important, social technology, to be able to make republics viable on the world stage? [2]).
A key difference between monarchies and republics is that each successive generation of leadership in a republic must win an intense competition to secure their position, unlike the heirs of a monarchy. Not only this, but the competitions are usually held quite often (for example, every 4 years in Denmark, every 3 years in New Zealand), which ensures that the competitive nature of the office is kept in the public mind very frequently, making it hard to become a de facto hereditary position. By holding a competition to fill the office, one ensures that, even if the leader doesn’t share the same vision as the original founder, they still have to be very competent to be appointed to the position.
I contend that the usual way of appointing successors to small organizations (appointment by the previous leader) and to corporations (elected, but by a small body in a usually non-competitive fashion that is more similar to being appointed on a personal basis) is insufficiently competitive, and so is more similar to a hereditary monarchy than a republic, in this way.
---
[1] - This (The fact that the leaders of republics are often elected when their health is in decline) makes me think it may be a good idea to have a constitutional maximum age, after which individuals cannot be elected to certain important offices, to ensure that only people who are in their prime (and hence likely sufficiently healthy) can lead the nation.
[2] - The existence of elective monarchies also is suggestive that the theory may be meaningful, but it again raises the question of why elective monarchies weren’t more prominent. Maybe in practice elective monarchies were too likely to become effectively hereditary monarchies in all but name (c.f. the Hungarian kingdom and the Holy Roman Empire), that they didn’t distinguish themselves enough to have a competitive advantage.
[1]
Does this demonstrate:
a lack of younger leaders
older people have better shown themself (more time in which to do so, accumulate trust, etc.)
?
Elections (by means of voters) intentionally choose old leaders because that limits how long they can hold the position, or forces them to find a successor or delegate?
[2]
George Washington’s whole, only twice thing, almost seems more deliberate here. Wonder what would have happened if a similar check had been placed on political parties.
Regarding [1], people tend to vote for candidates they know, and politicians start out with 0 name recognition, which increases monotonically with age, always increasing but never decreasing, inherently biasing the process towards older candidates.
The two-term limit was actually not intended by Washington to become a tradition, he retired after his second term because he was declining in health. It was only later that it became expected for presidents not to serve more than 2 terms. I do think the term limit on the presidency is an important guard in maintaining the competitive and representative nature of the office, and I think it’s good to wonder if extending term limits to other things can be beneficial, though I am also aware of arguments pushing in the opposite direction
Citation? (I’ve only really read American Propaganda about this so not very surprised if this is the case, but hadn’t heard it before)
From Wikipedia: George Washington, which cites Korzi, Michael J. (2011). Presidential Term Limits in American History: Power, Principles, and Politics page 43, -and- Peabody, Bruce G. (September 1, 2001). “George Washington, Presidential Term Limits, and the Problem of Reluctant Political Leadership”. Presidential Studies Quarterly. 31 (3): 439–453:
A note on the part that says “to ensure that a truly contested presidential election could be held”: at this time, Washington’s health was failing, and he indeed died during what would have been his 3rd term if he had run for a 3rd term. If he had died in office, he would have been immediately succeeded by the Vice President, which would set an unfortunate precedent of presidents serving until they die, then being followed by an appointed heir until that heir dies, blurring the distinction between the republic and a monarchy.
Thanks!
My read on this is that it’s still obviously worthwhile to train a successor, but to consider giving them clear instructions to shut down the group when it’s time for them to move on, to avoid the problems that come with 3rd-generational leadership.
What’s different for the organizer and first successor, in terms of their ability to do the primary job of finding their successor? I also note the pattern you mention (one handoff mostly succeeds, community degrades rapidly around the time the first successor leaves with no great second successor). But I also have seen a lot of cases where the founder fails to hand off in the first place, and some where it’s handed off to a committee or formal governance structure, and then eventually dies for reasons that don’t seem caused by succession.
I wonder if you’ve got the causality wrong—communities have a growth/maintenance/decline curve, which varies greatly in the parameters, but not so much in the shape. It seems likely to me that the leaders/organizers REACT to changes in the community by joining, changing their involvement, or leaving, rather than causing those changes.
I’m not Ray, but I’ll take a stab --
The founder has a complete vision for the community/meetup/company/etc. They were able to design a thing that (as long as they continue putting in energy) is engaging, and they instinctively know how to change it so that it continues being great for participants.
The first successor has an incomplete, operational/keep-things-running-the-way-they-were type vision. They cargo-cult whatever the founder was doing. They don’t have enough vision to understand the ‘why’ behind all the decisions. But putting your finger on their precise blind spot is quite hard. It’s their “fault” (to the extent that we can blame anyone) that things go off the rails, but their bad decision-making doesn’t actually have short term impacts that anyone can see. Instead, the impacts come all at once, once they disappear, and there becomes common knowledge that it was a house of cards the whole time.
(or something. my models are fairly imprecise on this.)
Anyway, why did the founder get fooled into anointing the first successor even though they don’t have the skills to continue the thing? My guess is that there’s a fairly strong selection effect for founders combined with “market fit”—founders who fail to reach this resonant frequency don’t pick successors, they just fail. Whatever made them great at building this particular community doesn’t translate into skills at picking a successor, and that resonance may not happen to exist in any other person. Another founder-quality person would not necessarily have resonated with the existing community’s frequency, so there could also be an anti-selection effect there.
My model differs from yours. In my view, the first successor isn’t the source of most problems. The first successor usually has enough interaction and knowledge transfer from the founder, that they are able to keep things working more-or-less perfectly fine during their tenure, but they aren’t able to innovate and create substantial new value, since they lack the creativity and vision of the founder. In your terms, they are cargo-culting, but they are able to cargo-cult sufficiently well to keep the organization running smoothly; but when the second (and nth) successor comes in, they haven’t interacted much directly with the original founder, but instead are basing their decisions based, at most, on a vague notion of what the founder was like (though are often better served when they don’t even try to follow in the footsteps of the founder), and so are unable to keep things working according to the original vision. They are cargo-culting a cargo-cult, which isn’t enough to keep things working the way they need to work, at which point the organization stops being worth keeping around.
During the reign of the founder, the slope of the value created over time is positive, during the reign of the first successor, the slope is approximately zero, but once the second successor and beyond take over, the slope will be negative.
Asking people to “taboo [X word]” is bad form, unless you already know that the other person is sufficiently (i.e. very) steeped in LW culture to know what our specific corner of internet culture means by “taboo”.
Without context, such a request to taboo a word sounds like you are asking the other person to never use that word, to cleanse it from their vocabulary, to go through the rest of their life with that word permanently off-limits. That’s a very high, and quite rude, ask to make of someone. While that’s of course not what we mean by “taboo”, I have seen requests to taboo made where it’s not clear that the other person knows what we mean by taboo, which means it’s quite likely the receiving party interpreted the request as being much ruder than was meant.
Instead of saying “Taboo [X word]”, instead say “could you please say what you just said without using [X word]?”—it conveys the same request, without creating the potential to be misunderstood to be making a rude and overreaching request.
Step 1: Play the game taboo.
Step 2: Request something like “Can we play a mini-round of taboo with *this word* for 5 minutes?”
*[Word X]*
Alternatively, ‘Could you rephrase that?’/‘I looked up what _ means in the dictionary, but I’m still not getting something...’
I see you tabooed “taboo”.
Indeed, this is the right approach to LW lingo… only, sometimes it expands the words into long descriptions.
The Roman Kingdom and Roman Empire both fell because of ineffective leaders. The Roman Republic fell because of extremely competent, but autocratic, leaders.
Who brought it down, or who were too essential and when they died it collapsed?
The Kingdom was overthrown; the last kings were not particularly well-loved by the people, and when King Tarquin raped Lucretia, the wife of an important general, the people deposed him and established the Republic, in particular creating the rule that any man who tried to make himself king could be killed on the spot without reprecussions.
The Roman Republic gave way to the Empire not all at once, but over the course of several different periods of leadership (since the consuls, the main leaders of the Republic, were elected for 1 year terms that couldn’t be immediately repeated, there’s a long list of leaders for any era). Julius Caesar did not start the end of the Republic, but he put the final nails in the coffin, having led an army in insurrection against the government, and becoming a king in all but name by the end of his life. The assassination of Caesar led to a series of civil wars, which ended with his nephew Augustus becoming Emperor of Rome. Needless to say, Julius Caesar and Augustus were both very competent men, in addition to many of the men who rivaled them for power, and all involved (with the exception of Augustus, who inherited his influence from Caesar) owed their influence to having been elected by the people of Rome.
As for the fall of the Empire, really the history of the fall of the Empire is just the history of the Empire, period. Sure, there were good Emperors who ruled well and competently, and the fullest extent of the reach of the Empire was after the Republic had already been overthrown, but for every good Emperor, there’s another bad Emperor who treats his populace in the cruelest ways imaginable, and blunders away influence and soft power, to mirror him. Already as soon as the first Emperor Augustus died, we get Tiberius, who wasn’t exactly great, then Caligula, whose name has justly become synonymous with overflowing sadism and needless excess.
Rome grew to become the great power that it was during the Republic, and the story of the Empire is the story of that great power slowly crumbling and declining under the rule of cruel and incompetent leaders, punctuated by the occasional enlightened ruler who would slow that decline for another 20 or 30 years.
I would like to see a page like TalkOrigins, but about IQ. So that any time someone confused but generally trying to argue in good faith posts something like “but wasn’t the idea of intelligence disproved scientifically?” or “intelligence is a real thing, but IQ is not” or “IQ is just an ability to solve IQ tests” or “but Taleb’s article/tweet has completely demolished the IQ pseudoscience” or one of the many other versions… I could just post this link. Because I am tired of trying to explain, and the memes are going to stay here for a foreseeable future.
I’d like a page like this just so I can learn about IQ without having to dig through lots of research myself.
Historical precedents for general vs. narrow AI
Household robots vs. household appliances: Score One for Team Narrow
Vehicles on roads vs. a network of pipes, tubes, and rails: Score one for Team General
Ships that can go anywhere vs. a trade network of ships optimized for one specific route: Score one for Team General
(On the ships thing—apparently the Indian Ocean trade was specialized prior to the Europeans, with cargo being transferred from one type of ship to another to handle different parts of the route, especially the red sea which was dangerous to the type of oceangoing ship popular at the time. But then the Age of Sail happened.)
Obviously this is just three data points, two of which seem sorta similar because they both have to do with transporting stuff. It would be good to have more examples.
I recommend The Meme Machine, it’s a shame it didn’t spawn a huge literature. I was thinking a lot about memetics before reading it, yet still I feel like I learned a few important things.
Anyhow, here’s an idea inspired by it:
First, here is my favorite right way to draw analogies between AI and evolution:
Evolution : AI research over time throughout the world
Gene : Bit of code on Github
Organism : The weights of a model
Past experiences of an organism : Training run of a model
With that as background context, I can now present the idea.
With humans, memetic evolution is a thing. It influences genetic evolution and even happens fast enough to influence the learning of a single organism over time. With AIs, memetic evolution is pretty much not a thing. Sure, the memetic environment will change somewhat between 2020 and whenever APS-AI is built, but the change will be much less than all the changes that happened over the course of human evolution. And the AI training run that produces the first APS-AI may literally involve no memetic change at all (e.g. if it’s trained like GPT-3).
So. Human genes must code for the construction of a brain + learning procedure that works for many different memetic environments, and isn’t overspecialized to any particular memetic environment. Whereas the first APS-AI might be super-specialized to the memetic environment it was trained in.
This might be a barrier to building APS-AI; maybe it’ll be hard to induce a neural net to have the right sort of generality/flexibility because we don’t have lots of different memetic environments for it to learn from (and even if we did, there’s the further issue that the memetic environments wouldn’t be responding to it simultaneously) and maybe this is somehow a major block to having APS capabilities.
More likely, I think, is that APS-AI will still happen but it’ll just lack the human memetic generality. It’ll be “overfit” to the current memetic landscape. Maybe.
This is a Humble Bundle with a bunch of AI-related publications by Morgan & Claypool. $18 for 15 books. I’m a layperson re the material, but I’m pretty confident it’s worth $18 just to have all of these papers collected in one place and formatted nicely. NB increasing my payment from $18 to $25 would have raised the amount donated to the charity from $0.90 to $1.25--I guess the balance of the $7 goes directly to Humble.
https://www.humblebundle.com/books/ai-morgan-claypool-books
On Berkeley coworking:
I’ve recently been looking through available Berkeley coworking places.
The main options seem to be WeWork, NextSpace, CoWorking with Wisdom, and The Office: Berkeley. The Office seems basically closed now, CoWorking with Wisdom seemed empty when I passed by, and also seems fairly expensive, but nice.
I took a tour of WeWork and Nextspace. They both provide 24⁄7 access for all members, both have a ~$300/m option for open coworking, a ~$375/m for fixed desks, and more for private/shared offices. (At least now, with the pandemic. WeWork is typically $570/month for a dedicated desk apparently).
Both WeWork and NextSpace were fairly empty when I visited, though there weren’t many private offices available. The WeWork is much larger, but its split among several floors that I assume barely interact with each other.
Overall the NextSpace seemed a fair bit nicer to me. The vibe was more friendly, the receptionist much more friendly, there were several sit/stand desks, and I think I preferred the private offices. (They were a bit bigger and more separated from the other offices). That said, the WeWork seemed a bit more professional and quiet, and might have had a nicer kitchen.
If you look at the yelp/reviews for them, note that the furniture of the NextSpace changed a lot in the last ~1.5 years, so many of the old photos are outdated. I remembered one NextSpace in SF that didn’t seem very nice, but this one seemed better. Also, note that they have a promotion to work there for 1 day for free.
https://www.yelp.com/biz/nextspace-coworking-berkeley-berkeley-3
https://www.yelp.com/biz/wework-berkeley-berkeley-2?osq=WeWork
It’s amazing how mismanged Evernote is.
Their attempt to strategically pivot away from being about remembering information is deeply flawed.
They update their app to a new design and for 3 months the app just crashed when I start it on my phone (I have a Google Pixel 3A which isn’t that non-standard).
This Sunday, the app didn’t save two notes I made, and now notes can’t be saved.
I’ve had to leave Evernote over the new app, and am so sad about it.
Sounds horrible—I’m happy that I mostly use textfiles, and sync them using whatever mechanism works best (currently, Git + iCloud, but that’s changed 6-8 times over the last few decades).
I find it interesting that you picked “mismanaged” as your root cause, as opposed to “incompetent” or just “failing”.
Releasing a new version when it’s very buggy looks to me like a management problem.
I don’t disagree, but “management problem” is an undifferentiated cause. You can say that everything that seems like a mistake from outside is a management problem. Calling it a QA problem would be more specific (though no more helpful in terms of actions that a bystander can take).