Also I think “put on” means “affected” and “worn out” means “anachronistic + a specific narrative of why it’s anachronistic, i.e. because the advice became worn out.”
Arjun Panickssery
Karma: 3,363
(I didn’t disagree-vote).
I considered “Latin,” which obviously comes from Latin, but via Old English so I think it’s fine. (Note that “Anglo” also comes from Latin, so a better case is that I should have just said “English.”) I don’t think it’s necessary to come up with made-up words for other places or languages that have their own names for this purpose.
I agree that “plain” and “appear” come from Old French and those were oversights. And it shows that it’s hard to actually write this way without oversights.
Quick search indicates that Churchill is the originator of the sentiment
Where did you hear that? The Fowlers’ classic 1906 guide says “Prefer the Saxon word to the Romance” and the opinion was already standard at that time.
Did you use Claude for the rewrite? I think you can do a much less weird one (and what is “Walnut Folk”?) e.g.
I think that “to talk with folks plainly, use words with Anglo-Saxon rather than Latin roots” is itself put on and worn out.
I think most good writers already have a good “feel” for which words are mainstream or over-the-top. But if you can’t tell, it’s better to look at how often the word appears rather than its roots.
And indeed this is a plainer style.
This is similar to @habryka’s reply here where I agree with the statements in the reply but I don’t think they respond to my objection.
If I understand your two points correctly they are that
An open-question critique of the idealization-procedure definition can be applied to any conceptual analysis. Yes, sure. (Irrelevant but I also don’t think the analysis of concepts is very useful.)
There is no is-ought gap because an “ought sentence” can be rephrased as an “is sentence.”
But these only address a weak “semantic” interpretation of my objection to the analysis when what I am questioning is why the proposed analysis produces normative authority. My complaint isn’t the general complaint that to define the good as the product of an idealization procedure is either trivial or false, but that there’s this actual thing (normative authority) that isn’t addressed. Likewise with (2), you can certainly rephrase an “ought sentence” into an “is sentence” but that doesn’t change it from a normative to a descriptive claim.
My question is about how an idealization procedure (like extrapolated volition or whatever else) can actually have moral authority if the whole procedure is specified in non-normative terms.
I agree with everything in this response. In particular, I don’t mean to “deny the ability to do some amount of logical inference on top of your preferences.”
My point is that it doesn’t answer the key metaethical question of why you ought to act according to any of those ideas.
For believers in scientific reductionism, moral realism based on a priori knowledge or fixed “human nature” or mental access to a realm of platonic moral truths is not plausible
Sure, but this is a case for nihilism or similar views.
In the moral realm, it may be that a combination of memetic and biological evolution would tend to cause strong convergence on certain norms in the long term
Sure, but
this doesn’t explain where the moral authority comes from, i.e. why you ought to follow the principles that could result from this process
in particular, the specific “evolutionary” formula invites an evolutionary debunking, because the theory of natural selection suggests that we converge on moral principles that tend to produce persistent societies or genes or similar, rather than ones which are morally good
a few of your points reference the “growth of knowledge” or “growth of reason or rationality” but I don’t see why (1) the described idealization procedure points toward those things or (2) why those things are good
This doesn’t really address my objection but just labels it.
If I understand correctly, Yudkowsky merely asserts that real moral knowledge is found by
running a certain logical function over possible states of the world, where this function is analytically identical to the result of extrapolating our current decision-making process in directions such as “What if I knew more?”, “What if I had time to consider more arguments (so long as the arguments weren’t hacking my brain)?”, or “What if I understood myself better and had more self-control?”
But this is an idealization procedure, and so it falls into my dichotomy:
In particular all of the theories based on an idealization procedure fail because either
The idealization procedure is taken to include moral knowledge, creating circularity, or
The idealization procedure only includes rationality in the making of non-moral judgments, knowledge of non-moral facts, etc, in which case this is a reductionist meta-ethics that doesn’t actually cross the is-ought gap (i.e. it remains an open question whether the idealized attitudes would be good).
I don’t see a clear moral/evaluative claim baked into the listed examples there, so therefore it maintains the problem of explaining why the outcome of the idealization procedure is actually good and why you ought to care about it, i.e. crossing the is-ought gap.
(My objection is similar or maybe the same to the open-question objection to analytic naturalism, of which analytic functionalism is one type.)
No, it just doesn’t assume that. It’s totally fine for different people to want different things, and for their extrapolated values to diverge, under Eliezer’s metaethics.
Ah ok. I admit I don’t know much about CEV, compared to the other two listed items in my top-level post. This document admits (emphasis):
Q9. How does the dynamic force individual volitions to cohere? (Frequently Asked)
The dynamic doesn’t force anything. The engineering goal is to ask what humankind “wants,” or rather what we would decide if we knew more, thought faster, were more the people we wished we were, had grown up farther together, etc. “There is nothing which humanity can be said to ‘want’ in this sense” is a possible answer to this question. Meaning, you took your best shot at asking what humanity wanted, and humanity didn’t want anything coherent.
It defines coherence as “Strong agreement between many extrapolated individual volitions which are unmuddled and unspread in the domain of agreement, and not countered by strong disagreement.” So while it is conceded, in passing, that there might not be a result, I assumed that Yudkowsky thinks it’s plausible, because otherwise it wouldn’t make sense to advocate for CEV as the target for AI alignment. (I guess it’s possible that he concluded that it would be better for an AI not to do anything in that case, as a safe failure mode, versus to act on a different alignment target.)
Clearly there are some actions you can take that make you think you will make better ethical judgements in the future.
This doesn’t address the is-ought gap. I agree that if you already accept moral realism then this kind of thing is a relevant consideration, but positing an idealization procedure doesn’t solve meta-ethics. Things like “sleeping enough” only corrects non-moral defects like fatigue but doesn’t address the question of whether the resulting judgments are objectively good. In contrast, the “be more the people you wished you were” in Yudkowsky’s idealization procedure introduces moral knowledge and values (insofar as “wishing” is an evaluative attitude), but that creates circularity.
In particular all of the theories based on an idealization procedure fail because either
The idealization procedure is taken to include moral knowledge, creating circularity, or
The idealization procedure only includes rationality in the making of non-moral judgments, knowledge of non-moral facts, etc, in which case this is a reductionist meta-ethics that doesn’t actually cross the is-ought gap (i.e. it remains an open question whether the idealized attitudes would be good).
My broader accusation is that this kind of talk is used for crypto-realism; people want to basically talk in terms of stance-independent moral facts. But they merely frame the discussion in terms of what their idealized self would believe, when in reality the idealization procedure is either circular or can’t cross the is-ought gap and introduce moral knowledge. You yourself just talked in terms of “progress on ethics” and “better ethical judgments” but by that you could mean either
“Progress on figuring out what my idealized self would think / what judgments he would make”—how does this illuminate any metaethics?
“Progress on figuring out objective, or stance-independent, ethics/judgments”—how would the idealized self be authoritative about that, especially if they diverge among people?
But what is the value of these “dispositions”? I certainly have some dispositions; for example, my intuition is that Mt. Kilimanjaro is more beautiful than a random pile of garbage.
This “extrapolation” concept assumes a bunch of stuff:
That if everyone underwent an idealization procedure, they would find some kind of common ground
That people should care, personally, about what their idealization procedure would produce
My point is that no “idealization procedure” solves the hard problem, which is crossing the is-ought gap i.e. going from facts about the world or about your impressions and deriving moral principles.
The age of peak democracy is already over. There was a period roughly from the American Revolution till the Spanish Civil War (i.e. 1776 to 1939) when popular revolts were a serious concern because the military technology favored strength in numbers even with amateur fighters. Since the 20th century, tanks and airplanes and rockets—expensive and specialist weaponry—allow even weak states to resist popular revolts (they can still suffer coups from low morale etc). Since the end of this period, democratic power has also declined, with legislatures subordinated to bureaucracies and where a global elite culture increasingly ignores the masses to institute unpopular policies. [Related (paywalled) reading from Ben Landau-Taylor.]
Europe is relatively less democratic (steamrolling the masses with unpopular policies like degrowth, mass immigration, etc. with less resistance than in America) while e.g. Latin America is more democratic, and America is in between. (In Latin America, elections are much more important because of their less complex bureaucracies, while I remember a funny German interview where a bureaucrat was asked for his thoughts on the upcoming election and he seemed not to understand the question, because of course he’s certain to maintain his post regardless of the results.)
AGI is a centralizing technology but with respect to democratic power I predict that it wouldn’t affect the trend because popular revolt is already an extremely limited concern, especially including factors not mentioned like the aging population.
I don’t see why the principle of dominance would give the wrong action. It just says that you should take an action if it is never improved by another action regardless of other actors.
Why is it obvious that you should one-box? Two-boxing is the dominant strategy.
It’s not clear what a “commitment” is this context. Usually people talk about “commitment devices” which constrain your options or change your future incentives, but just saying “I commit to one-boxing” doesn’t do anything like that.
Many LW people believe in one of a family of meta-ethical theories that don’t make sense to me.
Idealizing subjectivism (IdS): This theory says that X is intrinsically valuable, relative to an agent A, if and only if, and because, A would have some set of evaluative attitudes towards X, if A had undergone some sort of idealization procedure.” (definition from Joe Carlsmith who says “Idealizing subjectivism has been something like my best-guess meta-ethics. And lots of people I know take it for granted”).
Coherent extrapolated volition (CEV): Some say that ideally an AI should predict what people should want “if we knew more, thought faster, were more the people we wished we were, had grown up farther together.” The AI should use the desires that “converge” among everyone in some sense, but I also hear people talk about such-and-such person’s CEV (cf Habryka on “Vladimir Putin’s CEV”).
Ideal-observer theory (IOT): This is an academic theory that says that to say something is good is to say that an “ideal observer” would approve of it. Firth in “Ethical absolutism and the ideal observer” says this ideal observer should be omniscient with respect to natural facts, dispassionate, disinterested, consistent, “normal,” etc.
These are all framed slightly differently: IdS is an anti-realist theory of what to care about, CEV is about how to command an AI, and IOT is about what moral statements mean. But these theories don’t help with the hard problems with meta-ethics that they try to resolve or elide. In particular all of the theories based on an idealization procedure fail because either
The idealization procedure is taken to include moral knowledge, creating circularity, or
The idealization procedure only includes rationality in the making of non-moral judgments, knowledge of non-moral facts, etc, in which case this is a reductionist meta-ethics that doesn’t actually cross the is-ought gap (i.e. it remains an open question whether the idealized attitudes would be good).
I basically think these views are popular because while moral realism is not plausible, these idealization theories allow for crypto-realism where the exact same discussion is had but framed around this illusory target of our “idealized” selves, whose relevance or for whose actual convergence there isn’t any evidence.
Newcomb’s Problem creates an apparent conflict between
1) Dominance: pick the choice that never leaves you worse off
2) EV: pick the choice that maximizes your EV
Intuitively the dominance principle is more fundamental, and indeed it’s correct: EVM doesn’t mean you should do things that increase evidential probability 𝘳𝘦𝘭𝘢𝘵𝘪𝘷𝘦 𝘵𝘰 𝘺𝘰𝘶𝘳 𝘰𝘸𝘯 𝘪𝘯𝘧𝘰. And you can’t change the fixed past.
People who one-box are often interpreting the situation as if from an imagined earlier stage, similar to people who interpret the organ-transplant problem from an earlier stage of “deciding societal rules.” (Some have more exotic theories like causing things to happen backward in time.)
So the answer to the scenario is that the agent should two box—unrelatedly, if a different thought experiment presented you with the choice to “commit” (whatever that means) to one-boxing in such a scenario, then you should take that option.Some further reading:
Michael Huemer, 2021 — “The Solution to Newcomb’s Problem”
@basil.halperin, 2022 — “Newcomb’s problem is just a standard time consistency problem”
The last part surprised me because in general I don’t think of heard of considering whether other people make you “feel more calm and grounded [vs] unhappy and defensive” as an insight into whether those others are psychologically well. What is the theory of the connection there?
A 2019 study claims that spoken languages transmit info at a similar pace, i.e. that languages which apparently transmit more spoken info per syllable (Mandarin, German) are spoken more slowly than opposite languages like Japanese or Spanish. The gap between the fastest and lowest transmission (English and Thai) was 1.4x, though I don’t quite understand the way they measured information.
It’s possible that written communication doesn’t work that way, though. Languages with simpler and more rational orthographies like Spanish are easier to learn to read compared to English. So I think the most clear benefit would come from spelling reform and maybe the way numbers are spoken/written in words, rather than apparent efficiency improvements to spoken communication (e.g. you listed grammatical or vocabulary changes) that just lead people to talk more slowly because they’re bottlenecked by how fast they can produce or understand the information.
I can’t think of examples off the top of my head but I think that I often find myself wishing a certain react exist or noticing that none of the reacts indicate what I want.
Is this a distinction between expert and professional? For many fields, a merely professional level is way below an expert level, and for others that difference may just be less legible.
In my own case I saw your tweet and my thought was that a fifth skill is the “aura” of a notable politician or someone with a cult of personality, but it’s true that a merely “professional” politicians like a state assemblyman or city councilman may not have the same level of unreachable skills at all.
Bertrand Russell’s parents died by the time he was four years old and in 1876 he went to live with his grandfather, the last Whig prime minister, who as a young man had met Napoleon at Elba.
In 1966 at the age of 94 he met Paul McCartney and converted him to his anti-war stance on Vietnam.
Previously, I had figured that this lifespan (roughly 1870 to 1960) was the most extreme length of history for someone to live through. You have to be born early enough to remember a time when big cities didn’t have street lights or automobiles, while ideally living to see the atom bomb and first man in space (1961).
For example:
Churchill (1874–1965) rode in Britain’s last great cavalry charge (Omdurman 1898) and later in life ordered Britain’s first hydrogen-bomb program.
W.E.B. DuBois (1868–1963) was born three years after the Civil War but died before the March on Washington.
Other such lifespans were Laura Ingalls Wilder (1867–1957), Frank Lloyd Wright (1867–1959), W. Somerset Maugham (1874–1965), Picasso (1881–1973).
On Twitter people pointed outside the West, the same lifespan was even crazier, if they were born into totally pre-industrial worlds e.g.
“Syngman Rhee (1875-1965) who was one of the last people in Korea to pass the old Confucian civil service examinations and later became the President of South Korea, playing a pivotal role in the early Cold War … He was born in a preindustrial world where the old Confucian social order still held sway and nobody around him had ever seen a train and died in a world of jet planes and manned space travel.” @avrilbradley23
“A curious Surmic born then would have seen the forging of the Ethiopian Empire, the end of the witch-chiefs, Christian missionaries, the Italian conquest, literacy, new crops, and maybe the moon landing.” @Peter_Nimitz
Meanwhile I had figured I was going to live through the “Great Stagnation” and the world would gradually become a humongous nursing home. But it seems like AGI is likely to keep the show going.