I’m interested in axiology/value/utility—what things are valuable and why and when and how and to whom?
ihatenumbersinusernames7
Karma: 55
I agree, with the one caveat that I think there is a detectable difference in a moral world and an amoral world. But the difference is normative and not absolute. If someone thinks misery is just as good as flourishing, then there’s no way to differentiate.
...but what about someone who doesn’t believe in non-contradtiction? He is logically confused; the misery-preferer is normatively confused.
When you say “2 + 2 = 4” and then find 2 of something and 2 more and count 4, a logical intuition is instantiated.
When you say, “It would be good to help that old lady across the street,” and then you don’t help her, and regret it, a moral intuition is instantiated.
...I think?
Agreed that empirical knowledge can be tested, but what about knowledge not gained through the senses? You believe in the law of non-contradiction, right? There’s no experiment that can be run to prove it’s true. Maybe logical beliefs are similar to moral beliefs?
“Since a reward function external to AlphaZero, that “value” doesn’t change the way ours do since our values are within us.”
I still wonder if AlphaZero’s instrumental values can change...but I guess, since the reward function is fixed and external, optimizing based that reward keeps the instrumental values from getting weird.
Or here’s a better intuition pump: argue for something you believe in vehemently, but don’t act like it’s actually true.
Re: moral realism, I’ve got 2 for you:
Think about whatever you believe is right (or wrong) that you would argue for. You probably don’t argue that it’s just your opinion.
We can’t prove modus ponens or that A=A. We can’t prove that we’re not in a simulation. Therefore math and induction and science are all theories. We can’t have certainty. But we have intuitions about logic as well as morality.
By contrast, if you are in the endgame of a game of chess, or in an oligopolistic competition, you absolutely need to think about how other players will respond to your actions. You do need to search through a tree of “if I do this, that will happen” and compute that specifically for the specific game state you’re in, and recompute it every time the game state changes.
Since chess is bounded you really can calculate (and in the endgame you’re now able to). With oligopolistic competition, your power has increased and the players that matter to you are fewer, but the moves are not bounded—if Apple and Dell (or whoever) are in an oligopoly but then Apple creates the iPhone...things can change.
Another helpful way of putting #2 is: “Given that she can only win once”
As many have pointed out in the past, if this is the case, in the tails scenario she always wins on Mon(tails), so Tues(tails) never actually happens, because the game is over.
It will remain unsolved as long as reasonable people continue to disagree.
I’m not sure people disagree:
Given [her waking up], not knowing what day it is, she’s in the tails world 2⁄3 of the time.
Given [we’re about to play this game], there’s a 50⁄50 chance of heads or tails. So whatever day she wakes up on gives you no further information.
Does anyone disagree?
I really think the lack of a “moment” is key. It’s been closer to death by 1,000 papercuts...even on LW there’s not a consensus, e.g. everyone screaming from the rooftops, “oh, this Mythos thing is the tipping point, look out!!!”
Here are the best answers I have put together so far:
“1. Why don’t AlphaZero’s values change? When humans update and learn, our values do change.”
AlphaZero’s reward function remains static because it is external to AlphaZero’s weights. AlphaZero can change its weights to improve at achieving reward, but the reward function remains unchanged. (Reward hacking could be a potential issue for an AlphaZero-type system, but is greatly mitigated since within chess the rules are well defined.)
Since a reward function external to AlphaZero, that “value” doesn’t change the way ours do since our values are within us.
“2. I have some values that don’t seem to change (Like, I can’t seem to desire my own misery/suffering). Is the reason for that similar?”
No, there’s a big difference: our values are within us—but can they really not change? Maybe they can, but they just don’t (in neurotypical people?). Or maybe they do change / develop, but in subtle ways like this comment suggests.
Of course, I would never expect the majority of people to do something useful at such a time. But yes, I expected them to do something, or at least to fear the gravity of the moment, with at least some implications for their lives.
Did you really? I’d love to hear a longer account of what you expected this average Joe to do...and in response to what moment, exactly? ChatGPT? DeepSeek? OpenClaw? Anthropic ARR hitting $40b? I’m just not sure there’s been a Cuban Missle Crisis type of event to awaken this median person.
[Question] Did OpenClaw cause an update to your priors? Have there been other such moments since?
unless you are primarily studying the proponents rather than the ideas.
It seems you are studying the proponents, since the “fleshman” is their median belief/argument.
Yes, they’re not as far gone as the strawman would suggest. They understand nutrition. But also they’re not as conservative and rigorous in their considerations as the original study.
I just want to point out that it’s their belief(s) that you’re evaluating, not their arguments (steel or straw) for those beliefs.
Belief 1: Apples are healthy.
Belief 2: We should redirect funding.
Why do you need to know the group’s median argument/justification? You can just evaluate each belief.
there are a bunch of places where it’s possible to be reasonably confident despite strong disagreement between well-meaning experts.
the central cases I’m thinking about only involve non-foolish mistakes by experts.
we both think one should be quite suspicious of oneself if you’re more confident than experts on a controversial question...[but] the amount of suspicion is finite and can occasionally be overcome by object-level considerations.
That would imply the experts are not in agreement about the object level issue? Or that they aren’t incorporating it into their model of some meta-level issue?
One more to correct: “He then addresses the parent/car/coin”
Micron is still my favorite stock pick. Its impressive guidance caused analysts to raise 2026 earnings forecasts by 78%. That seems almost as dramatic as the NVIDIA’s May 2023 guidance which kicked off a 4x return over the next year.
Congrats on a great pick, up another ~50% ytd. What’s your mental model on when to back off on something like Micron? In previous cycles they look classically cheap (ie, high single digit PE based on cycle-high net margins) right before supply catches up to demand and the cycle turns for them. But obviously this cycle may be...well, “different.”
One thing I’m interested in is HBM, which I know is a small part of their revenue now, but seems like it will be expanding (something like exponentially) for years to come.
These trends and how the market will treat them are of course really hard to predict, but you’ve been exceptionally right on Micron so far so I’m very interested in your thoughts.
Thank you. I actually meant to make it a question. I think I will try to change that and see if it helps. I’m not sure I completely understand the use cases for questions vs posts.
Well, it tracks the truth that human values do exist, even if they vary somewhat. But agreed that it doesn’t tell you that there’s a source for normative values, or what that source might be.
Such a good point. Instrumental values are in the realm of logic (logically derived from terminal values) -- so given a terminal value, we can be logically right or wrong about the instrumental values that follow.
I have wondered (but perhaps haven’t stated clearly) if it is a fact about conscious minds that we have but one terminal value—a vague desire for “the good.” All our other values, even health, love, and not-misery, are instrumentally derived. If it is a fact that minds desire what they deem good (not that I can prove it), would that sway you at all toward moral realism?
In other words if it is a fact about minds that they desire the good, then the question becomes not “does goodness exist” but “what is good?”
Or would you just say well that makes goodness real for minds, but not objectively? This may be fair, but what does “objectively” mean in this case?