xpym

Karma: 177

xpym May 27, 2025, 8:26 AM
1 point
0
in reply to: Wei Dai’s comment on: The stakes of AI moral status

Humanity has been collectively trying to solve some philosophical problems for hundreds or even thousands of years, without arriving at final solutions.

Instead of using philosophy to solve individual scientific problems (natural philosophy) we use it to solve science as a methodological problem (philosophy of science).

But humans seemingly do have indexical values, so what to do about that?

But humans don’t have this, so how are humans supposed to reason about such correlations?

I would categorize this as incorporating feedback from reality, so perhaps we don’t really disagree much.

xpym May 27, 2025, 5:29 AM
1 point
0
in reply to: Wei Dai’s comment on: The stakes of AI moral status

Like today, there are people who deny the existence of qualia altogether, and think it’s an illusion or some such, so I imagine there will also be people in the future who claim that the material features you claim to give rise to qualia experiences, merely give rise to reports of qualia experiences.

I mean, there are still people claiming that Earth is flat, and that evolution is an absurd lie. But insofar as consensus on anything is ever reached, it basically always requires both detailed tangible evidence and abstract reasoning. I’m not denying that abstract reasoning is necessary, it’s just far less sufficient by itself than mainstream philosophy admits.

I still have to figure out what my values should be, right? Is your position that it’s entirely arbitrary, and any answer is as good as another (within the range)?

We do have meta-preferences about our preferences, and of course with regard to our meta-preferences our values aren’t arbitrary. But this just escalates the issue one level higher—when the whole values + meta-values structure is considered, there’s no objective criterion for determining the best one (found so far).

How do I know this is true? What feedback from reality can I use to decide between “questions without feedback from reality can only be answered arbitrarily” and “there’s another way to (very slowly) answer such questions, by doing what most philosophers do”

You can evaluate philosophical progress achieved so far, for one thing. I’m not saying that my assessment of it is inarguably correct (indeed, given that mainstream philosophy isn’t seriously discredited yet, reasonable people clearly can disagree), but if your conclusions are different, I’d like to know why.

xpym May 26, 2025, 12:00 PM
2 points
0
in reply to: Ape in the coat’s comment on: Meditations on Doge

And even now, during the afformentioned war, people still manage to claim that at least it’s not as bad as the 90s.

Because it isn’t (yet), at least for those lucky enough not to be drafted, or living in the border regions.

In this sense, reasonable experts such as Joseph Stiglitz were completely vindicated.

Sure, the 90s could’ve gone better, but I doubt that anything could’ve stopped the KGB from ending up in power. Yeltsin was far too clueless to prevent that.

xpym May 24, 2025, 6:54 AM
2 points
0
in reply to: Wei Dai’s comment on: The stakes of AI moral status

what if there are certain problems which by their nature just don’t have clear and quick feedback from reality?

Seems overwhelmingly likely to me that those problems will remain unsolved, until such time as we figure out how that feedback can be acquired. An example of a long-standing philosophical problem that could eventually be solved in this way is the problem of consciousness: if we’re eventually able to build artificial brains and “upload” ourselves, by testing different designs we’d be able to figure out which material features give rise to qualia experiences, and by what mechanisms.

Like for example, what should my intrinsic (as opposed to instrumental) values be?

We do receive feedback on this from reality, albeit slowly — through cultural evolution/natural selection. To the extent that this filter isn’t particularly strict, within the range it allows variation will probably remain arbitrary.

how do you know we’re not just making progress slowly

Because there’s no consensus that any major long-standing philosophical problem has ever been solved through philosophical methods.

what should we do?

Figure out where we’re confused and stop making same old mistakes/walking in circles. Build better tools which expand the range of experiments we can do. Try not to kill ourselves in the meantime (hard mode).

xpym May 23, 2025, 8:08 AM
1 point
0
in reply to: RogerDearnaley’s comment on: The stakes of AI moral status

Evolutionary ethics similarly provides a clear answer to “what are the criteria for moral patienthood?” — since morality comes into existence via evolution

I basically agree, except I think that “evolution” needs to be replaced by “natural selection” there. Moral intuitions are essentially how we implement game theory, and cultures with better implementations outcompete others. But just because all of moral patients so far have emerged through biological evolution, it would be a grave mistake to conclude that it’s the only way they could ever come into being.

If we happen to create minds smarter than ourselves (by any means), competition with them obviously wouldn’t be outside of realm of possibility, and therefore appropriate attitudes to that will have to be developed, if we’re at all interested in our continued success in that old game of natural selection. Since we already have useful terms like “moral patient” floating around, I don’t see a good reason not to straightforwardly extend them to these novel agents.

xpym May 23, 2025, 4:33 AM
3 points
2
in reply to: Wei Dai’s comment on: The stakes of AI moral status

Who else is doing this?

MacAskill is probably the most prominent, with his “value lock-in” and “long reflection”, but in general the notion of philosophical confusion/inadequacy seems a common component of various AI risk cases. I’ve been particularly impressed by John Wentworth. (1, 2, 3)

All of your links are to people proposing better ways of doing philosophy, which contradicts that it’s impossible to make progress in philosophy.

The point is that it’s impossible to do useful philosophy without close and constant contact with reality. Your examples of influential philosophical ideas (abolition of slavery, the Enlightenment, Communism) were coincidentally all responses to clear and major observable problems (the horrors of slavery, sectarian wars and early industrial working conditions, respectively).

xpym May 22, 2025, 6:30 PM
3 points
2
in reply to: Wei Dai’s comment on: The stakes of AI moral status

If that’s the case, why aren’t they at least raising the alarm for this additional AI risk?

My impression is that those few who at least understand that they’re confused do that, whereas most are also meta-confused.

If this isn’t possible, then what are philosophers doing? Are they all just wasting their time?

Not exactly an unheard of position.

provide evidence to policymakers that it really is a hard problem

I don’t think that philosophy/metaphilosophy has a good track record of providing strong evidence for anything, so policymakers aren’t predisposed to taking arguments from those quarters seriously. I expect that only a really dramatic warning shot can change the AI trajectory (and even then it’s not a sure bet — Covid was plenty dramatic, and yet no significant opposition to gain-of-function seems to have materialized).

xpym May 22, 2025, 3:06 PM
4 points
1
in reply to: Wei Dai’s comment on: The stakes of AI moral status

It seems very puzzling to me that almost no one is working on increasing AI and/or human philosophical competence in these ways

It seems clear enough to me that pretty much everybody is hopelessly confused about these issues, and sees no promising avenues for quick progress. On that note, I’m interested in your answer to Conor Leahy’s question in a comment to the linked post:

“What kind of questions can you make progress on without constant grounding and dialogue with reality? This is the default of how we humans build knowledge and solve hard new questions, the places where we do best and get the least drawn astray is exactly those areas where we can have as much feedback from reality in as tight loops as possible, and so if we are trying to tackle ever more lofty problems, it becomes ever more important to get exactly that feedback wherever we can get it!”

I agree with his perspective, and am curious where and why you disagree.

xpym May 22, 2025, 2:51 PM
1 point
0
in reply to: RogerDearnaley’s comment on: The stakes of AI moral status

This gives us a rational, scientifically-derived answer to some moral questions: if we predict that humans will have evolved moral intuitions that give a clear and consistent answer to a moral question across cultures, then it actually has an answer (for humans).

This attitude presupposes that circumstances in which human cultures find themselves can’t undergo quick and radical change. Which would’ve been reasonable for most of history — change had been slow and gradual enough for cultural evolution to keep up with. But the bald fact that no culture ever had to deal with anything like AGI points to the fatal limitation of this approach — even if people happen to have consistent moral intuitions about it a priori (which seems very unlikely to me), there’s no good reason to expect those intuitions to survive actual contact with reality.

xpym Apr 2, 2025, 12:49 PM
4 points
0
in reply to: avturchin’s comment on: VDT: a solution to decision theory

The problem is that it will not work outside the distribution.

Of course, but neither would anything else so far discovered...

xpym Mar 25, 2025, 2:13 PM
2 points
0
on: Elite Coordination via the Consensus of Power

More generally, we can think of wokeness as a consensus of power under which it was common knowledge that leaders would give in to demands made by left-wing activists.

Not quite right. Demands for outright communism are still not taken seriously, and I don’t expect this to change any time soon. Wokeness is the particular fusion of the “social justice” brand of feminism and BLM, which Dem leaders embraced because it served the moral/ideological function while not fundamentally threatening hard power structures.

A very relevant Scott Alexander’s post that you didn’t mention here is New Atheism: The Godlessness That Failed, in which he claimed that New Atheism was a “failed hamartiology”:

We watched the US population elect George W Bush and act like this was a remotely reasonable thing to do. We saw people destroying the environment, leaving the poor to starve, and denying gay people their right to live as normal members of society. We saw people endorsing weird ideas and conspiracy theories, from homeopathy and creationism to the Clintons murdering their enemies. We were always vaguely aware from reading the newspapers that some of these people existed. But now we were seeing and conversing with them every day.

And so we asked ourselves: what the hell is wrong with these people?

And New Atheism had an answer: religion.

That was it. It was beautiful, it was simple, it was perfect. We were the “reality-based community”. [...]

Gradually the Blue Tribe got a little bit more self-awareness and realized this was not a great idea. Their coalition contained too many Catholic Latinos, too many Muslim Arabs, too many Baptist African-Americans. [...]

Between 2008 and 2016, two things happened. First, Barack Obama replaced George W. Bush as president. Second, Ferguson. The Blue Tribe kept posing its same identity question: “Who am I? What defines me?”, and now Black Lives Matter gave them an answer they liked better “You are the people who aren’t blinded by sexism and racism.”

Again, it was beautiful, simple, and perfect. We were “the reality-based community”. They were ignoring Reason and basing all of their opinions on blind hatred and prejudice. There was nothing confusing or unsettling at all about the situation, and we did not need to question any of our own beliefs. It was just that some people had been brainwashed by white supremacy and an all-consuming desire to protect their own privilege, and so they did. Sin began with the apple tree in Eden; conservatism began with the cotton plant in Jamestown. Language separates us from the apes; not being blinded by bigotry separates us from the Republicans.

The right wing in the West still hasn’t come up with a memetically competitive (but not suicidal) replacement for its own failed hamartiology. Nietzsche observed that God died almost a century and a half ago. Clearly, it’s not easy for conservatives to rebuild their ideology from scratch without that centerpiece. MAGA seems to be the right at its most successful currently, and as far as I can tell, it’s pretty much the “reversed stupidity” of wokeness, so your warning that “there’s no guarantee that the same forces won’t drive them equally crazy” is too coy/belated.

What this suggests is that we need liberalism 2.0: an ideological paradigm robust enough to rein in the excesses of the consensus of culture, but non-coercive enough that it can spread without sparking another culture war.

The more pessimistic among us fear that without a Thirty Years’ War-tier disaster this idea won’t be able to catch on. But, of course, an AI-driven omnicide/rapture/??? could come first… All in all, interesting times.

xpym Mar 11, 2025, 10:36 AM
1 point
0
in reply to: Tapatakt’s comment on: when will LLMs become human-level bloggers?

When I consider the bloggers I actually subscribe to versus those I just occasionally read

It’s amusing that it couldn’t help itself prefacing a reasonable take with an obvious lie, almost as if to make the point better.

Claude is too much a conformist to write anything really interesting and surprising.

Well, so is much of the mainstream media, and yet people seem happy enough to consume that stuff.

xpym Mar 11, 2025, 10:22 AM
1 point
0
in reply to: Viliam’s comment on: when will LLMs become human-level bloggers?

what about humans who would do the same thing?

Presumably, this happens: https://slatestarcodex.com/2016/12/12/might-people-on-the-internet-sometimes-lie/

I do often notice how the top upvoted reddit comment in big subs is confidently wrong, with a more correct/nuanced take sitting much lower.

xpym Feb 17, 2025, 11:04 AM
9 points
8
on: Virtue signaling, and the “humans-are-wonderful” bias, as a trust exercise

Insofar as it’s required for you to pretend that people are nicer than they are to be kind to them, I think you should do that. But your impact will be better if you at least note it if that’s what you’re doing

Unlikely to work for Mr. Portman. Living the life of systematic lies and pretension is difficult and cognitively demanding. Being pro-socially self-deceived to some degree is a much simpler strategy, which is probably why evolution converged on it (paired with some amount of psychopathy to balance/exploit excesses).

policy analysis

Most people obviously aren’t cut out for that, and are happier for it, if they live in a reasonably high-trust society.

xpym Jan 17, 2025, 2:47 PM
1 point
0
in reply to: Steven Byrnes’s comment on: What Is The Alignment Problem?

I bet the person says “no”.

I agree, but I think it’s important to mention issues like social desirability bias and strategic self-deception here, coupled with the fact that most people just aren’t particularly good at introspection.

it’s conflicting desires, not conflicting values

It’s both, our minds employ desires in service of pursuing our (often conflicting) values.

Insofar as different values are conflicting, that conflict has already long ago been resolved, and the resolution is: the action which best accords with the person’s values, in this instance, is to get up.

I’d rather put it as a routine conflict eventually getting resolved in a predictable way.

Another example: if someone says “I want to act in accordance with my values” or “I don’t always act in accordance with my values”, we recognize these as two substantive claims. The first is not a tautology, and the second is not a self-contradiction.

Indeed, but I claim that those statements actually mean “I want my value conflicts to resolve in the way I endorse” and “I don’t always endorse the way my value conflicts resolve”.

xpym Jan 17, 2025, 10:21 AM
3 points
2
in reply to: Steven Byrnes’s comment on: What Is The Alignment Problem?

the feeling of my head remaining on the pillow is motivating, but the self-reflective idea of myself being in bed is demotivating

This seems to be an example of conflicting values, and its preferred resolution, not a difference between a value and a non-value. Suppose you’d find your pillow replaced by a wooden log—I’d imagine that the self-reflective idea of yourself remedying this state of affairs would be pretty motivating!

xpym Dec 25, 2024, 10:48 AM
8 points
3
in reply to: Carl Feynman’s comment on: What are the strongest arguments for very short timelines?

For at least six months now, we’ve had software assistants that can roughly double the productivity of software development.

Is this the consensus view? I’ve seen people saying that those assistants give 10% productivity improvement, at best.

In the last few months, there’s been a perceptible increase in the speed of releases of better models.

On the other hand, the schedules for headline releases (GPT-5, Claude 3.5 Opus) continue to slip, and there are anonymous reports of diminishing returns from scaling. The current moment is interesting in that there are two essentially opposite prevalent narratives barely interacting with each other.

xpym Dec 20, 2024, 7:45 AM
1 point
0
in reply to: Jiro’s comment on: Effective Altruism FAQ

the principles of EA imply that

The principles of Christianity not only imply that, they clearly spell it out: “If you want to be perfect, then go and sell your possessions and give the money to the poor”, and yet Christianity was uncontroversial in the West for centuries, and the current secular “common sense” morality hasn’t diverged particularly far. EAs just take ostensibly common sense principles far too seriously compared to the unspoken social consensus, in a way that’s cringe for normal people. Critics don’t really have a principled response to the core EA ideas either, but they don’t want to appear morally delinquent, so they generally try to dismiss EAs without seriously engaging.

xpym Dec 17, 2024, 2:02 PM
2 points
0
on: Circling as practice for “just be yourself”
When circling was first discussed here, there was a comment that led to a lengthy discussion about boundaries, but nobody seemed to dispute its other main claim, that “it is highly unlikely that [somebody] would have 3-11 people they reasonably trusted enough to have [group] sex with”. Do you agree with that statement, and if so, do you think that the circling/sex analogy is invalid?

xpym Dec 9, 2024, 1:10 PM
1 point
0
on: Neutrality

And it may not be our permanent condition. The future may hold something more like a “foundation” or a “framework” or a “system of the world” that people actually trust and consider legitimate.

Our current condition is a product of our material circumstances, and those definitely aren’t permanent in their essential character, as many people have variously noted. Things are still very much in flux, and any eventual medium-to-long term frameworks would significantly depend on (possibly wildly divergent) trajectories that major trends will take in the foreseeable future. Of course, even marginal short-term improvements to the memetic hellscape would be welcome, but seriously expecting something substantial and lasting is premature.