CS PhD student
Abhimanyu Pallavi Sudhir
[Question] Political Roko’s basilisk
I’m not sure your interpretation of logical positivism is what the positivists actually say. They don’t argue against having a mental model that is metaphysical, they point out that this mental model is simply a “gauge”, and that anything physical is invariant under changes of this gauge.
One can absolutely construct a utility function for the robot. It’s a “shooting-blue maximizer”. Just because the appearing utility function is wrong doesn’t mean there isn’t a utility function.
Disagree. Daria considers the colour of the sky an important issue because it is socially important, not because it is of actual cognitive importance. Ferris recognizes that it doesn’t truly change much about his beliefs, since their society doesn’t have any actual scientific theories predicting the colour of the sky (if they did, the alliances would not be on uncorrelated issues like taxes and marriage), and bothers with things he finds to be genuinely more important.
This seems to be relevant to calculations of climate change externalities, where the research is almost always based on the direct costs of climate change if no one modified their behaviour, rather than the cost of building a sea wall, or planting trees.
[Question] Godel in second-order logic?
[Question] Utility functions without a maximum
[Question] A way to beat superrational/EDT agents?
- [deleted]
Meaningful things are those the universe possesses a semantics for
I agree, and one could think of this in terms of markets: a market cannot capture all information about the world, because it is part of the world.
But I disagree that this is fundamentally unrelated—here too the issue is that it would need to represent states of the world corresponding to what belief it expresses. Ultimately mathematics is supposed to represent the real world.
You overstate your case. The universe contains a finite amount of incompressible information, which is strictly less than the information contained in. That self-reference applies to the universe is obvious, because the universe contains computer programs.The point is the universe is certainly a computer program, and that incompleteness applies to all computer programs (to all things with only finite incompressible information). In any case, I explained Godel with an explicitly empirical example, so I’m not sure what your point is.
It’s really not, that’s the point I made about semantics.Eh that’s kind-of right, my original comment there was dumb.
That’s syntax, not semantics.
I don’t think that’s exactly true. But why do you think that follows from what I wrote?
I think that the philosophical questions you’re describing actually evaporate and turn out to be meaningless once you think enough about them, because they have a very anthropic flavour.
Betting on what is un-falsifiable and un-verifiable
conditionalization is not the probabilistic version of implies
P Q Q| P P → Q T T T T T F F F F T N/A T F F N/A T Resolution logic for conditionalization:
Q if P or True
Resolution logic for implies:
Q if P or None
current LLMs vs dangerous AIs
Most current “alignment research” with LLMs seems indistinguishable from “capabilities research”. Both are just “getting the AI to be better at what we want it to do”, and there isn’t really a critical difference between the two.
Alignment in the original sense was defined oppositionally to the AI’s own nefarious objectives. Which LLMs don’t have, so alignment research with LLMs is probably moot.
something related I wrote in my MATS application:
-
I think the most important alignment failure modes occur when deploying an LLM as part of an agent (i.e. a program that autonomously runs a limited-context chain of thought from LLM predictions, maintains a long-term storage, calls functions such as search over storage, self-prompting and habit modification either based on LLM-generated function calls or as cron-jobs/hooks).
-
These kinds of alignment failures are (1) only truly serious when the agent is somehow objective-driven or equivalently has feelings, which current LLMs have not been trained to be (I think that would need some kind of online learning, or learning to self-modify) (2) can only be solved when the agent is objective-driven.
-
The “Dutch books” example is not restricted to improper priors. I don’t have time to transform this into the language of your problem, but the basically similar two-envelopes problem can arise from the prior distribution:
f(x) = 1/4*(3/4)^n where x = 2^n (n >=0), 0 if x cannot be written in this form
Considering this as a prior on the amount of money in an envelope, the expectation of the envelope you didn’t choose is always 8⁄7 of the envelope you did choose.
There is no actual mathematical contradiction with this sort of thing—with prior or improper priors, thanks to the timely appearance of infinities. See here for an explanation:
https://thewindingnumber.blogspot.com/2019/12/two-envelopes-problem-beyond-bayes.html