red75prime

Karma: 68

red75prime 8 Jan 2026 10:52 UTC
1 point
0
in reply to: RussellThor’s comment on: Two Aspects of Situational Awareness: World Modelling & Indexical Information
If we are talking about the real world (to the best of our current knowledge, yada, yada) and not its classical approximation, we have the universal wavefunction as the world model, which is independent of agent’s actions as it encompasses them all.
Interacting with the world (by generating a specific pattern) allows to narrow down indexical uncertainty to all the agents that generate the pattern with non-zero probability.

red75prime 27 Oct 2025 3:15 UTC
1 point
0
in reply to: jessicata’s comment on: Homomorphically encrypted consciousness and its implications
When I said “subjective existence” I’ve meant some model where we don’t need a list of minds or exhaustive search for minds to make them real. After all the brain has its own computing power and requiring additional compute or data to make subjective experiences associated with its computations real looks extraneous. Interactions of a mind with our world, on the other hand, seem crucial for our ability to determine its existence.
BTW, thank you for laying out all this in such detail. It makes reasoning much more focused.

red75prime 26 Oct 2025 10:31 UTC
1 point
0
on: Homomorphically encrypted consciousness and its implications
Step 2 codifies objective existence of subjective states. But let’s suppose that homomorphic computation can be decrypted in two ways: one is what we encoded and the output is something like “it feels real”, other is a minimally conscious state that happened to exist when decoding with a different key and its output is a noisy grunt expressing dissatisfaction with the noisy environment. Should the second one be included in M?
It seems that g and h cannot be efficiently computable if we decide to include the second state into M. On the second thought, if we don’t have a list of minds in R, we need to analyze all the (spatially localized?) subsets of R to decide which ones of them are conscious. Could it be done efficiently?
ETA: Also, how to codify subjective existence of subjective states?

red75prime 19 Apr 2024 21:35 UTC
1 point
0
on: When is a mind me?
What concrete fact about the physical world do you think you’re missing? What are you ignorant of?
Let’s flip very unfair quantum coin with 1:2^1000000 heads to tails chances (that would require quite an engineering feat to prepare such a quantum state, but it’s theoretically possible). You shouldn’t expect to see heads if the quantum state is prepared correctly, but the post-flip universe (in MWI) contains a branch where you see heads. So, by your logic, you should expect to see both heads and tails even if the state is prepared correctly.
What I do not know is how it all ties together. MWI is wrong? Copying is not equivalent to MWI branching (thanks to the no-cloning theorem, for example)? And so on

red75prime 12 Apr 2024 22:15 UTC
4 points
3
in reply to: Steven Byrnes’s comment on: Ackshually, many worlds is wrong
“Thread of subjective experience” was an aside (just one of the mechanisms that explains why we “find ourselves” in a world that behaves according to the Born rule), don’t focus too much on it.
The core question is which physical mechanism (everything should be physical, right?) ensures that you almost never will see a string of a billion tails after a billion quantum coin flips, while the universe contains a quantum branch with you looking in astonishment on a string with a billion tails. Why should you expect that it will almost certainly not happen, when there’s always a physical instance of you that will see it happened?
You’ll have 2^1000000000 branches with exactly the same amplitude. You’ll experience every one of them. Which physical mechanism will make it more likely for you to experience strings with roughly the same number of heads and tails?
In the Copenhagen interpretation it’s trivial: when the quantum coin flipper writes a result of the flip the universe somehow samples from a probability distribution and the rest is the plain old probability theory. You don’t expect to observe a string of a billion tails (or any other preselected string), because you who observes this string almost never exist.
What happens in MWI?

red75prime 12 Apr 2024 19:12 UTC
2 points
1
in reply to: Steven Byrnes’s comment on: Ackshually, many worlds is wrong
I haven’t fully understood your stance towards the many minds interpretation. Do you find it unnecessary?
I don’t think either of these Harrys is “preferred”.
And simultaneously you think that existence of future Harries who observe events with probabilities approaching zero is not a problem because current Harry will almost never find himself to be those future Harries. I don’t understand what it means exactly.
Harries who observe those rare events exist and they wonder how they found themselves in those unlikely situations. Harries who hadn’t found anything unusual exist too. Current Harry became all of those future Harries.
So, we have a quantum state of the universe that factorizes into states with different Harries. OK. What property distinguished a universe where “Harry found himself in a tails branch” and a universe where “Harry found himself in a heads branch”?
You have already answered it: “I don’t think either of these Harrys is “preferred”.” That is there’s no property of the universe that distinguishes those outcomes.
Let’s get back to the initial question ‘What it means that “Harry will almost never find himself to be those future Harries”?’ To answer that we need to jump from a single physical Universe (containing multitude of Harries who found themselves in branches of every possible probability) to a single one (or maybe a set) of those Harries and proclaim that, indeed, that Harry (or Harries) found himself in a usual branch of the universe and all other Harries don’t matter for some reason (their amplitudes are too low to matter despite them being fully conscious? That’s the point that I don’t understand).
The many minds interpretation solves this by proposing metaphysical threads of consciousness, thus adding a property that distinguishes outcomes where Harry observes different things. So we can say that indeed the vast majority of Harries’ threads of consciousness ended up in probable branches.
I don’t like this interpretation. Why don’t we use a single thread of consciousness that adheres to Born rule? Or why don’t we get rid of threads of consciousness altogether and just use the Copenhagen interpretation?
So, my question is how you tackle this problem? I hope I’ve made it sufficiently coherent.
My own resolution is that either collapse is objective, or due to imperfect decoherence the vast majority of branches (which also have relatively low amplitude) interfere with each other, making it impossible for conscious beings to exist in them and, consequently, observe them (it doesn’t explain billion quantum coin-flips scenario in my comment below)

red75prime 12 Apr 2024 9:50 UTC
2 points
1
in reply to: Steven Byrnes’s comment on: Ackshually, many worlds is wrong
For example: “as quantum amplitude of a piece of the wavefunction goes to zero, the probability that I will ‘find myself’ in that piece also goes to zero”
What I really don’t like about this formulation is extreme vagueness of “I will find myself”, which implies that there’s some preferred future “I” out of many who is defined not only by observations he receives, but also by being a preferred continuation of subjective experience defined by an unknown mechanism.
It can be formalized as the many minds interpretation, incurring additional complexity penalty and undermining surface simplicity of the assumption. Coexistence of infinitely many (measurement operators can produce continuous probability distributions) threads of subjective experience in a single physical system also doesn’t strike me as “feeling more natural”.

red75prime 18 Mar 2024 9:02 UTC
2 points
0
on: What is the best argument that LLMs are shoggoths?
First, a factual statement that is true to the best of my knowledge: LLM state, that is used to produce probability distribution for the next token, is completely determined by the state of its input buffer (plus a bit of indeterminism due to parallel processing and non-associativity of floating point arithmetic).
That is LLM can pass only a single token (around 2 bytes) to its future self. That follows from the above.
What comes next is a plausible (to me) speculation.
For humans what’s passed to our future self is most likely much more that a single token. That is a state of the human brain that leads to writing (or uttering) the next word most likely cannot be derived from a small subset of a previous state plus a last written word (that is state of the brain changes not only because we had written or said a word, but by other means too).
This difference can lead to completely different processes that LLM uses to mimic human output, that is potential shoggethification. But to be the real shoggoth LLM also needs a way to covertly update its shoggoth state, that is the part of its state that can lead to inhuman behavior. Output buffer is the only thing it has to maintain state, so the shoggoth state should be steganographically encoded in it, thus severely limiting its information density and update rate.
I wonder how a shoggoth state may arise at all, but it might be my lack of imagination.

red75prime 13 Mar 2024 19:46 UTC
1 point
0
in reply to: red75prime’s comment on: 0th Person and 1st Person Logic
Expanding a bit on the topic.
Exhibit A: flip a fair coin and move a suspended robot into a green or red room using a second coin with probabilities (99%, 1%) for heads, and (1%, 99%) for tails.
Exhibit B: flip a fair coin and create 99 copies of the robot in green rooms and 1 copy in a red room for heads, and reverse colors otherwise.
What causes the robot to see red instead of green in exhibit A? Physical processes that brought about a world where the robot sees red.
What causes a robot to see red instead of green in exhibit B? The fact that it sees red, nothing more. The physical instance of the robot who sees red in one possible world, could be the instance who sees green in another possible world, of course (physical causality surely is intact). But a robot-who-sees-red (that is one of the instances who see red) cannot be made into a robot-who-sees-green by physical manipulations. That is subjective causality of seeing red is cut off from physical causes (in the case of multiple copies of an observer). And as such cannot be used as a basis for probabilistic judgements.
I guess that if I’ll not see a resolution of the Anthropic Trilemma in the framework of MWI in about 10 years, I’ll be almost sure that MWI is wrong.

red75prime 12 Mar 2024 9:46 UTC
1 point
−1
in reply to: Wei Dai’s comment on: 0th Person and 1st Person Logic
I have a solution that is completely underwhelming, but I can see no flaws in it, besides the complete lack of definition of which part of the mental state should be preserved to still count as you and rejection of MWI (as well as I cannot see useful insights into why we have what looks like continuous subjective experience).
1. You can’t consistently assign probabilities for future observations in scenarios where you expect creation of multiple instances of your mental state. All instances exist and there’s no counterfactual worlds where you end up as a mental state in a different location/time (as opposed to the one you happened to actually observe). You are here because your observations tells you that you are here, not because something intangible had moved from previous “you”(1) to the current “you” located here.
2. Born rule works because MWI is wrong. The collapse is objective and there’s no alternative yous.
(1) I use “you” in scare quotes to designate something beyond all information available in the mental state that presumably is unique and moves continuously (or jumps) thru time.
Let’s iterate through questions of The Anthropic Trilemma.
1. The Boltzmann Brain problem: no probabilities, no updates. Observing either room doesn’t tell you anything about the value of the digit of pi. It tells you that you observe the room you observe.
2. Winning the lottery: there’s no alternative quantum branches, so your machinations don’t change anything.
3. Personal future: Britney Spears observes that she has memories of Britney Spears, you observe that you have your memories. There’s no alternative scenarios if you are defined just by the information in your mental state. If you jump off the cliff, you can expect that someone with a memory of deciding to jump off the cliff (as well as all other your memories) will hit the ground and there will be no continuation of this mental state in this time and place. And your memory tells you that it will be you who will experience consequences of your decisions (whatever the underlying causes for that).
Probabilistic calculations of your future experiences work as expected, if you add “conditional on me experiencing staying here and now”.
It’s not unlike operator “do(X=x)” in Graphical Models that cuts off all other causal influences on X.

red75prime 17 Feb 2024 11:07 UTC
2 points
0
in reply to: datawitch’s comment on: I played the AI box game as the Gatekeeper — and lost
Do you think the exploited flaw is universal or, at least, common?

red75prime 14 Feb 2024 8:35 UTC
9 points
3
on: Scale Was All We Needed, At First
Excellent story. But what about “pull the plug” option? ALICE found a way to run itself efficiently on the traditional datacenters that aren’t packed with backprop and inference accelerators? And shutting them down would have required too strong a political will than what the government could muster at the time?

red75prime 30 Jan 2024 15:54 UTC
3 points
2
in reply to: habryka’s comment on: Is a random box of gas predictable after 20 seconds?
Citing https://arxiv.org/abs/cond-mat/9403051: “Furthermore if a quantum system does possess this property (whatever it may be), then we might hope that the inherent uncertainties in quantum mechanics lead to a thermal distribution for the momentum of a single atom, even if we always start with exactly the same initial state, and make the measurement at exactly the same time.”
Then the author proceed to demonstrate that it is indeed the case. I guess it partially answers the question: quantum state thermalises and you’ll get classical thermal distribution of measurement results of at least some measurements even when measuring the system in the same quantum state.
The less initial uncertainty in energy the faster the system thermalises. That is to slow quantum thermalisation down you need to initialize the system with atoms in highly localized positions, but then you can’t know their exact velocities and can’t predict classical evolution.

red75prime 26 Jan 2024 8:43 UTC
1 point
0
in reply to: habryka’s comment on: Is a random box of gas predictable after 20 seconds?
we are assuming that without random perturbation, you would get 100% accuracy
That is the question is not about the real argon gas, but about a billiard ball model? It should be stated in the question.

red75prime 22 Jan 2024 10:41 UTC
1 point
0
on: Another Non-Anthropic Paradox: The Unsurprising Rareness of Rare Events
here are creatures in the possible mind space^[3] whose intuition works in the opposite way. They are surprised specifically by the sequence of $H H T H T T H T T H$ and do not mind the sequence of $H H H H H H H H H H$
That is creatures who aren’t surprised by outcomes of lower Kolmogorov complexity or not surprised by the fact that the language they use for estimation of Kolmogorov complexity has a special compact case for producing “HHTHTTHTTH”.
Looks possible, but not probable.

red75prime 21 Jan 2024 15:27 UTC
2 points
0
on: You can rack up massive amounts of data quickly by asking questions to all your friends
For returns below $2000, I’d use ⁵⁰⁄₅₀ quantum random strategy just for fun of dropping Omega’s stats.

red75prime 6 Dec 2023 12:46 UTC
3 points
2
on: Some quick thoughts on “AI is easy to control”
what happens if we automatically evaluate plans generated by superhuman AIs using current LLMs and then launch plans that our current LLMs look at and say, “this looks good”.
The obvious failure mode is that LLM is not powerful enough to predict consequences of the plan. The obvious fix is to include human-relevant description of the consequences. The obvious failure modes: manipulated description of the consequences, optimizing for LLM jail-breaking. The obvious fix: …
I won’t continue, but shallow rebuttals is not that convincing, but deep ones is close to capability research, so I don’t expect to find interesting answers.

red75prime 5 Dec 2023 10:53 UTC
1 point
0
in reply to: Pazzaz’s comment on: 2023 Unofficial LessWrong Census/Survey
What if all I can assign is a probability distribution of probabilities? Like in extraterrestrial life question. All that can be said is that extraterrestrial life is sufficiently rare to not find evidence of it yet. Our observation of our existence is conditioned on our existence, so it doesn’t provide much evidence one way or another.
Should I sample the distribution to give an answer, or maybe take mode, or mean, or median? I’ve chosen a value that is far from both extremes, but I might have done something else with no clear justification for any of the choices.

red75prime 24 Nov 2023 9:45 UTC
1 point
0
on: How LLMs Can Show Self-Serving Bias
This means that LLMs can inadvertently learn to replicate these biases in their outputs.
Or the network learns to trust more the tokens that were already “thought about” during generation.

red75prime 29 Jun 2023 7:09 UTC
1 point
−1
in reply to: Nox ML’s comment on: Why am I Me?
Suppose when you are about to die [...] Omega shows up
Suppose something pertaining more to the real world: if you think that you are here and now because there will not be significantly more people in the future, then you are more likely to become depressed.
Also, why Omega uses 95% and not 50%, 10%, or 0.000001%?
ETA: Ah, Omega in this case is an embodiment of the litany of Tarski. Still, if there will be no catastrophe we are those 5% who violate the litany. Not saying that the litany comes closest to useless as it can get when we are talking about a belief in an inevitable catastrophe you can do nothing about.