I mean that there isn’t a property of logical counterfactuals in the universe itself. However, once we’ve created a model (/map) of the universe, we can then define logical counterfactuals as about asking a particular question about this model. We just need to figure out what that question is.
You’ve explained the system. But what’s the motivation behind this?
Even though I only have a high level understanding of what you’re doing, I generally disagree with this kind of approach on a philosophical level. It seems like you’re reifying logical counterfactuals, when I see them more like an analogy, ie. positing a logical counterfactual is an operation that takes place on the level of the map, not the territory.
Can you tell me more about your ideas related to logical counterfactuals? They’re an area of been working on as well.
I’m hugely in favour of this. There have been quite reasonable questions raised about how much Less Wrong improves us and how much it sucks up our time.
Okay, so there is an additional assumption that these strings are all encoded as infinite sequences. Instead, they could be encoded with a system that starts by listing the number of digits or −1 if the sequence if infinite, then provide those digits. That’s a pretty key property to not mention (then again, I can’t criticise too much as I was too lazy to read the PDF). Thanks for the explanation!
I agree that “Why is this rock this rock instead of that rock?” is a good place to start, even if they aren’t perfectly analogous. Now, it isn’t entirely clear what is being asked. The first question that we could be asking is: “Why is this rock the way that it is instead of the way that rock is?”, in which case we could talk about the process of rock formation and the rock’s specific history. Another question we could be asking is, “Why is this rock here at this time instead of that rock?” and again we’d be talking about history and how people or events moved it. We could even make anthropic arguments, “This rock isn’t a million degrees because if it were that hot it would not longer be a rock” or “This rock isn’t a diamond and this is unsurprising as they are rare”. Here we’d be asking, “Given a random rock, why are we most likely to be observing certain characteristics?”
One difference with the human example is that the human is asking the question, “Why am I me instead of someone else?” So you can also reason about your likely properties on the basis of being the kind of being who is asking that question. Here the question is interpreted as, “Why is the entity asking this question this entity instead of another entity?”.
Another issue which becomes clearer is the symmetry. Barrack Obama might ask, “Why am I me instead of the Pope?” whilst at the same time the Pope asks, “Why am I me instead of Barrack Obama?”. So even if you had been someone else, you might very well of been asking the same question. I think this ties well into the notion of surprise. Let’s suppose a million people receive a social security number and you receive 235,104. You might argue, “How surprising, there was only a one in a million chance of receiving this number!”. However you could have said this regardless of which number you’d been given, so it isn’t that surprising after all.
Another question that could be asked is, “Why is my consciousness receiving the qualia (subjective experience) from this physical body?” In this case, the answer depends on your metaphysics. Materialists would say this is a mistaken question as qualia don’t exist. Christianity might say it’s because God chose to attach this soul to this body. Other spiritual theories might have souls floating around which inhabit any body which is free (although this raises questions such as: what if no soul chooses to inhabit a body and which soul gets to inhabit which body). Lastly, there’s theories like property dualism where the consciousness is a result of the mental properties of particles so that the consciousness corresponding to any one particular body couldn’t be attached to anyone else without breaking the laws of the universe. So as described in my post Natural Structures and Definitions, this last interpretation is one of those questions that is conditionally meaningful to ask.
Hmm, I’m still not following. Limits are uncomputable in general, but I just need one computational function where I know the limits at one point and then I can set it to p+1 instead. Why wouldn’t this function still be computable? Maybe “computable function” is being defined differently than I would expect.
My understanding of the arguments against using a utility maximiser is that proponents accept that this will lead to sub-optimal or dominated outcomes, but they are happy to accept this because they believe that these AIs will be easier to align. This seems like a completely reasonable trade-off to me. For example, imagine that choosing option A is worth 1 utility. Option B is worth 1.1 utility if 100 mathematical statements are all correct, but −1000 otherwise (we are ignoring the costs of reading through and thinking about all 100 mathematical statements). Even if each of the statements seems obviously correct, there is a decent chance that you messed up on at least 1 of them, so you’ll most likely want to take the outside view and pick option A. So I don’t think it’s necessarily an issue if the AI is doing things that are obviously stupid from an inside view.
“Because all computable functions are continuous”—how does this make any sense? Why can’t I just pick a value x=1 and if it’s left limit and right limit are p, set the function to p+1 at x=1.
“Is a fleeting emotional sense of certainty over 1 minute, worth automatically discarding the potential $5-million outcome?”—I know it’s mostly outside of what is being modelled here, but suspect that someone who takes the 90% bet and wins nothing might experience much more than just a fleeting sense of disappointment, much more than someone who takes the 45% chance and doesn’t win.
Do you have any empirical evidence?
This is a really good point, I’m disappointed that he didn’t respond to it.
“The propensity and frequentist views regard as nonsense the notion that we could talk about the probability of a mathematical fact”—couldn’t a frequentist define a reference class using all the digits of Pi? And then assume that the person knows nothing about Pi so that they throw away the place of the digit?
What did he believe changed?
Sure you have to be using the word in some way, but there’s not guarantee that there’s a meaningful concept that can be extracted from it or whether the term is just used in ways that are hopelessly confused.
“You don’t need to guess what someone means, or what level of discussion they’re looking for”—yes, that is part of the point of providing possible interpretations—to help with the clarification
“It is the same if you ask “What is consciousness?” You must already being using that word to refer to some thing or phenomenon that you are familiar with, and what you are asking about is what that thing is made of, or how that thing works, questions about the world, not about words”—philosophy discussions ask this all the time without presuming that the definition is already known.
The appropriate calculation is whether the marginal value of the upgrade is > $500, where the value of $500 is the marginal value of having that additional money.j
Yep, it can be assigned that if you use the fixed point definition of truth.
Didn’t work, just showed the triple ticks