Interested in math, Game Theory, etc.
Pattern
Karma: 2,075
I think the point was distinguishing between reasonable and unreasonable replies to people you value communication with. All else being equal—in the event using such a word doesn’t move them from out of the reasonable category for you—a preference is being given for not doing these things.
(I still do similar things with people who value it, because I know people who enjoy talking about language, but otherwise I don’t find it as time effective as ‘Don’t Feed The Trolls’, though this tends to be easier said than done, and communicating with people who have poor epistemic/conversational, etc. standards can be difficult (due to inferential distance, or other priorities) - some people go on about today being a ‘post-truth’ era, because they’ve talked to too many people who just don’t care about truth. There’s a feeling you get when you walk away from a discussion knowing that neither of you got anything out of it, and communication didn’t take place. Related links.)
Following this post’s policy can increase the utility of reasonable people you talk to, and I find that useful because I value reasonable discussion.
What I’m wondering is: is there a general rule underlying this about the follies of allowing causally-indistinguishable-in-retrospect effects to differently affect our anticipation? Can somebody formalize this?
I’ve seen similar stuff with people doing the Monty Hall problem. In some cases, not everyone agrees on the answer—like the Sleeping Beauty problem.
Other thoughts:
I think Cromwell’s rule (0 and 1 are not probabilities) is relevant here. While it may be unlikely that if two copies of you fought, that both would die, they are ‘evenly matched’ and if we do this too many times—serially - (with the winner of a 1v1 round going to the next box to do it again), there is nonzero chance no one walks out on the other side. On the other hand, if 1023 identical copies are made of you, and then everyone fights it out, it still sounds like a riskier procedure than keeping a ‘just in case’ copy, and if you must, having it fight the victor. (Or the backup could be the original, if you think that sort of thing’s important. Just because someone can make something that seems like an exact copy of you doesn’t mean that it is. This ‘duplicate and fight yourself for a prize’ sounds like a great way to alter say, the genetic makeup of the world’s population, without people noticing, because they never experience the procedure—just appearing somewhere they’d never been.)
That being said, I’ve always wondered why people always want their duplicates dead. It’s not exactly immortality, but it might improve your odds… and since I was already wondering what you do with all the dead bodies from the Duplicate Games, preserving them sounds like an idea (though they did get murdered by themselves, which might be traumatic), although the resources involved might make that prohibitively expensive. (Would you play these games with the knowledge that the people running the game would keep the bodies? Use them as fertilizer? Reuse the atoms and molecules because humans are made out of similar compounds? I’m slightly wary of someone else getting even a damaged exact copy of my brain, let alone 1023.)
I’m not sure about that actually, it seems implementation dependent-it’s certainly outside the current framework.
What you said would be true if RandomBot is truly random. In reality, RandomBot would probably be written using a PRNG, which is deterministic. In this game, both bots are given their opponent’s source code as an input. Reasonably, that should include the PRNG, and its seed. Consequently, RandomBot would probably be treated in each round as a CooperateBot or DefectBot-whichever it is at the time. (This might break down the conclusion-the bots don’t actually reason about long term consequences, that is all outside the code. They only deal with the program they are against now, and whether ‘today’ (in the current round) they will cooperate, etc.). It might mess with the proofs-PrudentBot could find that RandomBot is either CooperateBot or DefectBot this round, but if it proving that RandomBot will cooperate with it, is the same as proving that RandomBot’s first call the PRNG gets back an even number, then this proof is about something that (potentially) already happened. Then, proving ‘how it will treat CooperateBot’, could end up being a proof about what it would do if CooperateBot was its opponent next round. This is however, a hypothetical implementation nitpick, about how the programs might not perform as intended outside their original context, in which they faithful implement an idea, in a way that might not generalize without additional work.
Introducing things this way would change the dynamics-a bot which changes how it behaves based on a variable that’s different every round can have a more complicated strategy, which takes advantage of the costs of modeling, or (by using a counter) can try to grow to dominance in the population, then change strategies.
One might open up a similar wealth of possibilities by allowing the bots to know the current population, or the lineup of rounds (the tournament’s bracket), or work out proofs that handle mixed strategies (there is a 99% chance my opponent will cooperate, and a 1% chance it’ll do something random because its a ML algorithm which does random things epsilon=1% of the time in order to learn, etc.).
It could be more extreme than this result implies-I’m not sure how realistic the modeling costs are.
When A is looking for a proof about B(A), and B looking for a proof about A(B), it seems they should either
1) have the same cost which is proportional to the sum of their complexity as measured by length, since A+B=B+A or
2) have the cost for each time a program models the other program be dependent on the other program’s length-proving CooperateBot cooperates with you should be easier if only because Cooperate Bot is a shorter program, which isn’t trying to prove things about you, which means the fact it has your source code doesn’t matter—if it did, this would effectively increases its length. I think FairBot cooperating with itself requires a bit more of a search than CooperateBot doing so, and should have an appropriate cost.
FairBot shouldn’t just be penalized epsilon for modeling CooperateBot, it should be penalized based on the length of Cooperate Bot.
CooperateBot is defined by CB(X)↔⊤.
DefectBot is defined by DB(X)↔⊥.
FairBot is defined by FB(X)↔□(X(FB)).
PrudentBot is defined by PB(X)↔□(X(PB)∧(X(DB)→□⊥)).
Their definitions here also reflect that the modeling bots are longer, especially PrudentBot.
TL;DR
The costs could just be (a function of) the length of the proof*, or the amount of time it took to find and check.
*Still reasonable with caching, although costs would be much smaller.
I was wondering if that was intentional when I came here-here the post doesn’t seem to mention it, so I wasn’t sure if it was a bug just happening to me, thanks for mentioning it!
I’m doing the same thing.
What makes statements you control important?
“You assign probability less than 1⁄2 to this sentence,” but perhaps you wish you didn’t.
Why would you wish to assign a different probability to this statement?
I’m curious how a program could take another program and figure out its purpose is to detect cats.
So her tests weren’t “powerful” enough to “prove” her hypothesis.
Someone takes a class. They take a test, which they have to pass to pass the class. They pass the test and the class.
How do we know they couldn’t have passed the test before taking the class?
Alternatively, from xkcd (from memory):
Person 1: I used to think correlation implied causation.
Person 1: Then I took a stats class.
Person 1: Now I don’t.
Person 2: So it helped.
Person 1: Maybe.
Hover text: correlation doesn’t point to causation, but it inclines its head and raises its eyebrows.
Why would the simplest program that describes all your observations so far be a specification of the entire universe along with your location?
Reputation can be modeled as the idea that you have another ‘resource’ - that something like the number of promises you’ve made and the number you’ve kept (and their importance) is public knowledge.
When reputation is incorporated, agents compare the value of the $100 against the value of the reputation lost by doing so.
The problem could be easily changed to include ‘and you will lose no reputation, because no one thinks the driver is asking for a reasonable amount’ or ‘and you have p chance of losing r reputation worth u dollars’.
Is this question related to “How could we fully explain the difference between red and green to a colorblind person?”
(found at: https://www.lesswrong.com/posts/3wYjyQ839MDsZ6E3L/seeing-red-dissolving-mary-s-room-and-qualia)
How long is it, and what’s the new feature?
How did the tin ceiling tiles work?
I noticed skill succession without power succession wasn’t discussed—though the first thing that came to my mind was someone being trained etc. to takeover, and leaving instead.
Technical point: Cherry picking is choosing examples to suit one’s story. Example-less evaluation is conjecture/theorizing because there are no cherries present, only characterizations.
The principle of explosion isn’t a problem for all logics.
I think in a way, the problem with Parfit’s Hitchhiker is—how would you know that something is a perfect predictor? In order to have a probability p of making every prediction right, over n predictions, only requires a predictor be right in their predictions with probability x, where x^n>=p. So they have a better than 50% chance of making 100 consecutive predictions right if they’re right 99.31% of the time. By this metric, to be sure the chance they’re wrong is less than 1 in 10,000 (i.e. they’re right 99.99% of the time or more) you’d have to see them make 6,932 correct predictions. (This assumes that all these predictions are independent, unrelated events, in addition to a few other counterfactual requirements that are probably satisfied if this is your first time in such a situation.)
1) Why would it have a goal of getting out
2) such that it would halt if it couldn’t?
3) Conversely, if it only halts iff the goal is unreachable (which we assume it figures out, in the absence of a timeline), then if it doesn’t halt the goal is reachable (or it believes so).
To suppose that a being halts if it cannot perform its function requires a number of assumptions about the nature of its mind.
I was suggesting that what model you should use if your current one is incorrect is based on how you got your current model, which is why it sounds like ‘I prefer real world problems’ - model generation details do seem necessarily specific. (My angle was that in life, few things are impossible, many things are improbable—like getting out of the desert and not paying.) I probably should have stated that, and that only, instead of the math.
by representing the input of the problem explicitly I’ve created an abstraction that is closer to the real world than most of these problems are.
Indeed. I found your post well thought out, and formal, though I do not yet fully understand the jargon.
Where/how did you learn decision theory?
There seem to be some edge cases.: for instance, would an alternative Mary know everything about heart attacks without having one herself? Well, she would know everything except what a heart attack feels like, and what it feels like is a quale.
On the other hand, would someone who has a heart attack know as much as Mary about what’s going on?
Would Mary learn any new information over the course of a heart attack?
You seem to think LWers/rationalists are people who live in their heads, and are missing out on the benefits of “formative processes” that are (also) focused on the body.
Does practicing martial arts improve cognitive reflection or “Trying harder”? Or do the results vary too much to say?