You have italics :-)
There’s a “help” link right below the comments box to the right—LW uses Markdown, apparently.
Benya
Karma: 1,864
When I go to my userpage and click on the title of this post (above one of my comments on this post), it links to
http://lesswrong.com/a/5/issues_bugs_and_requested_features/
But that page 404′s...
If agreement votes aren’t going to be used, why not do away with them altogether and just use the current system to vote based on quality only?
I like Jess’s proposal because I think it has a better chance of working in practice. Most of us, I think, do want to express agreement / disagreement, and I think separating it out into a separate vote would work better with real humans’ cognitive systems than relying on people following an explicit instruction to ignore one of their motivations. [Yes, I would like to see a study testing this assumption somehow, but in the meantime, that’s the prediction my subjective probability is going into...]
Besides, I would find the agree/disagree info interesting. And I think it probably reduces “me too” posts. And the info presumably could be used for the “most controversial” page.
(edited: s/separate out their motivations/ignore one of their motivations/)
Shut up and do the impossible, and dependencies.
The concept of fighting a rearguard action against the truth.
It doesn’t track the bugs/requests in this thread, at this point, but here’s the official issues list (as linked from About Less Wrong).
Edit: Many points from this thread have made it there now (thanks to wmoore).
Because if the person who modded you up had written a “me too” post instead and the three people who modded gspence down had all written “me not” posts, we would have four essentially content-free posts clobbering up the thread.
Yes, maybe you can make a point that people should either make a new point or not speak at all, because just stating an opinion may be likely to be biased. But (a) I don’t think it’s going to work—saying that what happened to gspence’s comment isn’t what should happen doesn’t change the fact that it did happen with the current model; and (b), well, I’d like to state my “me too”/”me not”! :-) Yes, if not stating opinions really does significantly debias, that would outweigh that concern, but I’m pretty skeptical about that actually happening, so the expected utility from the agree/disagree buttons wins out for me.
“You, yes, you can reinvest the proceeds of your earlier investments!” You may not beat the market like Warren Buffett. But if you think about a whole civilization practicing that rule, we do better nowadays than historical societies with no banks or stock markets.
By the way -- (only very tangentially related to the topic of this post, I’ll admit, but hopefully just enough to pass muster) -- does anybody happen to know a good introduction to the theory behind this statement? Are there any introductory textbooks (microecon? macroecon?) that make the case, for example? Think of me as a very naive reader, who would be tempted to ask “clearly it is people doing actual work who create wealth, how does everybody reinvesting get people to do more / more useful work” and would like to hear the actual answer that convinced you in the first place.
Thanks! :-)
No, no, I do do sarcasm, but that wasn’t a specimen. :) Thanks! I’ve put it on my reading list.
I believe I agree with the intuition. Does it say anything about a problem like the above, though? Does the villain decide not to poison the hero, because the hero would not open the box even if the villain decided to poison the hero? Or does the hero decide to open the box, because the villain would poison the hero even if the hero decided not to open the box? Is there a symmetry-breaker here? -- Do we get a mixed strategy à la the Nash equilibrium for Rock-Paper-Scissors, where each player makes each choice with 50% probability?
(I’m assuming we’re assuming the preference orderings are: The hero prefers no poison to opening the box to dying; the villain prefers the box opened to no poison to the hero dying [because the latter would be a waste of perfectly good poison].)
I’m not sure why I’m getting downmodded into oblivion here. I’ll go out on a limb and assume that I was being incomprehensible, even though I’ll be digging myself in deeper if that wasn’t the reason...
In classical game theory (subgame-perfect equilibrium), if you eat my chocolate, it is not rational for me to tweak your nose in retaliation at cost to myself. But if I can first commit myself to tweaking your nose if you eat my chocolate, it is no longer rational for you to eat it. But, if you can even earlier commit to definitely eating my chocolate even if I commit to then tweaking your nose, it is (still in classical game theory) no longer rational for me to commit to tweaking your nose! The early committer gets the good stuff.
Eliezer’s arguments have convinced me that a better decision theory would work like Vladimir says, acting as if you had made a commitment in all situations where you would like to make a commitment. But as far as I can see, both the nose-tweaker and the chocolate-eater can do that—speaking in intuitive human terms, it comes down to who is more stubborn. So what does happen? Is there a symmetry breaker? Can it happen that you commit to eating my chocolate, I commit to tweaking your nose, and we end up in the worst possible world for both of us? (Well, I’m pretty confident that that’s not what Eliezer’s theory (not shown) would do.)
Borrowing from classical game theory, perhaps we say that one of the two commitment scenarios happens, but we can’t say which (1. you eat my chocolate and I don’t tweak your nose; 2. you don’t eat my chocolate, which is a good thing because I would tweak your nose if you did). In the simple commitment game we’re considering here, this amounts to considering all Nash equilibria instead of only subgame perfect equilibria (Nash = “no player can do better by changing their strategy”—but I’m allowed to counterfactually tweak your nose at cost to myself if we don’t actually reach that part of the game tree at equilibrium). But of course, if you accept Eliezer’s arguments, Nash equilibrium is wrong in general, and in any case, it’s not obvious to me if “either of the two scenarios can happen” is the right solution to this game.
To make the implicit motivation behind these two comments explicit: I’m worried that there’s a danger of writing “the rightful owner will keep their chocolate” on the bottom line, noticing that a proper decision theory would allow them to retaliate, and saying “done!” without even considering whether the same logic allows the nefarious villain to spitefully commit to eating the chocolate anyhow. If the theory says that either of the two commitment outcomes may happen, ok, but I think it deserves mention. And if the theory says is something else, I want to know that too. :-)
This seems like an empirical proposition. Does anybody here know what cryonics believers say who’ve seen friends or loved ones frozen?
Unless I’m misunderstanding something, this is true for the Brier score, too: http://en.wikipedia.org/wiki/Scoring_rule#Proper_score_functions
I’m not following your calculations exactly, so please correct me if I’m misunderstanding, but it seems that you are assuming that the student chooses an option and a confidence for that option? My understanding was that the student chooses a probability distribution over all options and is scored on that. As for how to extend the Brier score to more than two options, I’m not sure whether there’s a standard way to do that, but one could always limit oneself to true/false questions… (in the log case you simply score log q_i, where q_i is the probability the student put on the correct answer, of course)
Ok, so you’re saying the total score the student gets is
f1(q_i*) + Sum_(i /= i*) f0(q_i)? I didn’t understand that from your original post, sorry.So does “(if) he score for a wrong answer was 0 (...) the only proper score function is the log” mean that if there are more than two options, log is the only proper score function that depends only on the probability assigned to the correct outcome, not on the way the rest of the probability mass is distributed among the other options? Or am I still misunderstanding?
Would this be a sufficient formalization of your idea (restricting to two players for simplicity)?
Given a finite two-player game G, construct the game G_M (for ‘merge’) as follows: each player picks both a strategy profile of G (i.e., a function assigning a strategy to each player), and a “fallback” strategy. If both players select the same strategy profile, they get the corresponding payoff from G. If they select different strategy profiles, they get the payoff according to their fallback strategies.
Contrast with the formalization of source-code swapping, for two players:
Given a finite two-player game G, construct the game G_S (for ‘source swapping’) as follows: each player chooses a program that takes the other players’ programs as input, that outputs the strategy to play in game G, and that terminates for all inputs. Players’ payoffs are determined by the strategies their programs choose.
The first is arguably simpler to reason about, but it seems that any contract the players may want to enter into would be a Nash equilibrium of G_M, and for every Nash equilibrium of G_M there is a Nash equilibrium of G_S that has the same payoffs:
Suppose s is a strategy profile of G and ((s,f1),(s,f2)) is a Nash equilibrium of G_M. Then, consider the programs P1 = “if the other player plays P2, play s_1; otherwise, play f1” and P2 = “if the other player plays P1, play s_2; otherwise, play f2,” constructed by mutual Quining. It’s easy to see that (P1,P2) is a Nash equilibrium with the same payoffs.
Suppose ((s,f1),(t,f2)) is a Nash equilibrium of G_M, with s != t. Then, neither party can do better by switching to a different fallback strategy. Let C_x be the constant program that outputs x regardless of its input. Again, it’s easy to see that (C_f1,C_f2) is a Nash equilibrium with the same payoffs.
I think that the converse is also true (for every Nash equilibrium of G_S, there is an equilibrium of G_M with the same payoffs), but I haven’t been able to prove it; the problem is to find appropriate fallback strategies in G, and the idea is to show that if there aren’t any, then to be a Nash equilibrium, P1 and P2 must act differently depending on what the program they’re passed does, and to do that in general, they would have to solve the halting problem. But so far, I haven’t been able to prove this.
I’d be happy to expand the “easy to see”s if desired :-)
This also tells us which outcomes of G are implementable as Nash equilibria in G_M and G_S: if an outcome gives each player no less than their security value, it’s implementable, and vice versa.
Hm, nice formulation, but no, not quite, I think. Security value is the best you can get if you move first, right? I think an outcome is Nash implementable in G_M iff it gives each player no less than they can get if they move second—the fallback strategy of the other player is fixed, the deviating player can take it as the “first move.”
As for G_S, I think any outcome can be Nash implemented if it gives each player no less than if they move second (simply use the translation from G_M), and any outcome that can be Nash implemented in G_S must give each player no less than if they move first (because otherwise they could deviate by using a constant function). But I’m not sure about the outcomes that fall between these two conditions.
But at least, this characterization makes clear that most outcomes we’re usually interested in can indeed be implemented in both G_M and G_S. Thanks, I was actually looking for a succinct way to state this condition for G_S earlier :-)
Hm. I see. Thanks for pointing that out. Embarrassingly, I completely ignored mixed strategies above—implicitly assumed that the constructions of G_M and G_S would be over pure strategies of G, and analyzed only pure strategy profiles of G_M and G_S.
I do see how constructing G_M and G_S over mixed strategies of G would make the two values equal by the minimax theorem, but I think there are complications when we analyze mixed strategies. Mixed Nash equilibria of G_S can enforce correlated play in G, even without relying on cryptography, as follows: Have the pure strategies in the equilibrium correspond to a common quined program (as before) plus a natural number < n, for some given n. Then the program adds the different players’ numbers modulo n, and uses the result to determine the strategy profile in G. If a player chooses their number with uniform probability through a mixed strategy of G_S, then they know that all sums modulo n are equally likely, no matter how the other players choose their numbers. So everybody choosing their numbers uniformly at random can be a Nash equilibrium, because no player can change the expected result by unilaterally switching to a different way of choosing their number.
But G_M, as defined above (whether over pure or mixed strategies), can, as far as I can see, not enforce correlated play in G.
A natural way to rectify this is to define the first component of G_M to be neither just a pure nor just a mixed strategy profile of G, but an arbitrary distribution over pure strategy profiles of G—a “correlated strategy profile.” Can the earlier proof then be used to show that G_M has a mixed strategy profile realizing a certain outcome ⇔ G has “correlated strategy profile” realizing that outcome that pays each player at least their security value ⇔ G_S has a mixed strategy profile realizing that outcome?
Self-modification is the correct justification for updateless decision theory
If a UDT agent is presented with a counterfactual mugging based on uncertainty of a logical proposition, it should attempt to resolve the logical uncertainty and act accordingly.
Ok, the intuition pump is problematic in that not only do you know what the first digit of pi is, it is also easy for the AI to calculate. Can you imagine a least convenient possible world in which there is a logical fact for Omega to use that you know the answer to, but that is not trivial for the AI to calculate? Would you agree that it makes sense to enter it into the AI’s prior?
My point was that since you’re consciously creating the AI, you know that Omega didn’t destroy Earth, so you know that the umpteenth digit of pi is odd, and you should program that into the AI. (A’ight, perhaps the digit is in fact even and you’re conscious only because you’re a Boltzmann brain who’s about to be destroyed, but let’s assume that case away.)
It would be nice to have jsMath installed (a Javascript renderer for TeX math—you just drop it in your page and it shows TeX math prettily). Yeah, you can read and write math in pure HTML, but… :-)