I am Andrew Hyer, currently living in New Jersey and working in New York (in the finance industry).
aphyer
Karma: 3,782
Hi, I’m Andrew, a college undergrad in computer science. I found this site through HPMOR a few years ago.
Sorry if this is a stupid question, but this tournament looked to me like a thinly disguised version of:
“Construct a robot that can read code and interpret what it means.”
which is a Really Hard Problem.
Is that not a fair description? Was there some other way to approach the problem?
The only way I can see to go about constructing a GOOD entrant to this is to write something that can take as its input the code of the opponent and interpret what it will actually DO, that can recognize the equivalence between (say):
return DEFECT
and
if 1: return DEFECT return COOPERATE
and can interpret things like:
if opp_code == my_code return COOPERATE return DEFECT
And I have no idea how to go about doing that. From the fact that the winning entrants were all random, it seems safe to say that no entrants had any idea how to go about doing that either.
Am I missing something here?
Not sure I agree with that. Emphasis on “think” undercuts the point: I wouldn’t say that I “think you can’t jump over the moon”, even though I do not have a formal proof of impossibility handy for that, I’d just say “you can’t do that.”
In fact, I almost like it better without the word “think” at all: “Whatever can’t be done, someone will come along and do it.” YMMV, though.
One question regarding the specifics of the ‘write stuff down to feel better about it’ that doesn’t seem to be covered:
After (to take the example from the article) being laid off, I’ll probably have a rather negative view of the event. I will feel bad about it, and I will want to blame others. When I write my “feelings regarding the layoff”, if they wind up as an angry rant about “my stupid worthless boss” and “my goddamn no-good backstabbing coworkers”, does that still work? Or do I need to be more even-handed about this, identify things I might have done wrong and reasons I might have deserved it, admit that it was mostly my fault, etc...?
I don’t have access to the books you cite as sources, so it’s possible that this is covered specifically in them. But if you can actually boost “happiness, self-esteem, health, and psychological and physical well-being” by writing down an angry rant, I would find that rather surprising.
This. This this this. I feel like the “fire in a crowded theater” example is a pretty painfully outdated one. If you imagine a more crowded theater made of dry wood with no sprinklers/fire extinguishers, it becomes a lot more reasonable to expect people to panic at the thought of a fire.
Do you have any comments on the content of the article beyond this? The article makes a couple claims:
The existence of Lone Genius Bias—do you think it exists?
The relevance of Lone Genius Bias—do you think that, given Lone Genius Bias, you might be underestimating the odds of the US government developing AI and overestimating the odds of some nerd in a basement developing AI?
The source of Lone Genius Bias—do you think Lone Genius Bias comes from the small group size in the ancestral environment?
Your comment makes it clear that you disagree with the third of these. And that’s a pretty fair response: post hoc evo-psych is dangerous, prone to bias, and usually wrong. But your comment ALSO seems to say that you think this entitles you to completely disregard the first two points of the article: the author’s arguments for the existence of Lone Genius Bias and his arguments that it’s misleading us as to where AI is likely to come from.
If you agree with the rest of the article, but dislike the evo-psych part, you can probably find a more polite way to phrase that. If you disagree with the rest of the article as well, you should be counter-arguing the rest of the article on its own merits, rather than zeroing out the one weakest point and disregarding every other point the author made.
This is an old article, and it’s possible that this question has already been asked, but I’ve been looking through the comments and I can’t find it anywhere. So, here it is:
Why does it matter? If many-worlds is indistinguishable from the Copenhagen Interpretation by any experiment we can think of to do, how does it matter which model we use? If we ever find ourselves in a scenario where it actually does matter which one we use—one where using the wrong model will result in us making some kind of mistake—then we now have an experiment we can do to determine which model is correct. If we never find ourselves in such a position, it doesn’t matter which model we decided on.
When phrased this way, Science doesn’t seem to have such a serious problem. Saying “Traditional Science can lead to incorrect conclusions, but only about things that have no actual effect on the world” doesn’t sound like such a searing criticism.
I forget where I saw this (might actually have been elsewhere on LW?), but I encountered the idea that a component of the backward-reverent ages was the Roman Empire. When your civilization is built on the remnants of Roman roads that are better than anything you can make, it’s forgivable to view the world as having fallen from grace.
The humans aren’t doing what the math says. The humans must be broken.
Something that bothers me about this tournament: I feel like a competitive tournament doesn’t actually reward the kind of strategy that is meant to do well in Prisoner’s Dilemna. As a (highly oversimplified) example, consider three bots who have the scores:
A: 10 B: 9 C: 2
Here, A is ‘winning.’ Suppose B can make a move that costs A 3 points and costs itself 1 point, leading to:
A: 7 B: 8 C: 2
B’s payoff function has dropped. However, from a ‘winning the tournament’ approach, B has gone from 2nd to 1st, and so this outcome is now better for B. This feels wrong.
I doubt this was a really big issue here, but just on general principles I feel like competition by comparing scores is incompatible with a desire to explore the Prisoner’s Dilemma, since you’re turning a non-zero-sum game into a zero-sum game.
Besides the obvious benefit of being awesome, I think there could be a more serious benefit to this. One extreme failure mode when imagining the behavior of an AI is not merely to fail to imagine it as being superintelligent but to imagine it as being less intelligent than yourself, as not doing things you could think of (a la That Alien Message). A game that consisted of you, the player, needing to come up with increasingly complicated ways to trick these ‘shopkeeper’ agents could illustrate this pretty neatly.
The +10 is because the 10% fee does not apply to your original $100, only to profits. So if you would have $250 without the fee, rather than 250 * 0.9 = $225, you end up with $235.
It’s possible! I’m writing this from a very US-based perspective, where the main prediction markets I hear talked about are PredictIt and IEM (and I think IEM might be even worse for betting at scale?) If you have access to a prediction market where these problems are smaller/nonexistent, it will be more accurate. And 2% charge of profits is much lower, plus looking quickly at the Betfair website it seems to have a pretty good amount of liquidity at a reasonably small spread.
I’m not sure how accessible Betfair is to someone from the US from a legal/tax standpoint, so the overall point may still stand if you’re talking to someone in the US/if you think that a lot of the people informed on a given issue are in the US.
(And do you really have zero taxes on gambling profits? I would not have guessed that).
Fair enough, that part is worded more strongly than I can really justify. I’d definitely still say it’s quite possible that they are correct, and I still wouldn’t think you can easily show them to be wrong by EMH-style reasoning.
Ah, someone with the time and energy to proofread and LaTeX things for me! No objection, thank you very much! (I actually didn’t know other people were able to edit my posts though, how does that work?)
Sure, happy to change it. Does ‘Limits of current US Prediction Markets’ plus some commentary at the top to the effect of ‘this is based on a case study of PredictIt, per some comments if you are outside the US you may have better options’ sound reasonable?
Two replies here, I think:
First, are any of the things you suggest in fact legal for US-based bettors?
Second, how difficult in terms of time/inconvenience/up-front capital costs without the potential for leverage are they compared to the ways in which you can bet on e.g. the price of Google stock?
If you’re telling someone in the US “you should be willing to bet your views on prediction markets, if you’re right it’s free money” when what you mean is “it’s free money provided you’re willing to dedicate hours to working your money through crypto exchanges, committing tax fraud along the way”, that feels somewhat different.
I would be willing to tell someone who thinks Google shares are underpriced “go buy some then,” because buying Google stock at low cost is legal, very easy, and even if you have limited up-front cash the real financial markets offer opportunities for leverage (options etc.)
I would not be willing to tell someone who thinks Trump Wins shares are underpriced “go buy some then” when that process is time-consuming, difficult, and likely illegal.
Should be fixed now, thanks for the heads-up!
Adding some further comments on the performance of prediction markets in the 2020 election here—while the election is still ongoing, I think so far it seems to support the article. Some numbers here are based on my fallible and sleep-deprived human memory, if anyone disagrees/has a better source I can update this.
Coming into the election, 538 had Trump at a 10% chance to win (saying words to the effect of ‘Biden is in a very strong position, where he has several paths to victory even through a correlated polling error of the sort that happened to Clinton’). PredictIt and Betfair had Trump at 30-40%.
As the day went on, Trump did better than expected in several places (particularly Florida), and took a lead in several important states (PA, MI, WI), though with many mail-in and urban ballots remaining that were expected to be Democrat-favored. At 11pm, 538 wasn’t giving hard numbers but a couple places in their chat were talking about Trump being anywhere from a 20-50% chance to win and emphasizing that mail-in ballots were expected to shrink or remove his leads, while PredictIt and Betfair both had Trump at ~70% to win.
As of this writing on the afternoon of the next day, with no new surprises I am aware of except for the entirely as-predicted shrinking of Trump leads due to later-arriving ballots, 538 continues to say ‘Trump is behind but could still manage an upset’, while Betfair and PredictIt have swung from their previous ~70% to a ~20% that is in line with what 538 was saying last night.
While there are several interpretations you could place on these numbers, it rather looks to me like prediction markets overrated Trump from the beginning, heavily overreacted to Trump’s early outperformance, failed to predict the effect of late-arriving ballots until it actually happened, and then swung around and dropped their probabilities heavily based on this predictable news, while 538 did a lot better.
538 forecast: 10% → Trump gets positive news → 20-50% → predictable shrinkage in Trump lead → ~20%
Prediction market forecast: ~40% → Trump gets positive news → ~70% → predictable shrinkage in Trump lead → ~20%If we were dealing with an efficient market I wouldn’t expect this to happen. Then again, if we were dealing with an efficient market I would expect hedge funds to hire skilled analysts on 6-figure salaries to stay up all night tracking election results and arbitraging market swings like that. Until that happens, I will keep an eye on prediction markets, but I will trust 538 over them.
I realize that this comment has been up for a long time, but just in defense of Buzz Aldrin: the punch was less in response to the man claiming that he was wrong, and more in response to the man being verbally abusive (don’t believe everything you hear, search on Google for Buzz Aldrin Punch and you can get a video for yourselves.) There’s a difference between violence being the appropriate response to reasoned argument and violence being the appropriate response to abuse/someone else’s violence/etc.