It amuses me that ElectionBettingOdds.com is still operational and still owned by FTX Trading Ltd.
robo
Karma: 669
I think it is similar to option 4 or 6?
Yep! With the addendum that I’m also limiting the utility function by the same sorts of bounds. Eliezer in Pascal’s Muggle (as I interpret him, though I’m putting words in his mouth) was willing to bound agents subjective probabilities, but was not willing to bound agents utility functions.
Why is seconds the relevant unit of measure here?
The real unit is “how many bits of evidence you have seen/computed in your life”. The number of seconds you’ve lived is just something proportional to that—the Big Omega notation fudges away proportionality constant.
TLDR Kelly bets are risk avoidant. I think Kelly bets prevent you from pouring all your money into a pascal-mugging change of winning ungodly sums of money, but Kelly bets will pay a mugger exorbitant blackmail to avoid a pascal-mugging chance losing even a realistic amount money
---------
Starting with a pedantic point. None of the Pascal Mugging situations we’ve talked about are true Kelly bets. The mugger is not offering to multiply your cash bet if you win. Your winnings are saved lives, and they cannot be converted into a payroll.But we can still translate a Pascal’s mugging into the language of a Kelly bet. A translation of the standard Pascal mugging might be: the mugger offers to googolplex-le your money[1], and you think he has a one in a trillion chance of telling the truth. A Kelly bet would say that despite these magnificent EV of the payouts, you should put only ≈a trillionth of your wealth into this bet. So in this case, the one like the original Pascal’s mugging, the one you responded to, the Kelly bet does the “right” thing and doesn’t pay the mugger.
But now suppose the Pascal mugger says “I am a jealous god. If you don’t show your belief in Me by paying Me $90,000 (90% of your wealth), I will send you and a googolplex other people to hell and take all (or all but a googolplexth) of your wealth”. And suppose you think that there’s a 1 in 1 trillion chance he’s telling the truth.
Can we translate this into a Kelly bet? Yes! (I think?) The Kelly criteria tells you how to allocate your portfolio among many assets. Normally we assume there’s a “safe” asset, a “null” asset, one where you are sure to get exactly you money back (the asset into which you put most of your portfolio when you make a small bet). But that asset is optional. We can model this Kelly bet by saying there are two assets into which you can allocate your portfolio. Asset A’s payoff is “return 10% (loses 90%) of the bet with certainty”. Asset B’s payoff is “with probability 999,999,999,999⁄1 trillion (almost 1), return your money even, but with chance 1 in 1 trillion, lose ≈everything”. There is no “safe cash” option—you must split your portfolio between assets A and B.
Here, Kelly criterion really, really hates losing ≈all your bankroll. It says to put almost everything into the safe asset A (pay the mugger), because even a 1 in 1 trillion chance of losing ≈everything isn’t worth it. Log of ≈0 (lost almost everything) is a very negative number.
Perhaps it would be useful to write exact math out.
- ^
Importantly, I think for the math to work out he has to be offering a payoff proportional to your bet, not a fixed payoff?
- ^
Suppose the mugger says that if you don’t give him $5, he’ll take away 99.999999999999999% of your wealth. I don’t think Kelly bets save you there? The logarithms of Kelly bets help you on the positive side but hurt you on the negative side.
But what about Pascal’s Muggle? If you want to cancel out 3↑↑↑3 by multiplying it with a comparably small probability, the probability has to be incredibly, incredibly, small; smaller than a Bayesian can update after 13 billion years of viewing evidence. So where did that small number come from? If the super-exponenctial smallness came from priors, then you can’t update away from it reasonably—you’re always going to believe the proposition is false, even if given an astronomical amount of evidence. Are you biting the bullet and saying that even if you find yourself in a universe where this sort of thing seems normal and like it will happen all the time, you will say a priori this this apparently normal stuff is impossible?
You should bound your utility function (not just probabilities) on how much information your brain can handle. Your utility function’s dynamic range should never outpace your brain’s probability’s dynamic range. Also you shouldn’t claim to put $Googolpex utility on anything until you’re at least [1] seconds old.
Utility functions come from your preferences over lotteries. Not every utility function corresponds to a reasonable preference over lotteries. You can claim “My utility function assigns a value of Chaitin’s constant to this outcome”, but that doesn’t mean you can build a finite agent that follows that utility function (it would be uncomputable). Similarly, you can claim “my agent follows a utility function assigns to outcomes A B and C values of $0, $1, and $googolplex”, but you can’t build such a beast with real physics (you’re implicitly claiming your agent can distinguish between probabilities so fine that no computer with memory made from all the matter in the eventually observable universe could compute it).
And (I claim) almost any probability you talk about should be bounded by O(2^(number of bits you’ve ever seen)). That’s because (I claim) almost all your beliefs are quasi-empirical, even most of the a priori ones. For example, Descartes considered the proposition “The only thing I can be certain of is that I can’t be certain of anything” before quasi-empirically rejecting that proposition in favor of “I think, therefore I am”. Descartes didn’t just know a priori that proposition was false—he had to spend some time computing to gather some (mental) evidence. It’s easy to quickly get probabilities exponentially small by collecting evidence, but you shouldn’t get them more than exponentially small.
You know the joke about the ultrafinitist mathematician who says he doesn’t believe in the set of all integers? A skeptic asks “is 1 an integer?” and the ultrafinitist says “yes”. The skeptic asks “is 2 an integer?”, the ultrafinitist wait’s a bit, then says “yes”. The skeptic asks “is 100 an integer?”, the ultrafinitist waits a bit, waits a bit more, then says “yes”. This continues, with the ultafinitist waiting more and more time before confirming the existence of bigger and bigger integers, so you can never catch him in a contradiction. I think you should do something like that for small probabilities.
Here’s a pure quantum, information theoretic, no computability assumptions version that might or might not be illustrative. I don’t actually know if the quantum computer I’m talking about could be built—I’m going off intuition. EDIT I think this is 2 party quantum computation and none of the methods I’ve found are quite as strong as what I list here (real methods require e.g. a number of entangled qbits on order of the size of the computation).
You have two quantum computers, Alice and Bob, preforming the same computation steps. Alice and Bob have entangled qbits. If you observe the qbits of either Alice or Bob in isolation, you’ll forever get provably random noise from both of them. But if you bring Alice and Bob together and line up their qbits and something something mumble, you get a pure state and can read off their joint computation.
Now we have all sorts of fun thought experiments. You run Alice and Bob, separating them very far from one another. Is Alice currently running a mind computation? Provably not, if someone looked at Bob last year. But Bob is many many light years away—how can we know if someone looked at Bob? What if we separate Alice and Bob past each other’s cosmic horizons, such that the acceleration of the expanding universe makes it impossible for them to ever reach each other again even if they run towards each other at the speed of light? Or send Bob to Alpha Centauri and back at close to the speed of light so he’s aged only 1 year where Alice has aged 8. Has Alice been doing the mind thing for the past 7 years? Depends on whether you look at Bob or not.
(but I’ll note that for me, this version, like the homomorphic version, is mostly saying that your description of a quantum physics state shouldn’t be purely local. A purely local description must discard information, something something mixed state Von Neumann entropy)
Forgive me, I’m probably being stupid again 😬.
On efficient computability being necessary for reality: I’m not sure I understand the logic behind this. Would you not always get diagonalization problems if you want supervening “real” things to be blessed with R-efficiently computability? For example, take R to be something like a Solomonoff induction. R-efficiently computable there means Turing computable. For our M which supervenes on R, instead of Minds, let’s let M be the probability p of a given state. The mapping function g: R->M, mapping states to the probability of states, cannot be R-efficiently computed (no matter what sort of Turing machine or speed prior you use for R) for diagonalization reasons. So the probabilities of states aren’t a “real” thing? It seems like a lot of natural emergent things wouldn’t be R-efficiently computable.
On homomorphic encryption being un-reversible: quantum computers are reversible, right? So if you say physics is as powerful as a quantum computer, and you want homomorphic encryption to be uncomputable in polynomial time, you have to make P’s physics “state” throw quantum information away over time (which it could, in e.g. Copenhagen or objective collapse interpretations, but does not in e.g. many worlds) or maybe restrict the size of the physical universe you’re giving as state to not include information we radiated away many years ago (less than 62.9 billion light years).(Don’t feel obligated to reply)
Forgive me, I only scanned.
You’re talking about exponentially unlikely physical states, like the kind where you disintegrate from location 1 andjust by chancean identical copy of you appears in location 2 for no reason, or the thermodynamic arrow of time runs backwards, or states that encode a mind you can’t decode without the right homomorphic key but then the homomorphic key appears in your alphabet soupjust by chance, or your whole life was an elaborate prank for a reality TV show and most of the universe is actually made of cheese, or there’s a giant superintelligent pink elephant in every room butjust by chancenobody notices them, or the Easter Bunny and Harry Potter both appear and their magic worksjust by chanceeach time they try to use it (in a way conforming to the standard model), or whatever. These states with ≈0 measure might be theoretically possible but personally I don’t put much stock in thought experiments about them?
EDIT still only scanned, but I think I misread the post. I (unconfidently) think the post is about if someone homomorphically encrypts a mind computation, then moves the information in the key past the cosmic event horizon of the expanding universe so the information in the key and the encrypted mind can never return together again. (Or are exponentially unlikely to). You can get an effect like this by e.g. burning the key and letting the infrared light of the fire escape to the blackness of the night sky.
I suspect this about many things, e.g. the advice in the US to never talk to the police.
With the Streisand effect I’m less sure. Conflict sells. The areas in e.g. popular science I know the most about tend not to be the ones that are most established or important—they tend to be the ones that are controversial (group selection, deworming wars, arsenic biology).
Not if the ELO algorithm isn’t run to completion. It takes a long time to make large gaps in ELO, like between stockfish and Random, if you don’t have a lot of intermediate players. It’s hard for ELO to different between +1000 ELO and +2000 ELO—both mean “wins virtually all the time”.
Aren’t ELO scores conserved? The sum of the ELO scores for a fixed population will be unchanged?
The video puts stockfish’s ELO at 2708.4, worse than some human grandmasters, which also suggests to me that he didn’t run the ELO algorithm to convergence and stockfish should be stealing more score from other weaker players.
EDIT ChatGPT 5 thinks the ELOs you suggested for random are reasonable for other reasons. I’m still skeptical but want to point that out.
I do not believe random’s Elo is as high as 477. That Elo was calculated from a population of chess engines where about a third of them were worse than random.
I’m not at all convinced this isn’t a base rate thing. Every year about 1 in 200-400 people have psychotic episodes for the first time. In AI-lab weighted demographics (more males in their 20′s) it’s even higher. And even more people get weird beliefs that don’t track with reality, like find religion or Q-Anon or other conspiracies, but generally continue to function normally in society.
Anecdotally (with tiny sample size), all the people I know who became unexpectedly psychotic in the last 10 years did so before chatbots. If they went unexpectedly psychotic a few years later, you can bet they would have had very weird AI chat logs.
Light disagree. Prefix modifiers are cognitively burdensome compared to postfix modifiers. Imagine reading:
”What I’m about to say is a bit of a rant. I’m about 30% confident it’s true. Disclosure, I have a personal stake in the second organization involved. I’m looking for good counter arguments. Based on a conversation with Paul. I have a formal writeup at this blog post. Part of the argument is unfair, I apologize. I...”Gaaa, just give me something concrete already! It’s going to be hard enough understanding your argument as it is; it’s even harder for me to understand your argument while having to keep unresolved modifiers loaded in my mental stack.
Ha, and I have been writing up a long-form for when AI-coded-GOFAI might become effective, one might even say unreasonably effective.
LLMs aren’t very good at learning in environments with very few data samples, such as “learning on the job” or interacting with the slow real world. But there often exist heuristics, ones that are difficult to run on a neural net, with excellent specificity that are capable of proving their predictive power with a small number of examples. You can try to learn the position of the planets by feeding 10,000 examples into a neural network, but you’re much better off with Newton’s laws coded into your ensemble. Data constrained environments (like, again, robots and learning on the job) are domains where the bitter lesson might not have bite.
Back in the GOFAI days, when AI meant A* search, I remember thinking:
Computers are wildly superhuman at explicit (System 2 reasoning) like doing arithmetic or searching through chess moves
Computers are garbage at (System 1 reasoning), like recognizing a picture of a cat
When computers get good at System 1, they will be wildly superhuman at everything
Now transformers appear to be good at System 1 reasoning, but computers aren’t better at humans at everything. Why?
I think it comes down to:Computers’ System 1 is still wildly sub-human at sample efficiency; they’re just billions of times faster than humans
LLM’s work because they can train on an inhuman amount of reading material. When trained on only human amounts of material, they suck.
LLM Agents aren’t very good because they can’t learn on the job. Even dumb humans learn better instincts after a little on-the-job practice. We can just barely improve LLM’s System 1 from its System 2, but only by brute forcing an inhuman number of roll-outs.
Robots suck, because the real world is slow and we don’t have good tricks to train their System 1 by brute force.
We’re in a weird paradigm where computers are billions of times faster than humans, but thousands of times worse at learning from a datum.
I think I disagree. It’s more informative to answer in terms of value as it would be measured today, not value after the economy adjusts.
Suppose someone from 1800 wants to figure out how big a deal mechanized farm equipment will be for humanity. They call up 2025 and ask “How big a portion of your economy is devoted to mechanized farm equipment, or farming enabled by mechanized equipment?” We give them a tiny number. They also ask about top-hats, and we also give them a tiny number. From these tiny numbers they conclude both mechanized farm equipment and top-hats won’t be important for humanity.
EDIT The sort of situation I’m worried about your definition missing is if remote-worker AGI becomes too cheap to meter, but human hands are still valuable.
Would you agree your take is rather contrarian?
* This is not a parliamentary system. The President doesn’t get booted from office when they lose majority support—they have to be impeached[1].
* Successful impeachment takes 67 Senate votes.
* 25 states (half of Senate seats) voted for Trump 3 elections in a row (2016, 2020, 2024).
* So to impeach Trump, you’d need the votes of Senators from at least 9 states where Trump won 3 elections in a row.
* Betting markets expect (70% chance) Republicans to keep their 50 seats majority in the November Election, not a crash in support.- ^
Or removed by the 25th amendment, which is strictly harder if the president protests (requires 2⁄3 vote to remove in both House and Senate).
- ^
Make sure the people on the board of OpenAI were not catastrophically naive about corporate politics and public relations? Or, make sure they understand their naïveté well enough to get professional advice beforehand? I just reread the press release and can still barely believe how badly they effed it up.