Year 4 Computer Science student
find me anywhere in linktr.ee/papetoast
papetoast
Karma: 686
Grok 4 was SOTA in ARC-AGI-2 though, so I don’t feel like it is that surprising
I am curious about prior discussions too for how people explain it. I don’t know how to explain it because it seems so self-evident. Being able to reason from a sort of “third-person perspective” reasoning about yourself doesn’t just make you lose your reward system. You can use a sandbox simulation to think outside of the reward mechanism, but you are always inside the reward mechanism. The motivation to keep the reward system on is the reward system itself.
Either a you problem or a chatgpt problem, I can open it in incognito
Add it to your bio here so more people can see it!
I thought you can’t keep the name short while hinting at the mechanism, but ChatGPT quickly proved me wrong
Good job to chatgpt actually, I just dumped your post to it lol. Also you can just ask LLM to typeset the math but keep other content verbatim and im certain it will one shot it
But GPT 5.4 with August 2025 cutoff can now answer it with search, though still fails without search (both are first attempt).
That’s really cool, is there a wiki page for this formula? [Edit: Mechanical similarity but it’s a stub]
btw: LW supports latex
Yeah, I’m thinking about stuff like “do I sell my house and my two dogs to make this bet?”
Ok I understand what you’re saying now. My reaction is that we should just add the expected current value of all the money you will make in the future (maybe discounted and also conditional on you making the bet), to your current wealth and then kelly as if you have that much money. This seems like a valid critique of how people Kelly bet currently, but I still disagree the correct response to this hypothetical is “we should consider our utility functions as being way less curved than a logarithm”. I think people do genuinely value wealth roughly logarithmically so if you don’t make any money in the future then Kelly is correct.
I do understand “if you have an anual salary of 100k you shoudln’t trust kelly anywhere below like 500k” now and I agree.
Not making a real money bet because it seems difficult to operationalize / flush out the details enough that I would think I have enough edge. People imo will correctly give a more conservative number if they think the question is realistic, and they will give closer to 50% if they think it is an idealized scenario where all they have is money. But I will say 30% of people would give >=50% if they understand the scenario as mathematical.
Also, I was saying something weaker. I disagreed on “This strikes most people as being insanely agressive”. I am saying that people would, after being told the correct answer (i.e. in retrospect), think/tell you that the mathematically correct answer is not insanely aggressive. Even if 50% is higher than what they said would personally bet, I think most people would not say and would disagree that it is insanely aggressive.
This strikes most people as being insanely agressive
I don’t think so. ~Everyone can see that this is super obviously a good deal.
but this is paradoxical because the assumptions underpinning the analysis are actually wildly conservative
The bankrupt is infinitely bad assumption is implicit in the log-wealth model and definitely get some people confused when they first encountered it. It is definitely a wildly conservative assumption that is wrong, I don’t trust Kelly at like $0-$100. But it feels to me that this pitfall is not as big of an issue as you’re saying, because in most realistic situations 1. the utility of having low digit money won’t affect the Kelly calculation much for mediocre deals (intuition; didn’t math it out), 2. you want to discount on Kelly (or any modified-Kelly) anyways because of uncertainty in the probability/returns of the bets you’re making.
Sure, I just don’t think this is a big issue because people bottlenecked by motivation probably won’t be serious contributors, and it’s worth improving things for those that will.
Yeah. I am just thinking, if for example only 1% of the time is wasted on bad onboarding, then it isn’t important to optimize onboarding. I don’t have good intuition or data on how bad onboarding is right now though.
I basically don’t read academic paper styled posts, so I am not your target audience, but this alone probably filtered a lot of readers regardless of the actual writing. It is pretty common for posts in alignment forum to have no comments too.
I think I am implicitly just saying that prediction markets are not perfectly efficient. I think that is normal and still have great utility, I don’t think efficient markets are something to be worried about because they will only exist mathematically. All these behaviors that only make sense when the market is not efficient makes sense exactly because the market is not efficient. But, there are different degrees of inefficiency. It is a scale, and I think prediction markets are pretty accurate already even if it is not mathematically efficient. Have you seen the brier score on different prediction markets? They are obviously not perfect, but it is pretty close. Close enough that it sometimes makes sense to act as if market probability = true probability.
Also the prediction markets are not negative sum for society as a whole, the fee doesn’t just evaporate. It simply goes to the platform. Taking money from dumb people in a zero-sum game is fine.
Subsidy can come in another form, a player can spend money trying to elicit information so that they can use the information somewhere else.
Again I am assuming an imperfect world. There are a lot of public information but you don’t want to spend time aggregating them, instead you create a market and put $10k in the automatic market maker to incentivize people to make their best guess.
I personally preferred to get up early while No Magic Pill and niplav preferred to stay up late and get up at the usual time.
Nit: this article’s first author is niplav and I have no idea who is “I” here.
Also your footnote got fucked by the wysiwyg → markdown conversion
Isn’t a natural conclusion then that in reality prediction markets are not completely efficient? There are subsidy, irrational traders/gamblers, and risk hedging (yes I know you mentioned it).
Subsidy can come in another form, a player can spend money trying to elicit information so that they can use the information somewhere else.
Also efficiency can be relative to a trader even if everyone only trades on public information because in reality no one has the time to discover and aggregate all the public information.This is just wrong
Quick thoughts. Note that I’m not that into AI safety problems and am speaking in general about LW.
I think this is the case because much of the discussion here is happening “on the cutting edge” so to speak; most users are actively exploring different lines of thought with new ideas built on complex world models, and are not necessarily revisiting or referencing the years of prior knowledge and belief that underpin their thinking.
I agree. This seems downstream of the poor search in LW and wikitags. Better search is good, but absent that we should tell people to google with
query site:lesswrong.com. The UI also could link posts to related posts automatically, this is somewhat hard but definitely doable.I’m interested in seeing how this process could be made more efficient; I think a more streamlined “onboarding” process for newcomers could save a lot of valuable time and help people become more impactful quicker.
(This is not a confident statement I’m thinking out loud) I’m not sure if a significant portion of the time from newcomers to contributors is spent on things that would benefit from better onboarding, like even with perfect onboarding someone would still need to motivate themselves to read quite a large chunk of text to reach the pareto frontier.
Encourage and facilitate creation of “current beliefs” pages for each user, where they can detail their present-day positions on certain subjects (AI timelines are a relatively simple example), recount their intellectual journey / past updates, reference particularly impactful posts and threads, etc.
This seems like it will just create a bunch of stale pages that are also hard to find what you want because each person’s belief page will contain too many different topics. Also, although I’m sure with official site-level support more people will do this, the fact that I basically never seen someone do this makes me think that people just don’t like to spend so much time organizing their intellectual journey such that it is clear enough to other people; it seems genuinely time-consuming to put in this effort.
Other organizational tools: content tags are nice, but they are a little too broad to be very helpful IMO. Maybe a more sophisticated tagging scheme, or some way for users to individually tag/curate articles (and maybe comments) in a publicly visible way?
Similar to the “current beliefs” pages, I think the main bottleneck isn’t tooling, it is how much time people are spending to make things legible for other people/newcomers. I don’t think sophisticate tagging scheme is a good idea, I have never seen extremely fine grain tags work out. Also, with fine-grained tags the problem shifts to the discoverability of tags, how would you know you have found all the tags that are somewhat related? Also tags are time-consuming to maintain, the LW tagging AI works well currently but I think accuracy would fall with too many tags.
It would be nice to accurately situate individual posts within the larger discussion they are a part of and I don’t think the current “mentioned in” section does a good enough job at that.
Someone can just make a post organizing the past discussions! But again ~nobody does that.
https://danfrank.ca/daniel-isms-50-ideas-for-life-i-repeatedly-share/ (#20)
When you travel/move, enter into a new environment, you get to be the current version of yourself, and have everyone interact with that version of you. When you are at home or in an older environment, you have people who are simultaneously interacting with many different versions of you at once (ie people who knew the high school you, college you, early career you etc). This is challenging because it pressures you to also carry on multiple historical iterations of yourself, rather than being truly the present version of yourself you want to be in that moment.
You double posted. Also, what?