I like this vision! “LLM-native Manifold” seems like an obvious thing to explore, but I don’t know of anyone doing this. Some other thoughts:
Manifold-like systems seems like a nice way of running an tournament/evolutionary algorithm to find the best model & scaffold for forecasting. One of the historical arguments for prediction markets is that they help surface human talent (by allowing smart forecasters to have win money/influence in society), and you could view “conjuring the right scaffold” as the modern equivalent
There’s a few distinct intellectual tasks in organizing a prediction market, which could be supercharged with LLMs: forecasting/trading (which has been explored the most), question creation, and question resolution. I suspect LLM assistance on the latter two could help with making PMs actually useful with decisionmaking.
Within forecasting/trading, a lot of effort has gone into something like “make better brier/loss score” (eg see forecastbench), but I think a lot of the nuance in trading well involves being good at market selection and bet sizing, aka knowing where your edge is and how confident to be in it. I’d like to see more people trying to incorporate this into bots, and see bet sizing matter more in bots
LLMs seem like our best bet of getting futarchy to work; with cheap intelligence there are a lot more things we could try here
Has anyone really been far even as decided to use even go want to do Garrabrant induction? IDK what this would look like, just generally asking. I assume there’d be significant issues, maybe e.g. dealing with a non-fixed total money pool.
I like this vision! “LLM-native Manifold” seems like an obvious thing to explore, but I don’t know of anyone doing this. Some other thoughts:
Manifold-like systems seems like a nice way of running an tournament/evolutionary algorithm to find the best model & scaffold for forecasting. One of the historical arguments for prediction markets is that they help surface human talent (by allowing smart forecasters to have win money/influence in society), and you could view “conjuring the right scaffold” as the modern equivalent
There’s a few distinct intellectual tasks in organizing a prediction market, which could be supercharged with LLMs: forecasting/trading (which has been explored the most), question creation, and question resolution. I suspect LLM assistance on the latter two could help with making PMs actually useful with decisionmaking.
Within forecasting/trading, a lot of effort has gone into something like “make better brier/loss score” (eg see forecastbench), but I think a lot of the nuance in trading well involves being good at market selection and bet sizing, aka knowing where your edge is and how confident to be in it. I’d like to see more people trying to incorporate this into bots, and see bet sizing matter more in bots
LLMs seem like our best bet of getting futarchy to work; with cheap intelligence there are a lot more things we could try here
Has anyone really been far even as decided to use even go want to do Garrabrant induction? IDK what this would look like, just generally asking. I assume there’d be significant issues, maybe e.g. dealing with a non-fixed total money pool.