Hey there~ I’m Austin, currently building https://manifund.org. Always happy to meet LessWrong people; reach out at akrolsmir@gmail.com!
Austin Chen
Karma: 2,335
Yeah, I absolutely encourage people to write docs with funding proposals and share them around!
Common App for AIS could go farther, such as by:
Answering various annoying repeated technical questions to help fit a specific funder’s system (eg “who is fiscally sponsoring you”, “what are min/med/max versions of your project”)
Helping funders see when other funders have committed to fund an opportunity, to avoid overfunding
Sharing public or internal notes between funders, eg particular sensitive reasons why one might or might not fund this project, to reduce investigative work
In one sense, the SFF S-Process is already like a Common Application where many different kinds of funders (speculators, reviewers) look into different applications and share notes and fund them accordingly. Unfortunately the SFF S-process suffers from other usability problems (eg very complicated application form, and also is only open once a year, and is generally very slow to get funds to recipients); and they in practice haven’t succeeded at getting other funders (eg CG, Longview, OAIF & A\) bought in. I believe @habryka is hoping to improve on some of these points!
And for completeness, some reasons to be bearish on a common app: more groupthink maybe bad; tech/startup funding ecosystem doesn’t really have a “common app”. I’m not sure to what degree academic funding is like a common app. Common Apps seem to be more common in college application/medical residency settings.
Relatedly, one could also imagine a common app for various fellowships (MATS, PrincInt, SPAR, Horizon, Tarbell, etc) to whatever degree that makes sense.
Maybe a good fit for Summer Camp at the festival season this year!
I broadly agree with the takes in this post, and would be very excited for designs/implementations of markets that properly price in value of information. (to wit: I’d like to be able to spend $10, $100, $1000 etc on a question like “what kind of person will we hire” and get answers that I think are worth the cost. Even more for questions whose value to society is diffuse, eg “will China invade Taiwan in 2027″)
If you are working on this, please reach out!
This is fun to see! A lot of questions that come to mind:
Have you asked your bot where it thinks its alpha comes from?
Have you tracked cost of compute for operating this over time? How does it compare to the mana spent (eg is it “profitable”). (I assume it’s deeply unprofitable after adding in the cost of your own time—future goals!)
Specifically, have you investigated trying to spend more compute (“thinking harder”) for better performance or higher liquidity markets?
Do you think that performance would scale well as your bot’s mana bankroll increases?
Also, please consider coming to Manifest!
I’ve been doing SLIT for the last 3 months. It was relatively convenient to sign up for, a single video call and then they mail you a little bottle for 3mo of liquid. The drops are reasonably pleasant, sweet tasting.
They do ask you to hold them under your tongue for 2min, which is a bit annoying (especially to have to do every day), but my guess is net it’s still much easier than shots.
I’m not sure if my pollen allergies have dropped much yet—apparently it’s the second shipment where they ramp up the pollen concentration, and the whole thing takes 6 to 12mo to be effective.
In the meantime, I’ve been also using Flonase, which I think is also pretty good (better than Zyrtec).
It only takes one thoughtless gambler with a large bankroll to outweigh a bunch of high signal traders. Or a bunch of low signal traders with a collectively large bankroll to outweigh a high signal trader.
yes—but one of the nice fundamental properties of prediction markets is that over time, thoughtful people accumulate larger bankrolls
and yes, I think Metaculus comments are generally quite good; Manifold’s are sometimes good, Polymarket/Kalshi are approximately garbage. This is I think partly cultural effects (and product decisions) about who comments vs trades and how those get represented, but yes also reflects something important about the distribution of the underlying audience—that Metaculus has a handful of extremely thoughtful forecasters, polymarket may have that plus a thousand degen gamblers. my contention again is that the structure of markets happily means that the latter can still be quite accurate.
I don’t know if you’ve seen https://brier.fyi/ (and, imo their results should be taken with a grain of salt, though also I might just be salty); but my main takeaway is that they’re all pretty calibrated, and broadly could be cited much more (whether market or poll)
(thanks)
*one thoughtful trader with a large bankroll to outweigh a bunch of low signal traders.
(granted, at the tails, it does become more expensive to further lower the odds. Eg at 5%, you’re paying $19 per $1 downwards)
I think this is overblown and mostly highlights the difficulty of getting prediction markets to proper % at the tails, due to opportunity cost of money.
On the subject of participation, a sometimes underappreciated fact is that prediction markets allow much deeper participation in the form of voting with more dollars. Even if most most participants are naive, it only takes one thoughtful
(granted, at the
Thanks for continually writing about these kinds of political opportunities, in public!
Some people & places you may want to reach out to to float this article (and your volunteer signup form), chosen for having some context on NYC specifically:
Zvi Mowshowitz
Jessie of collider.nyc
@Screwtape / Skyler who organizes the east coast megameetup
Overcoming Bias/ACX/Rationality NYC (maybe, @Robi Rahman?)
(happy to intro, and apologies if these are obvious!)
Thanks for writing this! Some reasons I would steelman continued funding towards tetlockian or PM-style forecasting:
Source and screen for talent. There sure is some correlation between forecasting well and doing important things in EA. Just picking some people I know: Joel Becker, another former #1 Manifold trader, went on to join METR and then do their famous uplift studies. Eli Lifland went on to help make AI 2027. Peter Wildeford started Rethink Priorities, now IAPS. And some of your own track record in making good early-stage EA grants is here.
Beyond that, a bunch of smart and interesting people have expressed a lot of interest in forecasting, from banner bearers like Scott Alexander and Vitalik Buterin and Robin Hanson, to surprising cases like Anthony Giovanetti (of Slay the Spire) to <anon famous AI researcher who DM’d me> to Sam Altman. I do think there’s some amount of intellectual fashion-ism going on here, but also, you should fish where the fish are.
When funding is abundant, one bottleneck becomes finding (and building consensus around) talent; if the only thing that a bunch of money spent on forecasting does is to identify good people, that may be worth it.Fast, accurate info in times of chaos. Prediction markets are actually quite good at distilling through noise during times of high uncertainty, eg recently around russia/ukraine war and the iran war. Manifold’s usage numbers spike every time there’s a crisis. Because PMs pay a high premium towards being speedy, they’re often the fastest trustworthy source of data. If the world becomes more chaotic due to faster tech growth, it may be quite valuable to have this place to stay up to date.
New unlocks from growth in AI capabilities. Historically, one very expensive input into forecasting is forecaster time. As LLMs catch up to top human forecasters, it’ll soon be cheap to get a calibrated answer to any question one might ask. On priors, this should help us with making better decisions or making futarchy possible. I agree this is somewhat speculative still, and wish more people were trying things in this space.
I included this story as a short anecdote about Marcus’s ability to spot talent, make active investments, and convince founders to take the leap, all of which I expect to transfer into helping start great AI x Animal orgs. I understand that different people in EA/AI safety have different takes about whether Mechanize specifically is good or bad—I happen to think good or at least neutral.
(And I take responsibility for any factual errors with this specific anecdote. Talking to Marcus just now, it seems like his main nudge was to convince Ege/Matthew/Tamay that the nonprofit structure was wrong for what they wanted to accomplish.)
the tension between job, career, and the actual work you think is important
- I spend a lot of my time thinking about what my soon-to-be patrons might want to fund.- Recently this has been Anthropic employees, which is weird and stressful in a bunch of ways (“Anthropic employees” are not a single coalition; many are friends and asking them for money is icky; all are busy, and already being swarmed by other people seeking their money, and therefore defensive).
- But historically it’s also been various potential funders, maybe OP/CG as the biggest of them. Which, on reflection, feels a bit insane given that OP/CG have never actually funded any of my work (and mostly haven’t funded the people funding my work!)
- I also think I have a pretty good track record of just, doing the thing and believing that money will come. We shipped Manifold v0 before we got the first grant; Manifest and Mox were internally funded first.
- I’m really tempted to give advice like “do great work and the money will follow”, and it’s kind of true, but also maybe generates a lot of bycatch?
- Probably my biggest patron by total $ was FTX Future Fund. That’s probably part of why I’m so defensive of them, even now. Maybe, half of Manifund is just keeping the spirit of the Future Fund alive.
- One way to avoid the downsides of patronage is to go direct, make your own money. Substack is the classic example. (But then, paying subscribers on Substack or the general internet landscape has its own set of preferences and downsides)
As a patron myself?
- on a small scale, I do enjoy funding stuff (mostly weird software or meta projects)
- and institutionally, Manifund sponsoring Inkhaven is an (indirect) way of supporting the arts. (supporting the supporters of the arts, I guess)
- beyond money, there are other ways to support creators, which a lot of my recent career has been about. For example, building tools for them (Manifold), events (Manifest), and operational support (ACX Grants).
Patronage vs other funding mechanisms
- I get the feeling patronage is kinda “cool”. Emergent Ventures was cool, Erik Hoel has a writeup on it, it was cool when Gwern got $100k from some startup founder.
- Maybe like 2021-2023ish there was a lot more “no strings attached microgrants are awesome” discourse. Beyond FTX stuff this was ACX Grants, Francisco San, Moth Fund, AI Grants. It feels somewhat out of fashion now.
Funding writing, specifically: If you have money (and maybe, a lot of money), how should you cause good writing and art to exist?
- Most directly, you can do the kind of patronage Jenn talks about, give money directly to good writers. Somehow this seems a lot rarer than it ought to be? Is there a missing product or norm here?
- You can just commission pieces. (Many times, I’ve tried to hire people to write for the Manifold or Manifund substack. Mysteriously this hasn’t worked out that well eg, no essay that hit top of Hacker News).- Another approach is to farm writers. Get a bunch of people writing at once, and see the winners. Which is the Inkhaven or other residency/fellowship/batch approach.
- Another is to start a journal or publishing outfit. Stripe Press, Asterisk Mag, Asimov Press.
- Orgs can also just sponsor fulltime writers. Sometimes this is a “fellow”, loosely affiliated. (Anthropic/OpenAI writing fellowship when?)
- Some thinktanks are just orgs that just write, Forethought, Institute for Progress feels like this.
- Unfortunately a lot of great writing is locked in the heads of people who have extremely high opportunity costs. It seems like OpenPhil basically just paid Joe Carlsmith to write stuff for a while, which seemed great. Also somehow Holden Karnofsky stopped running OP to write stuff for a while, which also seemed great. I’m always happy when Oli Habryka takes a break from running Lightcone to drop some new essays. (see also https://aarongertler.com/too-good/.)
- And essay competitions are a thing, ofc.
Essay competitions:
- Writing has the nice property of being cheap to assess, which is possibly why competitions are a reasonable structure
- EA is somehow: blessed with an abundance of great writers, and also really into prizes/competitions, and also has a terrible track record of the competitions working well. IMO, the Blog Building fellowship fell apart for ??? reasons, Cause Exploration Prizes didn’t turn out anything good, the best EA Fiction Contest submission was written by a judge. The best EA Red Teaming entries were not really submitted for the competition, iirc they were Scott’s and Holden’s.
- (though, it sure seems like I’m just anchored to my existing favorite writers)
- Lesswrong does a cool yearly “best of” retrospective voting/judging thing of writing from 2 years ago? Maybe that’s the true good cadence to judge things on.this braindump brought to you by the MANIFUND ESSAY PRIZE https://manifund.org/essay, submit by Fri Apr 24!
Hey Phillip! I wanted to say that I quite enjoyed reading this, as a fellow Catholic and fairly neurotypical (I think??) person. Having been around this scene for a few years now, it’s fun to find out what strikes newcomers as noteworthy; I’m excited for the rest of your Inkhaven writings!
Calibration City https://calibration.city/ is also a great resource for looking into questions about calibration of various prediction market platforms!
For Mox events, our rule of thumb is that attendance is 100% of Partiful Goings, or 50% of Luma RSVPs. Obviously a protest/march may have different dynamics, but this method would forecast ~120 participants.
Dumb question, why do this on a weekend instead of a weekday? I imagine a lot more employees show up on weekdays (though maybe protestors are more available on weekends and maximizing crowd size is important?)
One additional reason that capacity-building for AI safety seems good right now, is that very soon I expect there to be a lot more funding available for AI safety work, from Anthropic donors (see Front-Load Giving Because of Anthropic Donors? and this comment of mine) as well as a broader societal wakeup about risks from AI.
When money becomes more available, the bottleneck becomes “good opportunities/people to spend money on”, which is what capacity-building produces. Also starting asap seems important—capacity-building takes time to set up and bear fruit, and some kinds of capacity building have snowball-y effects (eg MATS).
Curious whether y’all considered Tiptap as a base for the editor, and if so why you decided against it?
(Tiptap is what we use for Manifold/Manifund and there are definitely some warts—eg markdown not being supported out of the box, though recently I think that’s changed—but mostly I’ve liked it.)
I like this vision! “LLM-native Manifold” seems like an obvious thing to explore, but I don’t know of anyone doing this. Some other thoughts:
Manifold-like systems seems like a nice way of running an tournament/evolutionary algorithm to find the best model & scaffold for forecasting. One of the historical arguments for prediction markets is that they help surface human talent (by allowing smart forecasters to have win money/influence in society), and you could view “conjuring the right scaffold” as the modern equivalent
There’s a few distinct intellectual tasks in organizing a prediction market, which could be supercharged with LLMs: forecasting/trading (which has been explored the most), question creation, and question resolution. I suspect LLM assistance on the latter two could help with making PMs actually useful with decisionmaking.
Within forecasting/trading, a lot of effort has gone into something like “make better brier/loss score” (eg see forecastbench), but I think a lot of the nuance in trading well involves being good at market selection and bet sizing, aka knowing where your edge is and how confident to be in it. I’d like to see more people trying to incorporate this into bots, and see bet sizing matter more in bots
LLMs seem like our best bet of getting futarchy to work; with cheap intelligence there are a lot more things we could try here
1) Thanks! I agree this seems like an important problem to solve.
One dumb idea is to make it more “trustless”, eg by recording the phone screen so that the other orgs don’t need rely on word of mouth, they can actually just watch the interview (or get an llm to transcribe and summarize, etc). Obviously you’d want to get opt in from all sides, etc.
2) I hadn’t seen that! It’s cool that exists but I think the ideal version would be better designed and aimed at going viral.