I see how “Local alignment != asymptotic alignment” is more accurate but I find the current title/claim easier to understand.
I coudl see them being a good pair, where the current title makes the claim and the local vs asymptotic stuff adds the mechanism. But the mechanism without the claim, I would fear, would fail to land for many people. Just sth to keep in mind as one datapoint if you ever do a followup post :)
Luc Brinkman
Karma: 142
Yeah, that’s tricky. My hope is that once you’re decently good at execution, you can recognize good execution, such that you can distinguish between the two.
What seems harder yet is “idea is flop and should be adjusted” vs “idea is flop and I should switch to a different idea” (a distinction which, by the way, lives on a spectrum)
The Lean Startup describes this as the “Pivot or Persevere” question.Where persevere is roughly: iterate more on the current idea.
And Pivot is roughly: iterate/change to a different idea.
There are movements that say people should quit sooner (see e.g. https://www.youtube.com/watch?v=3Xdyioqs5Ds ), and stories of startup founders who stubbornly stuck to their vision, rejection after rejection (whilst iterating).
Oh, also, getting advice from others is probably helpful on the execution front. If you’re underexecuting, people can probably recognize that, I expect. Whereas good advise on the product-market-fit side, i.e. “should I iterate on this idea or pivot to another one) is very hard to give/get.
Awesome. We’re about to create our facilitator training course so that could be one way to contribute. There’s loads of other ways too. Can you reach out in the intro-offers-asks channel on our server? https://discord.gg/nn7HrjFZ8E
For marking/tracking AI vs human written content, I’ve been watching Every’s new “Proof Editor” with some interest. https://proofeditor.ai/ Probably worth checking out their approach for inspiration. (Might be implementing something similar for our team’s internal custom Obsidian/Notion replacement)
Maybe the design could be inverted, where authors can label specific sections as human-written instead of labeling (the majority of sections as) AI-written? I think getting asssistance from AI is going to be the default for and more people, and trusting on people to be up to date with LW policies AND the philosophies behind them (including how AI writing doesn’t reflect internal thought processes and such) AND to self-report LLM content (when general social stigma works against that) feels like a lot of dependencies.
(I can see other problems with this inversed design but will share the above anyway to spur creativity).
There’s also something with “this section is human written” that feels nicer to me—more like an opt-in instead of a punishment.
Thanks for sharing your thoughts, Plex. I can imagine there are indeed some psychological considerations into having an effective commitment mechanism.
Is there anything in particular that having access to a list of people signed up for a mission statement like that would enable you, or those people, to do?
AI Safety Pledge
Inspired by the Founder’s Pledge and the 10% Pledge, we can offer people transitioning to an AI safety career to make an AI Safety Pledge. It could look something like this:
I pledge to spend the coming years of my career on AI safety.
If I don’t manage to do so, for example because I can’t find a job in AI Safety, I will donate 10% of my income to the AI safety movement.
If I ever do decide to move back into AI safety, I can receive back my contributions to support my AI safety work.
Note: this is a very early idea, not a fully fledged proposal. I am currently entertained by the idea of an AI Safety Pledge, but not convinced that it’s useful and desirable. I’m posting it here to see what opinions people have about this.
Theory of Change
Hopefully, this pledge will:
incentivize people to try harder to complete their career transition to AI safety
create more buy-in towards keeping people accountable to their good intentions, e.g. through a virtual career coach.
decrease the effective income gap between non-safety work (which would now be reduced by 10%) and AI safety work
incentivize people to keep trying moving back to AI safety even if they weren’t successful initially
Some of the risks:
it might stimulate earning-to-give, which 80’000 hours currently views as less effective than direct career contributions.
it might be perceived by the outside world as cult-like
AI safety work may be hard to define
What do you think? Could this be useful? What assumptions would need to be true for this to be impactful? What could be a simple way of testing those assumptions?
Yup, that would be a healthier view for society to hold. Sadly in pretty much any field that I’m aware of, companies are allowed to push ahead until accidents happen and until it’s proven that it’s unsafe.
We’ll need to find some way of overcoming that default since we’ll only get 1 real shot atn superintelligence alignment.