chanamessinger
Is this something Stampy would want to help with?
https://www.lesswrong.com/posts/WXvt8bxYnwBYpy9oT/the-main-sources-of-ai-risk
I think that incentivizes self-deception on probabilities. Also, P <10^-10 are pretty unusual, so I’d expect that to cause very little to happen.
Thanks!
When you say “They do, however, have the potential to form simulacra that are themselves optimizers, such as GPT modelling humans (with pretty low fidelity right now) when making predictions”
do you mean things like “write like Ernest Hemingway”?
Is it true that current image systems like stable diffusion are non-optimizers? How should that change our reasoning about how likely it is that systems become optimizers? How much of a crux is “optimizeriness” for people?
Why do people keep saying we should maximize log(odds) instead of odds? Isn’t each 1% of survival equally valuable?
In addition to Daniel’s point, I think an important piece is probabilistic thinking—the AGI will execute not based on what will happen but on what it expects to happen. What probability is acceptable? If none, it should do nothing.
Have you written about your update to slow takeoff?
Nice! Added these to the wiki on calibration: https://www.lesswrong.com/tag/calibration
Two Guts
Oh, whoops. I took from this later tweet in the thread that they were talking.
After years of tinkering and incremental progress, AIs can now play Diplomacy as well as human experts.[6]
Maybe this happened in 2022: https://twitter.com/polynoamial/status/1580185706735218689
Let me know if you have a cheerful price for this!
I will talk to the developer about it being open source—I think that was both of our ideals.
Do you know how to do this kind of thing? I’d be happy to pay you for your time.
This seems interesting to me but I can’t yet latch onto it. Can you give examples of secrets being one or the other?
Are you distinguishing between “secrets where the existence of the secret is a big part of the secret” and “secrets where it’s not”?
One of my feature requests! Just hard to do.
Calibrate—New Chrome Extension for hiding numbers so you can guess
Why would they be jokes?
Don’t know what you mean in the latter sentence.
This is great, and speaks to my experience as well. I have my own frames that map onto some of this but don’t hit some of the things you’ve hit and vice versa. Thanks for writing!