Also known as Max Harms. (I post AI alignment content under my other account.)
Not the same person as MaxH!
Raelifin
Karma: 638
This market looks semi-reasonable to me: https://manifold.markets/MaxHarms/when-will-the-red-heart-audiobook-c
It’s hard for me to make concrete predictions because I have a lot of agency, and it depends on things like my standards and priorities. It turns out I’m moving across town in December, so that will delay things. If I had to guess, I would say the market is over-estimating the chance it’ll be <January and >April and under-estimating my chances of getting it done in February, but :shrug:.
Thanks for such a glowing review!! I’m so glad you heart the book!
The least realistic part of Red Heart is simply that there’s a near-superhuman AI in the near future at all[2].
I’d be curious for the specific ways in which you feel that Yunna is unrealistically strong or competent for a model around the size of GPT-6.5 (which is where I was aiming for in the story). LessWrong has spoiler tags in case you want to get into
the ending. (Use >! at the start of a line to black it out.)
The story actually starts in an alternate-timeline October 2023. I knew the book would be a period-piece and wanted to lampshade that it’s unrealistically early without making it distracting. Glad to hear you didn’t pick up on the exact date.
Just to defend myself about AI 2027 and timelines, I think a loss-of-control event in 2028 is very plausible, but as I explain in the piece you link in the footnote, my expectation is actually in the early 2030s, due to various bottlenecks and random slowdowns. But also, error bars are wide. I think the onus should be on people to explain why they don’t think a loss of control in 2028 is possible, given the natural uncertainty of the future and the difficulty of prediction.
Regardless, thanks again. :)
With Crystal, I just slammed them out there with pretty minimal effort. I gave Society away for free, and didn’t make paperback copies until just recently. For Red Heart I thought the story might have broader appeal, and wanted to get over my allergy to marketing, so I reached out to a bunch of literary agents early this year. Very few were interested, and most gave no reason. One was kind enough to explain that as a white guy writing a book about China, it would be an uphill battle to find a publisher, and that I’d probably need a Chinese co-author to make it work. She estimated that optimistically I might be able to get it in stores in 2027. From my perspective that was way too slow, and since I already had experience self-publishing, I went down that route. Self-publishing is extremely easy these days, and can produce a product of comparable quality if you are competent and/or have a team. The main issue is marketing and building awareness; traditional publishing still acts as a gatekeeper in many ways. So I’m still extremely dependent on word-of-mouth recommendations.
Lovely to find yet another person who benefited from my stories. I hope you enjoy Red Heart! ❤️
If you want another fun story with the same gimmick, one of my favorite books is The Planiverse.
Yeah, these are good questions. I mostly don’t suggest people try to support themselves writing unless they already know they’re very good at storytelling, and even then it’s hard/rare. Instead, I think it’s good for people to experiment with it as a side-thing, ideally in addition to some useful technical work. (I’m very blessed that I get to work as a researcher at MIRI, for example, and then go home and write stories that are inspired by my research.) Don’t wait to be discovered by a literary agent; if you write something good, post it online! Only try to seriously monetize after you already have some success.
Regarding how to tell if your stories are good, I think the main thing is to get them in front of people who will be blunt, and find out what they say. LLMs are a good stepping-stone to this, if you’re hesitant to get a real human to read your work, though you’ll have to shape their prompt so that they’re critical and not sycophantic. Writing groups can also be a good resource for testing yourself.
Manifold market here: https://manifold.markets/MaxHarms/will-ai-be-recursively-self-improvi
Just wanted to remind folks that this is coming up on Saturday! I’m looking forward to seeing y’all at the park. It should be sunny and warm. Feel free to send me requests for snacks or whatever.
Is there a minimal thing that Claude could do which would change your mind about whether it’s conscious?
Edit: My question was originally aimed at Richard, but I like Mikhail’s answer.
Thanks! The creators also apparently have a substack: https://forecasting.substack.com/
Value of information
If you have multiple quality metrics then you need a way to aggregate them (barring more radical proposals). Let’s say you sum them (the specifics of how they combine are irrelevant here). What has been created is essentially a 25-star system with a more explicit breakdown. This is essentially what I was suggesting. Rate each post on 5 dimensions from 0 to 2, add the values together, and divide by two (min 0.5), and you have my proposed system. Perhaps you think the interface should clarify the distinct dimensions of quality, but I think UI simplicity is pretty important, and am wary of suggesting having to click 5+ times to rate a post.
I addressed the issue of overcompensating in an edit: if the weighting is a median then users are incentivized to select their true rating. Good thought. ☺️
Thanks for your support and feedback!
I agree that there are benefits to hiding karma, but it seems like there are two major costs. The first is in reducing transparency; I claim that people like knowing why something is selected for them, and if karma becomes invisible the information becomes hidden in a way that people won’t like. (One could argue it should be hidden despite people’s desires, but that seems less obvious.) The other major reason is one cited by Habryka: creating common knowledge. Visible Karma scores help people gain a shared understanding of what’s valued across the site. Rankings aren’t sufficient for this, because they can’t distinguish relative quality from absolute quality (eg I’m much more likely to read a post with 200 karma, even if it’s ranked lower due to staleness than one that has 50).
I suggested the 5-star interface because it’s the most common way of giving things scores on a fixed scale. We could easily use a slider, or a number between 0 and 100 from my perspective. I think we want to err towards intuitive/easy interfaces even if it means porting over some bad intuitions from Amazon or whatever, but I’m not confident on this point.
I toyed with the idea of having a strong-bet option, which lets a user put down a stronger QJR bet than normal, and thus influence the community rating more than they would by default (albeit exposing them to higher risk). I mainly avoided it in the above post because it seemed like unnecessary complexity, although I appreciate the point about people overcompensating in order to have more influence.
One idea that I just had is that instead of having the community rating set by the weighted mean, perhaps it should be the weighted median. The effect of this would be such that voting 5-stars on a 2-star post would have exactly the same amount of sway as voting 3.5, right up until the 3.5 line is crossed. I really like this idea, and will edit the post body to mention it. Thanks!
I agree with the expectation that many posts/comments would be nearly indistinguishable on a five-star scale. I’m not sure there’s a way around this while keeping most of the desirable properties of having a range of options, though perhaps increasing it from 10 options (half-stars) to 14 or 18 options would help.
My basic thought is that if I can see a bunch of 4.5 star posts, I don’t really need the signal as to whether one is 4.3 stars vs 4.7 stars, even if 4.7 is much harder to achieve. I, as a reader, mostly just want a filter for bad/mediocre posts, and the high-end of the scale is just “stuff I want to read”. If I really want to measure difference, I can still see which are more uncontroversially good, and also which has more gratitude.
I’m not sure how a power-law system would work. It seems like if there’s still a fixed scale, you’re marking down a number of zeroes instead of a number of stars. …Unless you’re just suggesting linear voting (ie karma)?
Ah! This looks good! I’m excited to try it out.
Yep. I’m aware of that. Our karma system is better in that regard, and I should have mentioned that.
Nice. Thank you. How would you feel about me writing a top-level post reconsidering alternative systems and brainstorming/discussing solutions to the problems you raised?
I also want to note that this proposal isn’t mutually exclusive with other ideas, including other karma systems. It seems fine to have there be an additional indicator of popularity that is distinct from quality. Or, more to my liking, would be a button that simply marks that you thought a post was interesting and/or express gratitude towards the writer, without making a statement about how bulletproof the reasoning was. (This might help capture the essence of Rule Thinkers In, Not Out and reward newbies for posting.)
Nope. If you have specific questions I’d be happy to answer them.