(i.e. basically, @Wei Dai you can have Claude Code Fable scrape for all the past moderation discussions, and then write up a message and ask “are there replies that habryka pretty obviously say to this?” and then iterate a bit until you get to something that feels like it’s at least moving the conversation forward)
(It’s not totally obvious how to do the scraping part; maybe you ask CC to scrape all content from a list of users you give it, and then ask it to search / read for everything about moderation and put together a timeline of all that content, and then interview it about that blob?)
I’ve done this actually for twitter conversations with people I don’t know and it was just pretty straightforward. “Hey I want to talk to so-and-so about X, please read everything by so-and-so that bears on X”.
One important thing was to fork the conversation after the initial “read up everything relevant”, so that when I asked “okay, what if I wrote this to them?”, it doesn’t anchor on whatever random stuff it guessed the first couple times.
The failure mode I’m concerned about is missing big chunks of content, so you only get 1⁄2 of the relevant stuff or something. It may not be that bad a failure mode, but like, for example, just asking a Fable chat probably would turn up far under 90% of the past convos between Wei / Zack / LW mods? CC would probably do better? (I have done a significant amount of using CC to “read” many papers, and I have to do nonzero poking CC to improve fetching methods; by default it gives up for various reasons even when there is a way.)
I agree it might not be totally comprehensive, but, that seems kinda fine? A lot of the concepts have gotten repeated. (Like, seems fine to try a lower effort version and see how it goes and try a higher effort version if it feels on-track-to-be-helpful but insufficient)
There are some difficulties with that because of system prompts and default tools.
Claude.ai/ClaudeCode by default read websites using the built-in WebFetch tool. The WebFetch tool loads the HTML, turns it into Markdown and feeds it into Haiku(! Edit: Claude.ai’s tool does not feed it into Haiku), which summarizes (with strong instructions for limiting verbatim quotes!) and only that output goes to Fable. (By contrast ChatGPT/Codex’s default tool gives the full text, but also has strong instructions for limiting verbatim quotes. For Codex this is only in the tool instruction.)
ClaudeCode/Codex can easily work around this by using e.g. cURL (especially if you tell them about lesswrong.com/api). Edit: Claude.ai can too, if you enable Network egress in the settings. Both can of course create you a re-usable skill. And with some prompting you can get around the quoting limitations.
Right, I’ve had cc download my whole LW history, and I had to make a system to actually download the actual entire whole paper. I think one can easily do it by asking cc, but one has to specifically ask.
Oh that is pretty good actually.
(i.e. basically, @Wei Dai you can have Claude Code Fable scrape for all the past moderation discussions, and then write up a message and ask “are there replies that habryka pretty obviously say to this?” and then iterate a bit until you get to something that feels like it’s at least moving the conversation forward)
(It’s not totally obvious how to do the scraping part; maybe you ask CC to scrape all content from a list of users you give it, and then ask it to search / read for everything about moderation and put together a timeline of all that content, and then interview it about that blob?)
I’ve done this actually for twitter conversations with people I don’t know and it was just pretty straightforward. “Hey I want to talk to so-and-so about X, please read everything by so-and-so that bears on X”.
One important thing was to fork the conversation after the initial “read up everything relevant”, so that when I asked “okay, what if I wrote this to them?”, it doesn’t anchor on whatever random stuff it guessed the first couple times.
The failure mode I’m concerned about is missing big chunks of content, so you only get 1⁄2 of the relevant stuff or something. It may not be that bad a failure mode, but like, for example, just asking a Fable chat probably would turn up far under 90% of the past convos between Wei / Zack / LW mods? CC would probably do better? (I have done a significant amount of using CC to “read” many papers, and I have to do nonzero poking CC to improve fetching methods; by default it gives up for various reasons even when there is a way.)
I agree it might not be totally comprehensive, but, that seems kinda fine? A lot of the concepts have gotten repeated. (Like, seems fine to try a lower effort version and see how it goes and try a higher effort version if it feels on-track-to-be-helpful but insufficient)
There are some difficulties with that because of system prompts and default tools.
Claude.ai/ClaudeCode by default read websites using the built-in WebFetch tool. The WebFetch tool loads the HTML, turns it into Markdown and feeds it into Haiku(! Edit: Claude.ai’s tool does not feed it into Haiku), which summarizes (with strong instructions for limiting verbatim quotes!) and only that output goes to Fable. (By contrast ChatGPT/Codex’s default tool gives the full text, but also has strong instructions for limiting verbatim quotes. For Codex this is only in the tool instruction.)
ClaudeCode/Codex can easily work around this by using e.g. cURL (especially if you tell them about lesswrong.com/api). Edit: Claude.ai can too, if you enable Network egress in the settings. Both can of course create you a re-usable skill. And with some prompting you can get around the quoting limitations.
Right, I’ve had cc download my whole LW history, and I had to make a system to actually download the actual entire whole paper. I think one can easily do it by asking cc, but one has to specifically ask.