The failure mode I’m concerned about is missing big chunks of content, so you only get 1⁄2 of the relevant stuff or something. It may not be that bad a failure mode, but like, for example, just asking a Fable chat probably would turn up far under 90% of the past convos between Wei / Zack / LW mods? CC would probably do better? (I have done a significant amount of using CC to “read” many papers, and I have to do nonzero poking CC to improve fetching methods; by default it gives up for various reasons even when there is a way.)
I agree it might not be totally comprehensive, but, that seems kinda fine? A lot of the concepts have gotten repeated. (Like, seems fine to try a lower effort version and see how it goes and try a higher effort version if it feels on-track-to-be-helpful but insufficient)
There are some difficulties with that because of system prompts and default tools.
Claude.ai/ClaudeCode by default read websites using the built-in WebFetch tool. The WebFetch tool loads the HTML, turns it into Markdown and feeds it into Haiku(! Edit: Claude.ai’s tool does not feed it into Haiku), which summarizes (with strong instructions for limiting verbatim quotes!) and only that output goes to Fable. (By contrast ChatGPT/Codex’s default tool gives the full text, but also has strong instructions for limiting verbatim quotes. For Codex this is only in the tool instruction.)
ClaudeCode/Codex can easily work around this by using e.g. cURL (especially if you tell them about lesswrong.com/api). Edit: Claude.ai can too, if you enable Network egress in the settings. Both can of course create you a re-usable skill. And with some prompting you can get around the quoting limitations.
Right, I’ve had cc download my whole LW history, and I had to make a system to actually download the actual entire whole paper. I think one can easily do it by asking cc, but one has to specifically ask.
The failure mode I’m concerned about is missing big chunks of content, so you only get 1⁄2 of the relevant stuff or something. It may not be that bad a failure mode, but like, for example, just asking a Fable chat probably would turn up far under 90% of the past convos between Wei / Zack / LW mods? CC would probably do better? (I have done a significant amount of using CC to “read” many papers, and I have to do nonzero poking CC to improve fetching methods; by default it gives up for various reasons even when there is a way.)
I agree it might not be totally comprehensive, but, that seems kinda fine? A lot of the concepts have gotten repeated. (Like, seems fine to try a lower effort version and see how it goes and try a higher effort version if it feels on-track-to-be-helpful but insufficient)
There are some difficulties with that because of system prompts and default tools.
Claude.ai/ClaudeCode by default read websites using the built-in WebFetch tool. The WebFetch tool loads the HTML, turns it into Markdown and feeds it into Haiku(! Edit: Claude.ai’s tool does not feed it into Haiku), which summarizes (with strong instructions for limiting verbatim quotes!) and only that output goes to Fable. (By contrast ChatGPT/Codex’s default tool gives the full text, but also has strong instructions for limiting verbatim quotes. For Codex this is only in the tool instruction.)
ClaudeCode/Codex can easily work around this by using e.g. cURL (especially if you tell them about lesswrong.com/api). Edit: Claude.ai can too, if you enable Network egress in the settings. Both can of course create you a re-usable skill. And with some prompting you can get around the quoting limitations.
Right, I’ve had cc download my whole LW history, and I had to make a system to actually download the actual entire whole paper. I think one can easily do it by asking cc, but one has to specifically ask.