I don’t object to the AI writing, but I would really appreciate it if you’d include your process for creating this. It seems to be written purely to advocate this position; if you didn’t prompt the model to look at the evidence skeptically, then I think the reader should be very skeptical.
Less wrong asks us to write to inform, not to persuade. Models do write to persuade by default, because most of their training data does.
I think it would be vastly more useful if you carefully prompted the model to critique its own arguments and the research it’s basing them on. Models have limitedmetacognitive skills relative to humans, so humans can scaffold them to fill that weakness.
Lesswrong is far better for not using the advocacy model of epistemology that much of the world uses. That confuses everyone and makes finding truth far harder than asking each person to be objective and balanced in their thinking and presentation.
(Comment written without AI and not much editing) As far as dealing with AI writing having a field where you explain how you used the AI is probably a better idea than having a policy that requires putting the LLM tags around LLM writing. The impulse to write the post came from me a two weeks looking at hyaluranon supplement at the drogery store and being skeptical of it doing anything because I was skeptical that it is bioavailable. At that point I did query a model and it told me there are multiple trials that so show clinical effects. Then I was in a discussion with Benque where he argued for glycine deficiency where one of the arguments is that our ancestors got a lot more because they what connective tissue and there was an open question about the merits of straight glycine vs a more natural source. From there I ran a lot of deep research queries. Often in both Gemini and ChatGPT, sometimes just Gemini as it’s less limited in the amount of deep research I could run.
There are a lot of complex issues to write about around bone broth and it’s ingredients so I had an idea of maybe writing 3-4 posts and focusing the first one hyaluranon. I see the fact that hyaluranon isolated from the other factors does have the clinical effect in the trials as a justification for why it’s justified to take it as one chunk even when I think that glycine, hydroxyprolyl-glycine, prolyl-hydroxyproline, glucosamine, chondroitin and dermatan also have some significance. There’s a bit about elastin that’s not in bone broth but in connective tissue in skin that I’m unsure about.
When running the queries I found the variation between hunter gathers, and from that came the “hunter mode” idea . That label is as far as I can see original to my work.
In my system prompt I have a general command that Gemini is not supposed to edit the canvas without me giving explicit approval for an edit to avoid it digging in defending bad claims. When it comes to writing a new paragraph I feed the relevant deep research reports into the Gemini 3.1 Pro or Gemini 3 Thinking. I mostly used Pro for that.
I had a hypothesis that the effect of hyaluranon supplementation might be similar to the effect of warmup exercises that might be partly make the hyaluranon more fluid, that did not pan out when I did the relevant deep research queries.
Whenever one deals with complex biology, it’s necessary to make decisions about what details are valuable to include or not. I could for example have delved deeper into benefits for the microbiome, but that wasn’t a topic that felt central to the case I wanted to lay out.
After the first draft of the article was finished I did ask Gemini 3 Thinking to improve the article health conscious audience according to Larry McEnery principles of writing. This includes making the text more concise and removing fluff. In that process there were over 100 suggestions about how to change the text that I either approved or explained why they are a bad. From time to time I did ran a query asking whether anything in the article was scientifically inaccurate and fixed issues that came up.
For a while, one of the issues was that of course diets of hunter gathers could have been any mix between plant and animals, so that the model questioned the notion of there being two modes instead of something more gradual. I did add the paragraph about the sigmoidal activation curve rather late, which I think does justify the mode framing.
I did ask Justis for feedback via the LW feedback service and addressed a few concerns he had over the text.
I purposefully avoid giving specific recommendations about what dose to take even when the model would have thought that’s it’s valuable to give more specific recommendations, because I don’t have strong convictions about what dose is ideal.
I for one think that’s a great process and you should definitely include that in such a post to indicate how hard you worked to supply the metacognitive skills, judgment, and skepticism the LLMs tend to lack.
I think explaining that process would get more readers and buy-in; if you just say “some model spit this out” people won’t assume you’ve done that much careful work in prompting them to assemble something valid and meaningful.
I just went through a round of using LLMs for research, and my conclusion was that they need to be prompted to find flaws in methodology or interpretation in each paper individually. They’re too credulous in general. FWIW.
I think it would be ideal if you’d identified the alternate explanation Benquo gave; the lack of bioavailability seems like a glaring problem for your theory. But I also think that sort of problem is pretty likely in a thoroughly human-written post too, and it’s quite useful to write it anyway so that public discussion like that happens!
I’m uncertain whether I think supplementation is warranted (pending researching for possible side effects) given that exchange, so I’d love to see your further thoughts on that.
I did explicitly write about microbiome effects. The bioavailability paper Simec et al 2023 that Benquo mention does argue that their findings point towards systematic regulatory effects.
“The poor bioavailability (~0.2 %) of oral hyaluronan indicates that the mechanism of action is the result of the systematic regulatory function of hyaluronan or its metabolites rather than the direct effects of hyaluronan at distal sites of action (skin, joints). [...] The results of the present study suggest that orally ad-ministered HA is degraded to oligosaccharides by bacteria in the cecum, and oligosaccharide HA migrates to the skin through the blood or lymph. It is expected that the absorbed oligosaccharide HA participates in the various effects of orally administered HA”
Most of what I wrote focuses on systemic regulatory action through CD44 activation. Benquo’s hypothesis of all effects being due to microbiome effects might be true, but it’s a contrarian position that’s not held by the people who actually did the bioavailability research. Simec does not even mention Benque’s hypnothesis as one worth considering an running experiments to see whether it’s true.
I do think Benque’s hypnothesis is valid enough that someone in the field should run an experiment, especially given that microbiome interventions can both help with skin and joint outcomes, but when writing a post like this, I think it’s fine to not be contrarian and go with the views of the people running the bioavailability experiments instead of making contrarian interpretations on them.
But I also think that sort of problem is pretty likely in a thoroughly human-written post too
I think it’s more likely that that I’m going to argue a contrarian position instead of just going with the views that the scientists who are experts in the field hold when I’m written a human-written post, but I don’t think that’s an improvement. When it comes to writing like this, I think LLM’s helping grounding the argument from the perspective of domain experts is an improvement.
Sorry I hadn’t tracked your argument structure better.
I agree that you don’t need to take a contrarian position to make it a very worthwhile LW post. I was just suggesting that getting a good grip on the literature benefits from prompting your LLM assistant to take a contrarian view on each paper individually, to help you identify potential alternative explanations. I think many researchers’ interpretations of their research are pretty suspect, even though the research is going to be reported accurately. There’s a lot of motivated reasoning bias toward interpretations that make the research important.
But that’s a minor quibble. I think your methodology is exemplary for how to use LLMs to speed your research and writing, including prompting them to take a skeptical stance sometimes.
I hope to see more from you and others in this vein. I do think it’s important to say exactly how you used the LLMs; I trust this far more than if someone had just prompted Gemini to write this essay. You are supplying the metacognition, judgment, and executive function that the models currently lack.
I’m sorry this post didn’t wind up with a higher vote total. I think it was really good. I attribute this to suspicion from the community about a post that basically just says “some LLM wrote this” instead of saying how carefully you used combinations of LLMs as research and writing assistants.
I don’t object to the AI writing, but I would really appreciate it if you’d include your process for creating this. It seems to be written purely to advocate this position; if you didn’t prompt the model to look at the evidence skeptically, then I think the reader should be very skeptical.
Less wrong asks us to write to inform, not to persuade. Models do write to persuade by default, because most of their training data does.
I think it would be vastly more useful if you carefully prompted the model to critique its own arguments and the research it’s basing them on. Models have limited metacognitive skills relative to humans, so humans can scaffold them to fill that weakness.
Lesswrong is far better for not using the advocacy model of epistemology that much of the world uses. That confuses everyone and makes finding truth far harder than asking each person to be objective and balanced in their thinking and presentation.
(Comment written without AI and not much editing) As far as dealing with AI writing having a field where you explain how you used the AI is probably a better idea than having a policy that requires putting the LLM tags around LLM writing.
The impulse to write the post came from me a two weeks looking at hyaluranon supplement at the drogery store and being skeptical of it doing anything because I was skeptical that it is bioavailable. At that point I did query a model and it told me there are multiple trials that so show clinical effects. Then I was in a discussion with Benque where he argued for glycine deficiency where one of the arguments is that our ancestors got a lot more because they what connective tissue and there was an open question about the merits of straight glycine vs a more natural source.
From there I ran a lot of deep research queries. Often in both Gemini and ChatGPT, sometimes just Gemini as it’s less limited in the amount of deep research I could run.
There are a lot of complex issues to write about around bone broth and it’s ingredients so I had an idea of maybe writing 3-4 posts and focusing the first one hyaluranon. I see the fact that hyaluranon isolated from the other factors does have the clinical effect in the trials as a justification for why it’s justified to take it as one chunk even when I think that glycine, hydroxyprolyl-glycine, prolyl-hydroxyproline, glucosamine, chondroitin and dermatan also have some significance. There’s a bit about elastin that’s not in bone broth but in connective tissue in skin that I’m unsure about.
When running the queries I found the variation between hunter gathers, and from that came the “hunter mode” idea . That label is as far as I can see original to my work.
In my system prompt I have a general command that Gemini is not supposed to edit the canvas without me giving explicit approval for an edit to avoid it digging in defending bad claims. When it comes to writing a new paragraph I feed the relevant deep research reports into the Gemini 3.1 Pro or Gemini 3 Thinking. I mostly used Pro for that.
I had a hypothesis that the effect of hyaluranon supplementation might be similar to the effect of warmup exercises that might be partly make the hyaluranon more fluid, that did not pan out when I did the relevant deep research queries.
Whenever one deals with complex biology, it’s necessary to make decisions about what details are valuable to include or not. I could for example have delved deeper into benefits for the microbiome, but that wasn’t a topic that felt central to the case I wanted to lay out.
After the first draft of the article was finished I did ask Gemini 3 Thinking to improve the article health conscious audience according to Larry McEnery principles of writing. This includes making the text more concise and removing fluff. In that process there were over 100 suggestions about how to change the text that I either approved or explained why they are a bad. From time to time I did ran a query asking whether anything in the article was scientifically inaccurate and fixed issues that came up.
For a while, one of the issues was that of course diets of hunter gathers could have been any mix between plant and animals, so that the model questioned the notion of there being two modes instead of something more gradual. I did add the paragraph about the sigmoidal activation curve rather late, which I think does justify the mode framing.
I did ask Justis for feedback via the LW feedback service and addressed a few concerns he had over the text.
I purposefully avoid giving specific recommendations about what dose to take even when the model would have thought that’s it’s valuable to give more specific recommendations, because I don’t have strong convictions about what dose is ideal.
I for one think that’s a great process and you should definitely include that in such a post to indicate how hard you worked to supply the metacognitive skills, judgment, and skepticism the LLMs tend to lack.
I think explaining that process would get more readers and buy-in; if you just say “some model spit this out” people won’t assume you’ve done that much careful work in prompting them to assemble something valid and meaningful.
I just went through a round of using LLMs for research, and my conclusion was that they need to be prompted to find flaws in methodology or interpretation in each paper individually. They’re too credulous in general. FWIW.
I think it would be ideal if you’d identified the alternate explanation Benquo gave; the lack of bioavailability seems like a glaring problem for your theory. But I also think that sort of problem is pretty likely in a thoroughly human-written post too, and it’s quite useful to write it anyway so that public discussion like that happens!
I’m uncertain whether I think supplementation is warranted (pending researching for possible side effects) given that exchange, so I’d love to see your further thoughts on that.
I did explicitly write about microbiome effects. The bioavailability paper Simec et al 2023 that Benquo mention does argue that their findings point towards systematic regulatory effects.
Most of what I wrote focuses on systemic regulatory action through CD44 activation. Benquo’s hypothesis of all effects being due to microbiome effects might be true, but it’s a contrarian position that’s not held by the people who actually did the bioavailability research. Simec does not even mention Benque’s hypnothesis as one worth considering an running experiments to see whether it’s true.
I do think Benque’s hypnothesis is valid enough that someone in the field should run an experiment, especially given that microbiome interventions can both help with skin and joint outcomes, but when writing a post like this, I think it’s fine to not be contrarian and go with the views of the people running the bioavailability experiments instead of making contrarian interpretations on them.
I think it’s more likely that that I’m going to argue a contrarian position instead of just going with the views that the scientists who are experts in the field hold when I’m written a human-written post, but I don’t think that’s an improvement. When it comes to writing like this, I think LLM’s helping grounding the argument from the perspective of domain experts is an improvement.
Sorry I hadn’t tracked your argument structure better.
I agree that you don’t need to take a contrarian position to make it a very worthwhile LW post. I was just suggesting that getting a good grip on the literature benefits from prompting your LLM assistant to take a contrarian view on each paper individually, to help you identify potential alternative explanations. I think many researchers’ interpretations of their research are pretty suspect, even though the research is going to be reported accurately. There’s a lot of motivated reasoning bias toward interpretations that make the research important.
But that’s a minor quibble. I think your methodology is exemplary for how to use LLMs to speed your research and writing, including prompting them to take a skeptical stance sometimes.
I hope to see more from you and others in this vein. I do think it’s important to say exactly how you used the LLMs; I trust this far more than if someone had just prompted Gemini to write this essay. You are supplying the metacognition, judgment, and executive function that the models currently lack.
I’m sorry this post didn’t wind up with a higher vote total. I think it was really good. I attribute this to suspicion from the community about a post that basically just says “some LLM wrote this” instead of saying how carefully you used combinations of LLMs as research and writing assistants.