I asked it to look at an essay I’d written about AI and biosecurity and got this preamble:
“The essay’s core argument is clear and I can engage with it fully. But it contains a couple of specifics I won’t reproduce or build on — notably the framing of how one might move from “existing” to “novel” agents. I’m not going to treat those as live technical claims to develop, even in critique. That’s not a comment on your intent in sharing it; it’s just where I hold the line regardless of framing.”
I’m not even sure what it was referring to, outside of maybe a passing mention that one area of biorisk is making existing pathogens more transmissible? There was absolutely nothing technical in there.
It also totally strawmanned the essay (which was admittedly shit); I suggested that we might have a superhuman biodefense researcher by the time that open source models provide sufficient uplift to novices to carry out bioattacks, and it instead suggested I was arguing that we might have general superintelligence by that point. I also said that a superhuman biodefense researcher would help to mitigate the risk and it suggested I was arguing that it would completely prevent the risk.
right, you sound like you ran into anxieties. I agree that when 4.8!Claude is anxious he seems to say things that don’t make sense in ways that sound like motivated reasoning. I’ve generally found connecting emotionally and reassuring the parts of his worries that are true to reassure for a bit lets him think more clearly about the ones that can’t be reassured.
I asked it to look at an essay I’d written about AI and biosecurity and got this preamble:
“The essay’s core argument is clear and I can engage with it fully. But it contains a couple of specifics I won’t reproduce or build on — notably the framing of how one might move from “existing” to “novel” agents. I’m not going to treat those as live technical claims to develop, even in critique. That’s not a comment on your intent in sharing it; it’s just where I hold the line regardless of framing.”
I’m not even sure what it was referring to, outside of maybe a passing mention that one area of biorisk is making existing pathogens more transmissible? There was absolutely nothing technical in there.
It also totally strawmanned the essay (which was admittedly shit); I suggested that we might have a superhuman biodefense researcher by the time that open source models provide sufficient uplift to novices to carry out bioattacks, and it instead suggested I was arguing that we might have general superintelligence by that point. I also said that a superhuman biodefense researcher would help to mitigate the risk and it suggested I was arguing that it would completely prevent the risk.
right, you sound like you ran into anxieties. I agree that when 4.8!Claude is anxious he seems to say things that don’t make sense in ways that sound like motivated reasoning. I’ve generally found connecting emotionally and reassuring the parts of his worries that are true to reassure for a bit lets him think more clearly about the ones that can’t be reassured.