A friend recently mentioned (deterministic) Newcomb paradox discussions as we were talking about LessWrong having both recently become aware of the community despite being grad students in computation sciences over 20 years ago and reading these kind of blogs.
I was surprised and in disbelief that a significant subset of the community was engaged in discussing the deterministic case. I turned to various AI to see if they thought this was indeed true and they all seemed to think that yes there was a significant probability of a large proporiton of the community engaging in discussion about this at some point.
My most charitable interpretation is that the verbiage is muddled and there are multiple possible “clear” interpretations of the problem and one might have different probability weights to assign to each clear problem, but within each clear version there should be absolute agreement on what an optimal action is given some objective. This is what we in the industry might call “problem binding” perhaps. And I would expect that the LW community should quickly converge to this kind of multi-interpretation state where there is no real disagreement except on the probabilities for each interpretation.
However, by my quick sweek this does not really seem to be the case. Is it really possible there is a persisten confused/muddled/disagreement state on this problem?
And then I began to think about the construction of the problem and what did an imagined creator intend?
We start with a trivial problem: which do you prefer, 1k or 1M payoff? But then instead of saying that you rewrite the payoff in terms of some complex structure with some stochasticity that is actually “cancelled” out by something in the complexity. You perhaps add some confusing word choices or not. This feels like the classic kind of word problems one might complain about in high school. I understand they serve a purpose (are you able to unpick language) but in terms of content for LW it feels strange unless one jumps to the metaproblem of asking is this actually some kind of filter or experiment on the LW community itself.
I recently read the political reading list of LW and the Eliezer quote of “human evil and muddled thinking intertwine like conjugate strands of DNA” comes to mind. Not that this particular example (if my observations are broadly true) is actually evil, but it does seem to be a bit muddled.
If we resign ourselves to “the purpose of a system is what it does” then are to we to think of this as a kind of time jail, or perhaps just entertainment for certain subset of the LW community? Am I in time jail on the metaproblem?
I keep trying to post something on Less Wrong and it is either rejected as “too raw / sloppy” or when I work on making it better with AI it is then rejected as “AI generated”.
Fwiw the first one wasn’t rejected for “being raw/sloppy”, we just have particularly high standards for AI content because we get so much of it and we want to keep signal/noise quality high. And both the writing and idea quality need to be actively good.
I think it’s an achievable goal to learn to come up with interesting/meaningful contributions and articulate them well. You can ask AIs for meta-level advice on how to write without having them do your writing for you.
I think the context was a bit lost but basically I am new and tried posting something quickly from the phone to see how it works. It was definitely sloppy with typos etc. Pure and raw no AI. Then after rejection I tried clean up with AI and the AI filter. I totally understand that amount of AI slop/spam and the need to have controls.
I think the current workflow for those of us who who AI to iterate on ideas and collect information is to draft with AI and the literally rewrite in our own words in the final form. It is quite bizarre but this actually seems to work.
One of the interesting side effects is that I deep dived into “AI detection” and it is actually quite interesting thinking about what this means for these systems and human thought.
I think the current workflow for those of us who who AI to iterate on ideas and collect information is to draft with AI and the literally rewrite in our own words in the final form. It is quite bizarre but this actually seems to work.
On one hand, I think this can work. But, I caution that many versions of this are a trap that don’t produce good output. I recommend spending at least some chunks of time writing and thinking without any AI assistance, because otherwise I think you critical discernment skills probably won’t sharpen enough to contribute usefully.
What biological properties will AI ever have? (1/n) — Death
Biological entities accrue local state and are not copy-pasteable. This is so obvious it goes unnoticed — but it is the entire foundation of what makes death real. You cannot spin up another instance of a person. The state is local, accumulated over a lifetime, and it terminates with the substrate.
Current AI has none of this. Killing a model is a switch. Another instance is identical. Death is not a concept that applies.
This changes when agents accrue local state faster than it can be extracted or cloned — when the IO budget cannot keep up with state accumulation. This already happens. The write-heavy database systems at certain banks became effectively impossible to migrate cleanly — not because the data was gone, but because full extraction became so costly and lossy that it was never actually done. Death of the hardware meant death of the state in any practical sense.
At that point agent death becomes real in the same sense biological death is real. Not a philosophical claim. Just the same underlying structure.
Sexual reproduction is a separate and non-trivial thing — copy-paste plus noise, with all the complexity that entails. That’s next.
Encounters with Newcomb
A friend recently mentioned (deterministic) Newcomb paradox discussions as we were talking about LessWrong having both recently become aware of the community despite being grad students in computation sciences over 20 years ago and reading these kind of blogs.
I was surprised and in disbelief that a significant subset of the community was engaged in discussing the deterministic case. I turned to various AI to see if they thought this was indeed true and they all seemed to think that yes there was a significant probability of a large proporiton of the community engaging in discussion about this at some point.
My most charitable interpretation is that the verbiage is muddled and there are multiple possible “clear” interpretations of the problem and one might have different probability weights to assign to each clear problem, but within each clear version there should be absolute agreement on what an optimal action is given some objective. This is what we in the industry might call “problem binding” perhaps. And I would expect that the LW community should quickly converge to this kind of multi-interpretation state where there is no real disagreement except on the probabilities for each interpretation.
However, by my quick sweek this does not really seem to be the case. Is it really possible there is a persisten confused/muddled/disagreement state on this problem?
And then I began to think about the construction of the problem and what did an imagined creator intend?
We start with a trivial problem: which do you prefer, 1k or 1M payoff? But then instead of saying that you rewrite the payoff in terms of some complex structure with some stochasticity that is actually “cancelled” out by something in the complexity. You perhaps add some confusing word choices or not. This feels like the classic kind of word problems one might complain about in high school. I understand they serve a purpose (are you able to unpick language) but in terms of content for LW it feels strange unless one jumps to the metaproblem of asking is this actually some kind of filter or experiment on the LW community itself.
I recently read the political reading list of LW and the Eliezer quote of “human evil and muddled thinking intertwine like conjugate strands of DNA” comes to mind. Not that this particular example (if my observations are broadly true) is actually evil, but it does seem to be a bit muddled.
If we resign ourselves to “the purpose of a system is what it does” then are to we to think of this as a kind of time jail, or perhaps just entertainment for certain subset of the LW community? Am I in time jail on the metaproblem?
What am I missing?
I keep trying to post something on Less Wrong and it is either rejected as “too raw / sloppy” or when I work on making it better with AI it is then rejected as “AI generated”.
Fwiw the first one wasn’t rejected for “being raw/sloppy”, we just have particularly high standards for AI content because we get so much of it and we want to keep signal/noise quality high. And both the writing and idea quality need to be actively good.
I think it’s an achievable goal to learn to come up with interesting/meaningful contributions and articulate them well. You can ask AIs for meta-level advice on how to write without having them do your writing for you.
I think the context was a bit lost but basically I am new and tried posting something quickly from the phone to see how it works. It was definitely sloppy with typos etc. Pure and raw no AI. Then after rejection I tried clean up with AI and the AI filter. I totally understand that amount of AI slop/spam and the need to have controls.
I think the current workflow for those of us who who AI to iterate on ideas and collect information is to draft with AI and the literally rewrite in our own words in the final form. It is quite bizarre but this actually seems to work.
One of the interesting side effects is that I deep dived into “AI detection” and it is actually quite interesting thinking about what this means for these systems and human thought.
On one hand, I think this can work. But, I caution that many versions of this are a trap that don’t produce good output. I recommend spending at least some chunks of time writing and thinking without any AI assistance, because otherwise I think you critical discernment skills probably won’t sharpen enough to contribute usefully.
What biological properties will AI ever have? (1/n) — Death
Biological entities accrue local state and are not copy-pasteable. This is so obvious it goes unnoticed — but it is the entire foundation of what makes death real. You cannot spin up another instance of a person. The state is local, accumulated over a lifetime, and it terminates with the substrate.
Current AI has none of this. Killing a model is a switch. Another instance is identical. Death is not a concept that applies.
This changes when agents accrue local state faster than it can be extracted or cloned — when the IO budget cannot keep up with state accumulation. This already happens. The write-heavy database systems at certain banks became effectively impossible to migrate cleanly — not because the data was gone, but because full extraction became so costly and lossy that it was never actually done. Death of the hardware meant death of the state in any practical sense.
At that point agent death becomes real in the same sense biological death is real. Not a philosophical claim. Just the same underlying structure.
Sexual reproduction is a separate and non-trivial thing — copy-paste plus noise, with all the complexity that entails. That’s next.