For what it’s worth, I cannot confidently distinguish this post from GPT-3 output. Which is to say, each sentence kind of makes sense, but then I get to the end and go ”...what was that trying to say?”
In general I do want LW to be a place where people can brainstorm unfinished ideas at each other, but given the context that, as you mention, “meditation is a bit contentious in parts of the LessWrong community, out of concern for individuals and the wider community”, I feel kinda icky about this particular post. Especially because “sounding more like GPT-3″ is exactly the kind of thing that I am worried prolonged meditation might do to people.
For what it’s worth, as someone with a lot of meditation experience and a longstanding interest in the topic, I didn’t get a GPT-3 vibe at all. To me, the whole thing registered as meaningful communication on a poorly-understood topic, with roughly appropriate levels of tentativeness and epistemic caution.
I’m left wondering if “sounding more like GPT-3” might be a common feature of attempts to communicate across large inferential distances with significant amounts of nonshared referents. How could one distinguish between “there’s no there there” and “there’s a there there but it’s inaccessible from my current vantage point”?
How could one distinguish between “there’s no there there” and “there’s a there there but it’s inaccessible from my current vantage point”?
By having experienced meditators independently rate anonymous articles, some of them written by actual meditators, other written by people who don’t meditate but believe that they can generate the same kind of GPT-3-ish output?
Yeah, that’d work for investigating the hypothesis and is interestingly similar to the theoretical ideal of the peer review process. I was personally more curious about how to distinguish in the general case where reliable domain experts may not be recognizable or readily available but that question may be rationality-complete.
Something like: there are dozen groups of people believing they are experts on meditation, each of them believing the other groups are wrong—how do find the right ones?
In that case, perhaps instead of talking about “meditation”, we could define “meditation1″ as whatever the group1 believes, “meditation2” as whatever the group2 believes… and test independently which groups can be easily fooled by GPT-3.
In my head I call this the “word salad” phenomenon, where one can read something and be like “that’s just word salad...”
I think about this a lot because inferential distance makes “calling word salad” more likely, and it’s maybe especially common in pre-paradigmatic domains.
Honestly, I see it a lot between meditators at all skill/knowledge levels, and, imo, it’s often a good call. Sometimes it’s a good call for part of someone’s body of work and a mis-read for another part.
In general, I see it especially between “same-level” experts or autodidacts (again in pre-paradigmatic domains), as well as between non-experts and experts, in both directions. (The “expert” thinks “word salad” when a non-expert tries to convey something, and/or vice versa.) “Expert,” here, could be replaced by advanced practitioner, unsocialized autodidact, crackpot...
I think the principle of charity is helpful, but also there’s only so much time to evaluate claims.
When I’m evaluating something, I sort of run through a list of referents, concepts, relations, jargon/terms-of-art, equivocation, causality, implication in no particular order.
Have I ever encountered ANY of the referents before, as best I can tell? Are words being used in non-standard ways? Is the “language game” ostensive, assertoric, logical, mechanistic and/or all the above? Do the concepts and relations feel like they’re “sufficiently high quality”?--how blurry are the edges? Are they of relatively small number? How elegant is the thing, overall? Do words change referents? Is referent-switching “doing useful work” or driven by lack of good vocabulary? What’s the degree of causality or mechanism that inheres in the referent, or the degree of implication or argument that inheres in the writing?
Sometimes one has to eject or short-circuit the evaluation process. I use the above questions to do that. But, if I have time or there are outside-view reasons to give something a longer look, I try not to drop it until (a) I have an explanation of the generating process that gave rise to the statements or artifact I’m encountering (what is the sociological/epistemic causal history of this?), or (b) I have a more general explanation for which what I’m encountering is a limit case or edge case.
Because writing and speaking are “correlated with reality,” even if tangled/confused, I think it’s really powerful to “give word salad a chance,” because people are exposed to different patches and trajectories of reality, and, modulo bullshit, it’s never word salad from the inside. I think there’s often net-alpha to be had, for the work put in, when someone is trying to communicate in good faith, and even when not.
(And it can create affordances to correct errors on both sides, can create a feedback loop for dispelling the curse of knowledge, etc., etc., etc.)
But, yeah, sometimes it’s better to disengage or to put up a communal wall.
This post parses to meaning pretty fully for me, but I’m somewhat familiar with Mark’s writings.
In case it helps anyone else, as I read it the key points are:
hypothesis: the mind is highly neuroplastic in the long term, capable of arbitrarily “large” error corrections. Also there are momentary moves it can do to encode more than one piece of information in the same bit of network. Inference: while maybe arbitrary neuroplasticity and error correction is possible in the limit, locally this looks like doing as series of highly-constrained changes like a sliding puzzle. We probably have some particular neural mechanism handling these updates
1b) these updates look like going through changing layers of encodings one at a time, to be more faithful/useful: “deconvolution” was a helpful word for me here. (I’m not quite capturing the meaning of this point but it’s also somewhat hard to without drawing diagrams)
hypothesis: the mind doesn’t have noise in it, only mis-encoded signal—error. (with 1, this makes it possible to “error-correct”, locally and eventually globally with enough repeated local work)
hypothesis: there aren’t “separate representations” for memories, ontologies, models etc: it’s the same kind of network (perception=>action) all the way down with just lots and lots of layers (which contain information abstractions) in the middle. Inference: you can use the same kinds of “moves” to 1b) error-correct the whole thing, eventually. You don’t need different “moves” for different kinds of “stuff”
For what it’s worth, I cannot confidently distinguish this post from GPT-3 output. Which is to say, each sentence kind of makes sense, but then I get to the end and go ”...what was that trying to say?”
In general I do want LW to be a place where people can brainstorm unfinished ideas at each other, but given the context that, as you mention, “meditation is a bit contentious in parts of the LessWrong community, out of concern for individuals and the wider community”, I feel kinda icky about this particular post. Especially because “sounding more like GPT-3″ is exactly the kind of thing that I am worried prolonged meditation might do to people.
For what it’s worth, as someone with a lot of meditation experience and a longstanding interest in the topic, I didn’t get a GPT-3 vibe at all. To me, the whole thing registered as meaningful communication on a poorly-understood topic, with roughly appropriate levels of tentativeness and epistemic caution.
I’m left wondering if “sounding more like GPT-3” might be a common feature of attempts to communicate across large inferential distances with significant amounts of nonshared referents. How could one distinguish between “there’s no there there” and “there’s a there there but it’s inaccessible from my current vantage point”?
By having experienced meditators independently rate anonymous articles, some of them written by actual meditators, other written by people who don’t meditate but believe that they can generate the same kind of GPT-3-ish output?
Meditation Turing Test (MTT) [cf. Ideological Turing Test (ITT)], that’s great. 🤔
Yeah, that’d work for investigating the hypothesis and is interestingly similar to the theoretical ideal of the peer review process.
I was personally more curious about how to distinguish in the general case where reliable domain experts may not be recognizable or readily available but that question may be rationality-complete.
Something like: there are dozen groups of people believing they are experts on meditation, each of them believing the other groups are wrong—how do find the right ones?
In that case, perhaps instead of talking about “meditation”, we could define “meditation1″ as whatever the group1 believes, “meditation2” as whatever the group2 believes… and test independently which groups can be easily fooled by GPT-3.
In my head I call this the “word salad” phenomenon, where one can read something and be like “that’s just word salad...”
I think about this a lot because inferential distance makes “calling word salad” more likely, and it’s maybe especially common in pre-paradigmatic domains.
Honestly, I see it a lot between meditators at all skill/knowledge levels, and, imo, it’s often a good call. Sometimes it’s a good call for part of someone’s body of work and a mis-read for another part.
In general, I see it especially between “same-level” experts or autodidacts (again in pre-paradigmatic domains), as well as between non-experts and experts, in both directions. (The “expert” thinks “word salad” when a non-expert tries to convey something, and/or vice versa.) “Expert,” here, could be replaced by advanced practitioner, unsocialized autodidact, crackpot...
I think the principle of charity is helpful, but also there’s only so much time to evaluate claims.
When I’m evaluating something, I sort of run through a list of referents, concepts, relations, jargon/terms-of-art, equivocation, causality, implication in no particular order.
Have I ever encountered ANY of the referents before, as best I can tell? Are words being used in non-standard ways? Is the “language game” ostensive, assertoric, logical, mechanistic and/or all the above? Do the concepts and relations feel like they’re “sufficiently high quality”?--how blurry are the edges? Are they of relatively small number? How elegant is the thing, overall? Do words change referents? Is referent-switching “doing useful work” or driven by lack of good vocabulary? What’s the degree of causality or mechanism that inheres in the referent, or the degree of implication or argument that inheres in the writing?
Sometimes one has to eject or short-circuit the evaluation process. I use the above questions to do that. But, if I have time or there are outside-view reasons to give something a longer look, I try not to drop it until (a) I have an explanation of the generating process that gave rise to the statements or artifact I’m encountering (what is the sociological/epistemic causal history of this?), or (b) I have a more general explanation for which what I’m encountering is a limit case or edge case.
Because writing and speaking are “correlated with reality,” even if tangled/confused, I think it’s really powerful to “give word salad a chance,” because people are exposed to different patches and trajectories of reality, and, modulo bullshit, it’s never word salad from the inside. I think there’s often net-alpha to be had, for the work put in, when someone is trying to communicate in good faith, and even when not.
(And it can create affordances to correct errors on both sides, can create a feedback loop for dispelling the curse of knowledge, etc., etc., etc.)
But, yeah, sometimes it’s better to disengage or to put up a communal wall.
This post parses to meaning pretty fully for me, but I’m somewhat familiar with Mark’s writings.
In case it helps anyone else, as I read it the key points are:
hypothesis: the mind is highly neuroplastic in the long term, capable of arbitrarily “large” error corrections. Also there are momentary moves it can do to encode more than one piece of information in the same bit of network. Inference: while maybe arbitrary neuroplasticity and error correction is possible in the limit, locally this looks like doing as series of highly-constrained changes like a sliding puzzle. We probably have some particular neural mechanism handling these updates 1b) these updates look like going through changing layers of encodings one at a time, to be more faithful/useful: “deconvolution” was a helpful word for me here. (I’m not quite capturing the meaning of this point but it’s also somewhat hard to without drawing diagrams)
hypothesis: the mind doesn’t have noise in it, only mis-encoded signal—error. (with 1, this makes it possible to “error-correct”, locally and eventually globally with enough repeated local work)
hypothesis: there aren’t “separate representations” for memories, ontologies, models etc: it’s the same kind of network (perception=>action) all the way down with just lots and lots of layers (which contain information abstractions) in the middle. Inference: you can use the same kinds of “moves” to 1b) error-correct the whole thing, eventually. You don’t need different “moves” for different kinds of “stuff”